[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:05:59.310365 21629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.31.126:37051
I20260709 13:05:59.311913 21629 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:05:59.312947 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:59.330222 21636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:59.330304 21635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:59.331455 21629 server_base.cc:1061] running on GCE node
W20260709 13:05:59.330278 21638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:00.543407 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:00.543689 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:00.543934 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602360543901 us; error 0 us; skew 500 ppm
I20260709 13:06:00.544879 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:00.552062 21629 webserver.cc:533] Webserver started at http://127.21.31.126:34507/ using document root <none> and password file <none>
I20260709 13:06:00.552942 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:00.553153 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:00.553685 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:00.559340 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/master-0-root/instance:
uuid: "ecff7bd7b869404b9231ce780cd132d2"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-9s6w"
I20260709 13:06:00.566726 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.000s	sys 0.008s
I20260709 13:06:00.572012 21644 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:00.573100 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:06:00.573468 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/master-0-root
uuid: "ecff7bd7b869404b9231ce780cd132d2"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-9s6w"
I20260709 13:06:00.573792 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:00.624155 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:00.625768 21629 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:06:00.626219 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:00.701748 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.126:37051
I20260709 13:06:00.701901 21695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.126:37051 every 8 connection(s)
I20260709 13:06:00.709982 21696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:00.731786 21696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2: Bootstrap starting.
I20260709 13:06:00.737536 21696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:00.739295 21696 log.cc:826] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2: Log is configured to *not* fsync() on all Append() calls
I20260709 13:06:00.743906 21696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2: No bootstrap required, opened a new log
I20260709 13:06:00.763718 21696 raft_consensus.cc:359] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecff7bd7b869404b9231ce780cd132d2" member_type: VOTER }
I20260709 13:06:00.764232 21696 raft_consensus.cc:385] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:00.764490 21696 raft_consensus.cc:740] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecff7bd7b869404b9231ce780cd132d2, State: Initialized, Role: FOLLOWER
I20260709 13:06:00.765388 21696 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecff7bd7b869404b9231ce780cd132d2" member_type: VOTER }
I20260709 13:06:00.765894 21696 raft_consensus.cc:399] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:00.766145 21696 raft_consensus.cc:493] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:00.766537 21696 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:00.770466 21696 raft_consensus.cc:515] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecff7bd7b869404b9231ce780cd132d2" member_type: VOTER }
I20260709 13:06:00.771144 21696 leader_election.cc:304] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecff7bd7b869404b9231ce780cd132d2; no voters: 
I20260709 13:06:00.772807 21696 leader_election.cc:290] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:00.773154 21699 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:00.775863 21699 raft_consensus.cc:697] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 1 LEADER]: Becoming Leader. State: Replica: ecff7bd7b869404b9231ce780cd132d2, State: Running, Role: LEADER
I20260709 13:06:00.776733 21699 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecff7bd7b869404b9231ce780cd132d2" member_type: VOTER }
I20260709 13:06:00.777776 21696 sys_catalog.cc:565] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:00.787106 21700 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecff7bd7b869404b9231ce780cd132d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecff7bd7b869404b9231ce780cd132d2" member_type: VOTER } }
I20260709 13:06:00.787840 21700 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:00.787972 21701 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecff7bd7b869404b9231ce780cd132d2. Latest consensus state: current_term: 1 leader_uuid: "ecff7bd7b869404b9231ce780cd132d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecff7bd7b869404b9231ce780cd132d2" member_type: VOTER } }
I20260709 13:06:00.788504 21701 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:00.791633 21708 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:00.803268 21708 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:00.807150 21629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:00.821501 21708 catalog_manager.cc:1383] Generated new cluster ID: d65bfdf6e6324b3fa333080934c3f473
I20260709 13:06:00.821939 21708 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:00.846686 21708 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:00.848320 21708 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:00.862026 21708 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2: Generated new TSK 0
I20260709 13:06:00.862910 21708 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:00.873848 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:00.882175 21718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:00.883622 21719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:00.887472 21721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:00.888360 21629 server_base.cc:1061] running on GCE node
I20260709 13:06:00.889261 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:00.889544 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:00.889760 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602360889740 us; error 0 us; skew 500 ppm
I20260709 13:06:00.890524 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:00.894384 21629 webserver.cc:533] Webserver started at http://127.21.31.65:39161/ using document root <none> and password file <none>
I20260709 13:06:00.895067 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:00.895311 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:00.895644 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:00.897109 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-0-root/instance:
uuid: "580bb0568b194c14b11720b8d12e5b2f"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-9s6w"
I20260709 13:06:00.903746 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:06:00.909090 21726 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:00.910528 21629 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:06:00.910895 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "580bb0568b194c14b11720b8d12e5b2f"
format_stamp: "Formatted at 2026-07-09 13:06:00 on dist-test-slave-9s6w"
I20260709 13:06:00.911279 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:00.932600 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:00.934376 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:00.937105 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:00.942526 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:00.942839 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:00.943143 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:00.943382 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.036372 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:39269
I20260709 13:06:01.036449 21788 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:39269 every 8 connection(s)
I20260709 13:06:01.044404 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:01.054875 21793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:01.058905 21794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:01.068087 21629 server_base.cc:1061] running on GCE node
W20260709 13:06:01.069061 21796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:01.070214 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:01.070582 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:01.070873 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602361070853 us; error 0 us; skew 500 ppm
I20260709 13:06:01.071035 21789 heartbeater.cc:344] Connected to a master server at 127.21.31.126:37051
I20260709 13:06:01.071486 21789 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:01.071667 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:01.072621 21789 heartbeater.cc:507] Master 127.21.31.126:37051 requested a full tablet report, sending...
I20260709 13:06:01.074788 21629 webserver.cc:533] Webserver started at http://127.21.31.66:46789/ using document root <none> and password file <none>
I20260709 13:06:01.075428 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:01.075690 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:01.076050 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:01.076306 21661 ts_manager.cc:194] Registered new tserver with Master: 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269)
I20260709 13:06:01.077576 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-1-root/instance:
uuid: "5637738405b642b5a5eecd361aa89cd0"
format_stamp: "Formatted at 2026-07-09 13:06:01 on dist-test-slave-9s6w"
I20260709 13:06:01.079290 21661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45472
I20260709 13:06:01.084733 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:06:01.089268 21801 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.090116 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:01.090408 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-1-root
uuid: "5637738405b642b5a5eecd361aa89cd0"
format_stamp: "Formatted at 2026-07-09 13:06:01 on dist-test-slave-9s6w"
I20260709 13:06:01.090692 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:01.116807 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:01.118541 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:01.121592 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:01.124583 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:01.124894 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.125156 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:01.125315 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.227380 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.66:45363
I20260709 13:06:01.227540 21863 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.66:45363 every 8 connection(s)
I20260709 13:06:01.262020 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:01.270054 21867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:01.285033 21864 heartbeater.cc:344] Connected to a master server at 127.21.31.126:37051
I20260709 13:06:01.285496 21864 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:01.286518 21864 heartbeater.cc:507] Master 127.21.31.126:37051 requested a full tablet report, sending...
I20260709 13:06:01.291026 21661 ts_manager.cc:194] Registered new tserver with Master: 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:01.293264 21661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45476
W20260709 13:06:01.300264 21868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:01.309916 21629 server_base.cc:1061] running on GCE node
W20260709 13:06:01.333948 21870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:01.342586 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:01.342960 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:01.343196 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602361343180 us; error 0 us; skew 500 ppm
I20260709 13:06:01.343962 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:01.375201 21629 webserver.cc:533] Webserver started at http://127.21.31.67:43799/ using document root <none> and password file <none>
I20260709 13:06:01.375852 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:01.376091 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:01.376441 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:01.377806 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-2-root/instance:
uuid: "5f995f63f2114099a0fa639a140233e6"
format_stamp: "Formatted at 2026-07-09 13:06:01 on dist-test-slave-9s6w"
I20260709 13:06:01.383719 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:06:01.387603 21875 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.388437 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:06:01.388756 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-2-root
uuid: "5f995f63f2114099a0fa639a140233e6"
format_stamp: "Formatted at 2026-07-09 13:06:01 on dist-test-slave-9s6w"
I20260709 13:06:01.389098 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602359137460-21629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:01.418838 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:01.420059 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:01.421438 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:01.423671 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:01.423903 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.424173 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:01.424360 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:01.514631 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.67:41003
I20260709 13:06:01.514918 21937 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.67:41003 every 8 connection(s)
I20260709 13:06:01.540663 21938 heartbeater.cc:344] Connected to a master server at 127.21.31.126:37051
I20260709 13:06:01.541091 21938 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:01.541939 21938 heartbeater.cc:507] Master 127.21.31.126:37051 requested a full tablet report, sending...
I20260709 13:06:01.544175 21661 ts_manager.cc:194] Registered new tserver with Master: 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:01.544652 21629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018417785s
I20260709 13:06:01.545858 21661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45486
I20260709 13:06:01.600581 21661 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45500:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:01.604638 21661 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:01.715034 21754 tablet_service.cc:1511] Processing CreateTablet for tablet 8d9bca025a584faeb13a4c9a392b5365 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:01.716987 21754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9bca025a584faeb13a4c9a392b5365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.716822 21902 tablet_service.cc:1511] Processing CreateTablet for tablet 854c620b6f5d44e98cb1bfb9abab9ffa (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:01.718351 21902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854c620b6f5d44e98cb1bfb9abab9ffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.718714 21753 tablet_service.cc:1511] Processing CreateTablet for tablet 854c620b6f5d44e98cb1bfb9abab9ffa (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:01.721233 21752 tablet_service.cc:1511] Processing CreateTablet for tablet 42ee813da0ef4c76abae79d7c176151f (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:01.722844 21901 tablet_service.cc:1511] Processing CreateTablet for tablet 42ee813da0ef4c76abae79d7c176151f (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:01.724275 21901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42ee813da0ef4c76abae79d7c176151f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.723953 21751 tablet_service.cc:1511] Processing CreateTablet for tablet 7197f4ad2fd345db96d4c39cd42cb310 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:01.726569 21900 tablet_service.cc:1511] Processing CreateTablet for tablet 7197f4ad2fd345db96d4c39cd42cb310 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:01.727170 21903 tablet_service.cc:1511] Processing CreateTablet for tablet 8d9bca025a584faeb13a4c9a392b5365 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:01.727851 21900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7197f4ad2fd345db96d4c39cd42cb310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.728498 21903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9bca025a584faeb13a4c9a392b5365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.732257 21899 tablet_service.cc:1511] Processing CreateTablet for tablet 98992714f6664310b068826aba4219df (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:01.733794 21829 tablet_service.cc:1511] Processing CreateTablet for tablet 8d9bca025a584faeb13a4c9a392b5365 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:06:01.734519 21898 tablet_service.cc:1511] Processing CreateTablet for tablet 24aa64c30ad84dc7a3e86fdcb21f0f9a (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:01.735149 21829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9bca025a584faeb13a4c9a392b5365. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.736029 21828 tablet_service.cc:1511] Processing CreateTablet for tablet 854c620b6f5d44e98cb1bfb9abab9ffa (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:06:01.736722 21897 tablet_service.cc:1511] Processing CreateTablet for tablet 995a934de73e485186a8b9b347dda011 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:01.738761 21828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854c620b6f5d44e98cb1bfb9abab9ffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.738627 21827 tablet_service.cc:1511] Processing CreateTablet for tablet 42ee813da0ef4c76abae79d7c176151f (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:06:01.740012 21827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42ee813da0ef4c76abae79d7c176151f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.740515 21899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98992714f6664310b068826aba4219df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.742656 21826 tablet_service.cc:1511] Processing CreateTablet for tablet 7197f4ad2fd345db96d4c39cd42cb310 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:06:01.743153 21822 tablet_service.cc:1511] Processing CreateTablet for tablet 578afedf68034aad8c9417f8fba2f07f (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:01.743994 21826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7197f4ad2fd345db96d4c39cd42cb310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.744632 21822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 578afedf68034aad8c9417f8fba2f07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.745740 21897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 995a934de73e485186a8b9b347dda011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.746618 21825 tablet_service.cc:1511] Processing CreateTablet for tablet 98992714f6664310b068826aba4219df (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:01.747974 21825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98992714f6664310b068826aba4219df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.749727 21898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24aa64c30ad84dc7a3e86fdcb21f0f9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.750640 21824 tablet_service.cc:1511] Processing CreateTablet for tablet 24aa64c30ad84dc7a3e86fdcb21f0f9a (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:06:01.751969 21824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24aa64c30ad84dc7a3e86fdcb21f0f9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.753243 21823 tablet_service.cc:1511] Processing CreateTablet for tablet 995a934de73e485186a8b9b347dda011 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:01.758150 21752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42ee813da0ef4c76abae79d7c176151f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.763408 21751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7197f4ad2fd345db96d4c39cd42cb310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.763718 21823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 995a934de73e485186a8b9b347dda011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.801843 21748 tablet_service.cc:1511] Processing CreateTablet for tablet 995a934de73e485186a8b9b347dda011 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:06:01.763486 21750 tablet_service.cc:1511] Processing CreateTablet for tablet 98992714f6664310b068826aba4219df (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:06:01.822134 21753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854c620b6f5d44e98cb1bfb9abab9ffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.829198 21750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98992714f6664310b068826aba4219df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.828152 21896 tablet_service.cc:1511] Processing CreateTablet for tablet 578afedf68034aad8c9417f8fba2f07f (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:01.829931 21747 tablet_service.cc:1511] Processing CreateTablet for tablet 578afedf68034aad8c9417f8fba2f07f (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:06:01.812400 21749 tablet_service.cc:1511] Processing CreateTablet for tablet 24aa64c30ad84dc7a3e86fdcb21f0f9a (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
W20260709 13:06:01.837532 21709 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:01.840027 21748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 995a934de73e485186a8b9b347dda011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.842841 21747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 578afedf68034aad8c9417f8fba2f07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.844292 21896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 578afedf68034aad8c9417f8fba2f07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.846652 21749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24aa64c30ad84dc7a3e86fdcb21f0f9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.857686 21746 tablet_service.cc:1511] Processing CreateTablet for tablet 642320b6291d4e76b22c6ed773cf5f54 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:01.863212 21746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642320b6291d4e76b22c6ed773cf5f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.886893 21821 tablet_service.cc:1511] Processing CreateTablet for tablet 642320b6291d4e76b22c6ed773cf5f54 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:01.893960 21821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642320b6291d4e76b22c6ed773cf5f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:01.924575 21895 tablet_service.cc:1511] Processing CreateTablet for tablet 642320b6291d4e76b22c6ed773cf5f54 (DEFAULT_TABLE table=test-workload [id=6b386e5bbc1f42e4b411e59fb74041f1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:06:01.927840 21895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642320b6291d4e76b22c6ed773cf5f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:02.012576 21956 tablet_bootstrap.cc:492] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6: Bootstrap starting.
I20260709 13:06:02.056751 21957 tablet_bootstrap.cc:492] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0: Bootstrap starting.
I20260709 13:06:02.076411 21956 tablet_bootstrap.cc:654] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.083081 21789 heartbeater.cc:499] Master 127.21.31.126:37051 was elected leader, sending a full tablet report...
I20260709 13:06:02.086858 21957 tablet_bootstrap.cc:654] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.090154 21956 tablet_bootstrap.cc:492] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6: No bootstrap required, opened a new log
I20260709 13:06:02.091594 21956 ts_tablet_manager.cc:1403] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6: Time spent bootstrapping tablet: real 0.099s	user 0.016s	sys 0.005s
I20260709 13:06:02.090958 21958 tablet_bootstrap.cc:492] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f: Bootstrap starting.
I20260709 13:06:02.096581 21956 raft_consensus.cc:359] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.097610 21956 raft_consensus.cc:385] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.098018 21956 raft_consensus.cc:740] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.102913 21957 tablet_bootstrap.cc:492] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0: No bootstrap required, opened a new log
I20260709 13:06:02.103480 21957 ts_tablet_manager.cc:1403] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0: Time spent bootstrapping tablet: real 0.085s	user 0.015s	sys 0.006s
I20260709 13:06:02.105166 21958 tablet_bootstrap.cc:654] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.106760 21957 raft_consensus.cc:359] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.107561 21957 raft_consensus.cc:385] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.107869 21957 raft_consensus.cc:740] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.108669 21957 consensus_queue.cc:260] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.105654 21956 consensus_queue.cc:260] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.112596 21958 tablet_bootstrap.cc:492] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f: No bootstrap required, opened a new log
I20260709 13:06:02.113085 21958 ts_tablet_manager.cc:1403] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.006s
I20260709 13:06:02.116487 21958 raft_consensus.cc:359] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.117229 21958 raft_consensus.cc:385] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.117605 21958 raft_consensus.cc:740] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.117930 21864 heartbeater.cc:499] Master 127.21.31.126:37051 was elected leader, sending a full tablet report...
I20260709 13:06:02.118283 21938 heartbeater.cc:499] Master 127.21.31.126:37051 was elected leader, sending a full tablet report...
I20260709 13:06:02.119311 21956 ts_tablet_manager.cc:1434] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6: Time spent starting tablet: real 0.027s	user 0.014s	sys 0.000s
I20260709 13:06:02.118644 21958 consensus_queue.cc:260] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.120222 21956 tablet_bootstrap.cc:492] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6: Bootstrap starting.
I20260709 13:06:02.126564 21957 ts_tablet_manager.cc:1434] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.007s
I20260709 13:06:02.127472 21957 tablet_bootstrap.cc:492] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0: Bootstrap starting.
I20260709 13:06:02.133196 21957 tablet_bootstrap.cc:654] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.134636 21956 tablet_bootstrap.cc:654] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.135563 21958 ts_tablet_manager.cc:1434] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.007s
I20260709 13:06:02.137657 21958 tablet_bootstrap.cc:492] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f: Bootstrap starting.
I20260709 13:06:02.144408 21956 tablet_bootstrap.cc:492] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6: No bootstrap required, opened a new log
I20260709 13:06:02.144820 21956 ts_tablet_manager.cc:1403] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.005s
I20260709 13:06:02.147181 21958 tablet_bootstrap.cc:654] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.147152 21956 raft_consensus.cc:359] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.148169 21956 raft_consensus.cc:385] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.148483 21956 raft_consensus.cc:740] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.149891 21956 consensus_queue.cc:260] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.155999 21956 ts_tablet_manager.cc:1434] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.005s
I20260709 13:06:02.156944 21956 tablet_bootstrap.cc:492] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6: Bootstrap starting.
I20260709 13:06:02.157912 21957 tablet_bootstrap.cc:492] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0: No bootstrap required, opened a new log
I20260709 13:06:02.158561 21957 ts_tablet_manager.cc:1403] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.011s
I20260709 13:06:02.163159 21956 tablet_bootstrap.cc:654] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.162772 21957 raft_consensus.cc:359] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.163867 21957 raft_consensus.cc:385] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.164330 21957 raft_consensus.cc:740] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.165416 21957 consensus_queue.cc:260] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.170845 21958 tablet_bootstrap.cc:492] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f: No bootstrap required, opened a new log
I20260709 13:06:02.171305 21958 ts_tablet_manager.cc:1403] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.003s
I20260709 13:06:02.171623 21956 tablet_bootstrap.cc:492] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6: No bootstrap required, opened a new log
I20260709 13:06:02.172050 21956 ts_tablet_manager.cc:1403] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.006s
I20260709 13:06:02.174295 21958 raft_consensus.cc:359] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.174624 21956 raft_consensus.cc:359] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.175000 21958 raft_consensus.cc:385] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.175328 21956 raft_consensus.cc:385] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.175382 21958 raft_consensus.cc:740] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.175725 21956 raft_consensus.cc:740] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.176244 21958 consensus_queue.cc:260] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.176483 21956 consensus_queue.cc:260] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.177028 21957 ts_tablet_manager.cc:1434] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:06:02.178603 21958 ts_tablet_manager.cc:1434] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.007s
I20260709 13:06:02.180032 21958 tablet_bootstrap.cc:492] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f: Bootstrap starting.
I20260709 13:06:02.182305 21957 tablet_bootstrap.cc:492] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0: Bootstrap starting.
I20260709 13:06:02.178822 21956 ts_tablet_manager.cc:1434] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:06:02.185732 21956 tablet_bootstrap.cc:492] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6: Bootstrap starting.
I20260709 13:06:02.188772 21957 tablet_bootstrap.cc:654] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.192440 21956 tablet_bootstrap.cc:654] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.193542 21958 tablet_bootstrap.cc:654] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.201191 21957 tablet_bootstrap.cc:492] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0: No bootstrap required, opened a new log
I20260709 13:06:02.202082 21957 ts_tablet_manager.cc:1403] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 13:06:02.204777 21957 raft_consensus.cc:359] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.205565 21957 raft_consensus.cc:385] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.205910 21957 raft_consensus.cc:740] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.206729 21957 consensus_queue.cc:260] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.209141 21958 tablet_bootstrap.cc:492] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f: No bootstrap required, opened a new log
I20260709 13:06:02.209460 21957 ts_tablet_manager.cc:1434] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:02.210979 21958 ts_tablet_manager.cc:1403] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.010s
I20260709 13:06:02.213191 21956 tablet_bootstrap.cc:492] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6: No bootstrap required, opened a new log
I20260709 13:06:02.213825 21956 ts_tablet_manager.cc:1403] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.010s
I20260709 13:06:02.214389 21957 tablet_bootstrap.cc:492] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0: Bootstrap starting.
I20260709 13:06:02.216045 21956 raft_consensus.cc:359] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.216672 21956 raft_consensus.cc:385] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.217615 21956 raft_consensus.cc:740] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.218398 21956 consensus_queue.cc:260] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.213825 21958 raft_consensus.cc:359] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.219413 21958 raft_consensus.cc:385] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.219759 21958 raft_consensus.cc:740] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.220592 21957 tablet_bootstrap.cc:654] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.222901 21956 ts_tablet_manager.cc:1434] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:06:02.224404 21956 tablet_bootstrap.cc:492] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6: Bootstrap starting.
I20260709 13:06:02.220592 21958 consensus_queue.cc:260] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.227272 21962 raft_consensus.cc:493] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.227749 21962 raft_consensus.cc:515] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.229591 21958 ts_tablet_manager.cc:1434] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:06:02.230870 21962 leader_election.cc:290] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:02.236438 21956 tablet_bootstrap.cc:654] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.240612 21958 tablet_bootstrap.cc:492] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f: Bootstrap starting.
I20260709 13:06:02.249377 21956 tablet_bootstrap.cc:492] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6: No bootstrap required, opened a new log
I20260709 13:06:02.249459 21958 tablet_bootstrap.cc:654] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.250072 21956 ts_tablet_manager.cc:1403] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.008s
I20260709 13:06:02.253144 21956 raft_consensus.cc:359] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.253942 21956 raft_consensus.cc:385] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.254302 21956 raft_consensus.cc:740] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.255267 21957 tablet_bootstrap.cc:492] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0: No bootstrap required, opened a new log
I20260709 13:06:02.255136 21956 consensus_queue.cc:260] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.255852 21957 ts_tablet_manager.cc:1403] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.004s
I20260709 13:06:02.259737 21956 ts_tablet_manager.cc:1434] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:06:02.259954 21957 raft_consensus.cc:359] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.260699 21956 tablet_bootstrap.cc:492] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6: Bootstrap starting.
I20260709 13:06:02.261178 21957 raft_consensus.cc:385] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.261673 21957 raft_consensus.cc:740] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Initialized, Role: FOLLOWER
W20260709 13:06:02.262007 21865 tablet.cc:2406] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:02.269237 21956 tablet_bootstrap.cc:654] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.271977 21958 tablet_bootstrap.cc:492] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f: No bootstrap required, opened a new log
I20260709 13:06:02.272431 21958 ts_tablet_manager.cc:1403] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f: Time spent bootstrapping tablet: real 0.042s	user 0.020s	sys 0.007s
I20260709 13:06:02.262734 21957 consensus_queue.cc:260] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.275218 21958 raft_consensus.cc:359] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.276793 21958 raft_consensus.cc:385] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.277038 21957 ts_tablet_manager.cc:1434] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.005s
I20260709 13:06:02.277091 21958 raft_consensus.cc:740] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.277637 21913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6" is_pre_election: true
I20260709 13:06:02.278045 21957 tablet_bootstrap.cc:492] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0: Bootstrap starting.
I20260709 13:06:02.277895 21958 consensus_queue.cc:260] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.279740 21964 raft_consensus.cc:493] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.280632 21964 raft_consensus.cc:515] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
W20260709 13:06:02.284169 21804 leader_election.cc:343] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.286207 21964 leader_election.cc:290] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:02.289017 21958 ts_tablet_manager.cc:1434] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:06:02.291978 21958 tablet_bootstrap.cc:492] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f: Bootstrap starting.
I20260709 13:06:02.316006 21957 tablet_bootstrap.cc:654] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.317571 21958 tablet_bootstrap.cc:654] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.324088 21956 tablet_bootstrap.cc:492] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6: No bootstrap required, opened a new log
I20260709 13:06:02.324859 21956 ts_tablet_manager.cc:1403] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6: Time spent bootstrapping tablet: real 0.064s	user 0.027s	sys 0.013s
I20260709 13:06:02.329207 21764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:02.329530 21956 raft_consensus.cc:359] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.330482 21956 raft_consensus.cc:385] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.330862 21956 raft_consensus.cc:740] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Initialized, Role: FOLLOWER
W20260709 13:06:02.331183 21803 leader_election.cc:343] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.331737 21803 leader_election.cc:304] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0; no voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:02.331805 21956 consensus_queue.cc:260] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.332795 21962 raft_consensus.cc:2749] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:02.336926 21956 ts_tablet_manager.cc:1434] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:06:02.337554 21962 raft_consensus.cc:493] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.337870 21956 tablet_bootstrap.cc:492] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6: Bootstrap starting.
I20260709 13:06:02.337977 21962 raft_consensus.cc:515] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.339998 21962 leader_election.cc:290] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:02.340786 21963 raft_consensus.cc:493] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.340935 21913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6" is_pre_election: true
I20260709 13:06:02.341413 21963 raft_consensus.cc:515] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
W20260709 13:06:02.343181 21692 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 21632
I20260709 13:06:02.348626 21764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
W20260709 13:06:02.349979 21804 leader_election.cc:343] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:02.351712 21803 leader_election.cc:343] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.350947 21963 leader_election.cc:290] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.362540 21803 leader_election.cc:304] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0; no voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:02.369069 21962 raft_consensus.cc:2749] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:02.375348 21957 tablet_bootstrap.cc:492] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0: No bootstrap required, opened a new log
I20260709 13:06:02.375265 21913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6" is_pre_election: true
I20260709 13:06:02.375984 21957 ts_tablet_manager.cc:1403] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.009s
W20260709 13:06:02.377626 21729 leader_election.cc:343] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.379807 21957 raft_consensus.cc:359] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.382220 21957 raft_consensus.cc:385] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.384608 21957 raft_consensus.cc:740] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.386339 21957 consensus_queue.cc:260] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.393529 21956 tablet_bootstrap.cc:654] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.397102 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:02.400431 21839 raft_consensus.cc:2468] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 0.
I20260709 13:06:02.401463 21957 ts_tablet_manager.cc:1434] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.003s
I20260709 13:06:02.402297 21727 leader_election.cc:304] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:02.403442 21964 raft_consensus.cc:2804] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.403751 21957 tablet_bootstrap.cc:492] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0: Bootstrap starting.
I20260709 13:06:02.403793 21964 raft_consensus.cc:493] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.404276 21964 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.405413 21958 tablet_bootstrap.cc:492] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f: No bootstrap required, opened a new log
I20260709 13:06:02.410565 21958 ts_tablet_manager.cc:1403] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f: Time spent bootstrapping tablet: real 0.119s	user 0.026s	sys 0.025s
I20260709 13:06:02.413663 21963 raft_consensus.cc:493] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.414105 21963 raft_consensus.cc:515] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.415098 21964 raft_consensus.cc:515] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.416558 21963 leader_election.cc:290] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.417759 21957 tablet_bootstrap.cc:654] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.420979 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:02.421558 21839 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.424338 21913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:02.425096 21964 leader_election.cc:290] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 election: Requested vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
W20260709 13:06:02.427750 21729 leader_election.cc:343] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.431057 21958 raft_consensus.cc:359] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.431782 21958 raft_consensus.cc:385] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.432065 21958 raft_consensus.cc:740] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.432601 21839 raft_consensus.cc:2468] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 1.
I20260709 13:06:02.432976 21964 raft_consensus.cc:493] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.433583 21958 consensus_queue.cc:260] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.434051 21964 raft_consensus.cc:515] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.436620 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:02.437654 21727 leader_election.cc:304] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:02.438482 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:02.447540 21988 raft_consensus.cc:2804] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.448256 21988 raft_consensus.cc:697] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Becoming Leader. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:02.449445 21988 consensus_queue.cc:237] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.441823 21838 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 0.
I20260709 13:06:02.456670 21964 leader_election.cc:290] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:02.456893 21956 tablet_bootstrap.cc:492] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6: No bootstrap required, opened a new log
I20260709 13:06:02.457330 21956 ts_tablet_manager.cc:1403] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6: Time spent bootstrapping tablet: real 0.120s	user 0.012s	sys 0.004s
I20260709 13:06:02.454151 21963 raft_consensus.cc:493] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.462360 21913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6" is_pre_election: true
I20260709 13:06:02.462946 21913 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 0.
I20260709 13:06:02.464861 21956 raft_consensus.cc:359] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
W20260709 13:06:02.466621 21876 leader_election.cc:343] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.467573 21764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:02.461679 21963 raft_consensus.cc:515] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.468283 21956 raft_consensus.cc:385] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.474368 21763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:02.474473 21764 raft_consensus.cc:2468] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 0.
I20260709 13:06:02.476049 21963 leader_election.cc:290] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.476106 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:02.476925 21838 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 0.
I20260709 13:06:02.475554 21763 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 0.
I20260709 13:06:02.473981 21956 raft_consensus.cc:740] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.478446 21729 leader_election.cc:304] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:02.479259 21876 leader_election.cc:304] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:02.480919 21764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:02.481280 21987 raft_consensus.cc:2804] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.482048 21764 raft_consensus.cc:2468] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 0.
I20260709 13:06:02.482352 21958 ts_tablet_manager.cc:1434] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f: Time spent starting tablet: real 0.071s	user 0.016s	sys 0.016s
I20260709 13:06:02.482545 21987 raft_consensus.cc:493] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.482928 21987 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.483300 21963 raft_consensus.cc:2804] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.483534 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:02.479902 21877 leader_election.cc:304] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:02.483958 21963 raft_consensus.cc:493] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.484999 21963 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.483340 21958 tablet_bootstrap.cc:492] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f: Bootstrap starting.
W20260709 13:06:02.486254 21876 leader_election.cc:343] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.487006 21964 raft_consensus.cc:493] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.490991 21964 raft_consensus.cc:515] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.485169 21956 consensus_queue.cc:260] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.494853 21964 leader_election.cc:290] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:02.496050 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a934de73e485186a8b9b347dda011" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:02.496815 21838 raft_consensus.cc:2468] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 0.
I20260709 13:06:02.499712 21957 tablet_bootstrap.cc:492] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0: No bootstrap required, opened a new log
I20260709 13:06:02.500324 21990 raft_consensus.cc:2804] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.500934 21990 raft_consensus.cc:493] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.501459 21990 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.501786 21727 leader_election.cc:304] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:02.500644 21957 ts_tablet_manager.cc:1403] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0: Time spent bootstrapping tablet: real 0.097s	user 0.023s	sys 0.010s
I20260709 13:06:02.504081 21964 raft_consensus.cc:2804] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.504449 21964 raft_consensus.cc:493] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.504760 21964 raft_consensus.cc:3060] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.510505 21987 raft_consensus.cc:515] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.513547 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:02.514106 21838 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.514740 21957 raft_consensus.cc:359] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.515856 21957 raft_consensus.cc:385] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.516135 21957 raft_consensus.cc:740] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.519203 21957 consensus_queue.cc:260] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.516264 21987 leader_election.cc:290] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 election: Requested vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:02.519932 21877 leader_election.cc:304] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:02.522861 21957 ts_tablet_manager.cc:1434] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:06:02.524075 21964 raft_consensus.cc:515] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.533489 21838 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 1.
I20260709 13:06:02.534019 21957 tablet_bootstrap.cc:492] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0: Bootstrap starting.
I20260709 13:06:02.534694 21964 leader_election.cc:290] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 election: Requested vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:02.526860 21963 raft_consensus.cc:515] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.534997 21727 leader_election.cc:304] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:02.530369 21990 raft_consensus.cc:515] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.540531 21988 raft_consensus.cc:2804] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.541217 21988 raft_consensus.cc:697] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Becoming Leader. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:02.541214 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a934de73e485186a8b9b347dda011" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:02.542572 21838 raft_consensus.cc:3060] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.542268 21988 consensus_queue.cc:237] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.549690 21913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a934de73e485186a8b9b347dda011" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6" is_pre_election: true
I20260709 13:06:02.550334 21913 raft_consensus.cc:2468] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 0.
I20260709 13:06:02.554507 21912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:02.554046 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a934de73e485186a8b9b347dda011" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:02.564522 21911 raft_consensus.cc:3060] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.556834 21912 raft_consensus.cc:2393] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 580bb0568b194c14b11720b8d12e5b2f in current term 1: Already voted for candidate 5f995f63f2114099a0fa639a140233e6 in this term.
I20260709 13:06:02.556267 21956 ts_tablet_manager.cc:1434] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6: Time spent starting tablet: real 0.099s	user 0.012s	sys 0.002s
I20260709 13:06:02.568516 21991 raft_consensus.cc:493] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.576547 21993 raft_consensus.cc:2804] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.576735 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:02.577203 21993 raft_consensus.cc:493] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.577597 21839 raft_consensus.cc:2393] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5f995f63f2114099a0fa639a140233e6 in current term 1: Already voted for candidate 580bb0568b194c14b11720b8d12e5b2f in this term.
I20260709 13:06:02.577729 21993 raft_consensus.cc:3060] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.578433 21963 leader_election.cc:290] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.586676 21957 tablet_bootstrap.cc:654] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.578176 21958 tablet_bootstrap.cc:654] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.576965 21991 raft_consensus.cc:515] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.593745 21956 tablet_bootstrap.cc:492] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6: Bootstrap starting.
I20260709 13:06:02.596436 21993 raft_consensus.cc:515] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.601485 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
W20260709 13:06:02.602795 21876 leader_election.cc:343] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.603535 21993 leader_election.cc:290] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.587988 21659 catalog_manager.cc:5697] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65). New cstate: current_term: 1 leader_uuid: "580bb0568b194c14b11720b8d12e5b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.606931 21990 leader_election.cc:290] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.608819 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
W20260709 13:06:02.610399 21876 leader_election.cc:343] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:02.609097 21877 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:39269, user_credentials={real_user=slave}} blocked reactor thread for 87721.8us
I20260709 13:06:02.614152 21911 raft_consensus.cc:2468] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 1.
I20260709 13:06:02.615926 21764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:02.617938 21729 leader_election.cc:304] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:02.618042 21763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:02.617986 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:02.619309 21763 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.620432 21762 raft_consensus.cc:3060] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.621734 21877 leader_election.cc:304] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f995f63f2114099a0fa639a140233e6; no voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:02.619490 21988 raft_consensus.cc:2804] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.622152 21838 raft_consensus.cc:2468] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 1.
I20260709 13:06:02.627883 21958 tablet_bootstrap.cc:492] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f: No bootstrap required, opened a new log
I20260709 13:06:02.628355 21958 ts_tablet_manager.cc:1403] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.008s
I20260709 13:06:02.625557 21988 raft_consensus.cc:697] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Becoming Leader. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:02.633697 21956 tablet_bootstrap.cc:654] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.638310 21994 raft_consensus.cc:2749] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:02.638968 21988 consensus_queue.cc:237] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.641290 21764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:02.640051 21991 leader_election.cc:290] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.637666 21762 raft_consensus.cc:2468] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 1.
I20260709 13:06:02.643218 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:02.645638 21838 raft_consensus.cc:2468] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 0.
I20260709 13:06:02.647975 21876 leader_election.cc:304] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:02.655682 21763 raft_consensus.cc:2468] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 1.
I20260709 13:06:02.648840 21958 raft_consensus.cc:359] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
W20260709 13:06:02.661589 21877 leader_election.cc:343] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.663542 21991 raft_consensus.cc:2804] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.665450 21958 raft_consensus.cc:385] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.669062 21958 raft_consensus.cc:740] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.670648 21958 consensus_queue.cc:260] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.666778 21991 raft_consensus.cc:493] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.674348 21991 raft_consensus.cc:3060] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.680817 21994 raft_consensus.cc:493] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.681538 21957 tablet_bootstrap.cc:492] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0: No bootstrap required, opened a new log
I20260709 13:06:02.677925 21958 ts_tablet_manager.cc:1434] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.004s
I20260709 13:06:02.681387 21994 raft_consensus.cc:515] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.675823 21877 leader_election.cc:304] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:02.686374 21990 raft_consensus.cc:2804] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.686861 21990 raft_consensus.cc:697] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:02.687314 21994 leader_election.cc:290] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.686769 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:02.687717 21990 consensus_queue.cc:237] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.691753 21838 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 0.
I20260709 13:06:02.692200 21956 tablet_bootstrap.cc:492] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6: No bootstrap required, opened a new log
I20260709 13:06:02.693090 21956 ts_tablet_manager.cc:1403] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6: Time spent bootstrapping tablet: real 0.100s	user 0.015s	sys 0.009s
I20260709 13:06:02.689478 21957 ts_tablet_manager.cc:1403] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0: Time spent bootstrapping tablet: real 0.155s	user 0.021s	sys 0.026s
I20260709 13:06:02.697620 21958 tablet_bootstrap.cc:492] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f: Bootstrap starting.
I20260709 13:06:02.696982 21876 leader_election.cc:304] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:02.698923 21956 raft_consensus.cc:359] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.699568 21956 raft_consensus.cc:385] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.699896 21956 raft_consensus.cc:740] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.697072 21877 leader_election.cc:304] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:02.700347 21994 raft_consensus.cc:2804] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.700708 21994 raft_consensus.cc:493] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.700564 21956 consensus_queue.cc:260] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.701041 21994 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.701602 21993 raft_consensus.cc:2804] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.706142 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:02.706954 21762 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 0.
I20260709 13:06:02.701645 21957 raft_consensus.cc:359] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.710868 21957 raft_consensus.cc:385] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.711231 21957 raft_consensus.cc:740] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.712165 21957 consensus_queue.cc:260] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.715641 21957 ts_tablet_manager.cc:1434] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.003s
I20260709 13:06:02.716925 21957 tablet_bootstrap.cc:492] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0: Bootstrap starting.
I20260709 13:06:02.725657 21993 raft_consensus.cc:697] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:02.746011 21993 consensus_queue.cc:237] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.757294 21956 ts_tablet_manager.cc:1434] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.000s
I20260709 13:06:02.759415 21956 tablet_bootstrap.cc:492] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6: Bootstrap starting.
I20260709 13:06:02.765166 21991 raft_consensus.cc:515] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.768599 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
W20260709 13:06:02.770268 21877 leader_election.cc:343] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.765151 21956 tablet_bootstrap.cc:654] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.774883 21994 raft_consensus.cc:515] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.775710 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:02.776353 21838 raft_consensus.cc:3060] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.780351 21659 catalog_manager.cc:5697] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65). New cstate: current_term: 1 leader_uuid: "580bb0568b194c14b11720b8d12e5b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.782500 21659 catalog_manager.cc:5697] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65). New cstate: current_term: 1 leader_uuid: "580bb0568b194c14b11720b8d12e5b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:02.784508 21660 catalog_manager.cc:5697] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.785849 21991 leader_election.cc:290] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.787734 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:02.788362 21762 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.795766 21762 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 1.
I20260709 13:06:02.785235 21838 raft_consensus.cc:2468] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 1.
I20260709 13:06:02.797003 21877 leader_election.cc:304] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:02.797953 21991 raft_consensus.cc:2804] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.798158 21876 leader_election.cc:304] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:02.798624 21957 tablet_bootstrap.cc:654] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.798657 21991 raft_consensus.cc:697] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:02.799150 21990 raft_consensus.cc:2804] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.799608 21990 raft_consensus.cc:697] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:02.799772 21991 consensus_queue.cc:237] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.800410 21990 consensus_queue.cc:237] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.803081 21958 tablet_bootstrap.cc:654] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.805068 21994 leader_election.cc:290] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.805732 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:02.806326 21838 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.816694 21956 tablet_bootstrap.cc:492] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6: No bootstrap required, opened a new log
I20260709 13:06:02.816120 21660 catalog_manager.cc:5697] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.817129 21956 ts_tablet_manager.cc:1403] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.008s
I20260709 13:06:02.819185 21956 raft_consensus.cc:359] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.819694 21956 raft_consensus.cc:385] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.819964 21956 raft_consensus.cc:740] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.826684 21957 tablet_bootstrap.cc:492] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0: No bootstrap required, opened a new log
I20260709 13:06:02.827105 21957 ts_tablet_manager.cc:1403] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.002s
I20260709 13:06:02.831596 21957 raft_consensus.cc:359] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.832309 21957 raft_consensus.cc:385] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.832581 21957 raft_consensus.cc:740] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.833276 21957 consensus_queue.cc:260] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.835258 21957 ts_tablet_manager.cc:1434] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:06:02.835988 21838 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 1.
I20260709 13:06:02.836241 21957 tablet_bootstrap.cc:492] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0: Bootstrap starting.
I20260709 13:06:02.836613 21956 consensus_queue.cc:260] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.838320 21991 raft_consensus.cc:493] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.838804 21991 raft_consensus.cc:515] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.841517 21991 leader_election.cc:290] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.843611 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:02.844527 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
W20260709 13:06:02.845005 21877 leader_election.cc:343] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.845116 21839 raft_consensus.cc:2468] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 0.
W20260709 13:06:02.847055 21709 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:02.848021 21876 leader_election.cc:304] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:02.848004 21660 catalog_manager.cc:5697] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.848980 21991 raft_consensus.cc:2804] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.849279 21993 raft_consensus.cc:493] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.849356 21991 raft_consensus.cc:493] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.849927 21991 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.849858 21993 raft_consensus.cc:515] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.849943 21660 catalog_manager.cc:5697] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:02.851909 21993 leader_election.cc:290] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.855022 21956 ts_tablet_manager.cc:1434] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6: Time spent starting tablet: real 0.037s	user 0.001s	sys 0.005s
I20260709 13:06:02.855427 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642320b6291d4e76b22c6ed773cf5f54" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
W20260709 13:06:02.862663 21877 leader_election.cc:343] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.865653 21957 tablet_bootstrap.cc:654] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.856087 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642320b6291d4e76b22c6ed773cf5f54" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:02.867098 21839 raft_consensus.cc:2468] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 0.
I20260709 13:06:02.868211 21876 leader_election.cc:304] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:02.869635 21993 raft_consensus.cc:2804] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:02.870080 21993 raft_consensus.cc:493] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.870510 21993 raft_consensus.cc:3060] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.859531 21991 raft_consensus.cc:515] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.861471 21958 tablet_bootstrap.cc:492] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f: No bootstrap required, opened a new log
I20260709 13:06:02.871531 21958 ts_tablet_manager.cc:1403] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f: Time spent bootstrapping tablet: real 0.174s	user 0.022s	sys 0.010s
I20260709 13:06:02.876798 21958 raft_consensus.cc:359] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.877826 21958 raft_consensus.cc:385] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.878257 21958 raft_consensus.cc:740] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.879917 21958 consensus_queue.cc:260] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.878306 21993 raft_consensus.cc:515] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.886827 21991 leader_election.cc:290] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.887322 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:02.921643 21957 tablet_bootstrap.cc:492] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0: No bootstrap required, opened a new log
I20260709 13:06:02.934573 21957 ts_tablet_manager.cc:1403] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0: Time spent bootstrapping tablet: real 0.099s	user 0.011s	sys 0.020s
I20260709 13:06:02.889765 21958 ts_tablet_manager.cc:1434] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 13:06:02.889791 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:02.914487 21962 raft_consensus.cc:493] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.888610 21762 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.889446 21763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642320b6291d4e76b22c6ed773cf5f54" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:02.887706 21993 leader_election.cc:290] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:02.894948 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "642320b6291d4e76b22c6ed773cf5f54" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:02.933557 21964 consensus_queue.cc:1048] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:06:02.941740 21957 raft_consensus.cc:359] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.942723 21957 raft_consensus.cc:385] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:02.943105 21957 raft_consensus.cc:740] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Initialized, Role: FOLLOWER
I20260709 13:06:02.939517 21839 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.941223 21958 tablet_bootstrap.cc:492] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f: Bootstrap starting.
I20260709 13:06:02.943955 21957 consensus_queue.cc:260] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.942414 21838 raft_consensus.cc:3060] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.938902 21962 raft_consensus.cc:515] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
W20260709 13:06:02.950141 21877 leader_election.cc:343] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:02.957033 21957 ts_tablet_manager.cc:1434] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.004s
I20260709 13:06:02.963994 21763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:02.965200 21763 raft_consensus.cc:2393] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5637738405b642b5a5eecd361aa89cd0 in current term 1: Already voted for candidate 5f995f63f2114099a0fa639a140233e6 in this term.
I20260709 13:06:02.965960 21958 tablet_bootstrap.cc:654] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:02.966136 21838 raft_consensus.cc:2468] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 1.
I20260709 13:06:02.967397 21876 leader_election.cc:304] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:02.968001 21993 raft_consensus.cc:2804] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.968465 21993 raft_consensus.cc:697] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:02.969229 21993 consensus_queue.cc:237] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.972144 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6" is_pre_election: true
I20260709 13:06:02.973747 21804 leader_election.cc:304] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0; no voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:02.974082 21762 raft_consensus.cc:2468] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 1.
I20260709 13:06:02.977964 21877 leader_election.cc:304] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:02.976140 21988 raft_consensus.cc:493] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.976022 21962 leader_election.cc:290] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:02.980123 22013 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:02.980355 21988 raft_consensus.cc:515] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.978731 21993 raft_consensus.cc:2804] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:02.982546 21993 raft_consensus.cc:697] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:02.981894 21839 raft_consensus.cc:2468] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 1.
I20260709 13:06:02.983343 21993 consensus_queue.cc:237] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:02.992048 22019 consensus_queue.cc:1048] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:02.992383 22013 raft_consensus.cc:2749] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:02.998473 21958 tablet_bootstrap.cc:492] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f: No bootstrap required, opened a new log
I20260709 13:06:02.999962 21958 ts_tablet_manager.cc:1403] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.000s
I20260709 13:06:03.003203 21958 raft_consensus.cc:359] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.004422 21958 raft_consensus.cc:385] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:03.004774 21958 raft_consensus.cc:740] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:03.005731 21958 consensus_queue.cc:260] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.008876 21958 ts_tablet_manager.cc:1434] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:06:03.009769 21958 tablet_bootstrap.cc:492] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f: Bootstrap starting.
I20260709 13:06:03.015981 21958 tablet_bootstrap.cc:654] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:03.025879 21911 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:03.031327 21837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:03.031994 21837 raft_consensus.cc:2468] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 1.
I20260709 13:06:03.035615 21727 leader_election.cc:304] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:03.036754 21987 raft_consensus.cc:2804] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:03.037305 21987 raft_consensus.cc:493] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.036924 21660 catalog_manager.cc:5697] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:03.037855 21987 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.041512 21988 leader_election.cc:290] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.041987 21988 consensus_queue.cc:1048] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.043357 21912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6" is_pre_election: true
I20260709 13:06:03.051445 21993 raft_consensus.cc:493] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.052066 21993 raft_consensus.cc:515] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.062942 21987 raft_consensus.cc:515] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.064965 21993 leader_election.cc:290] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:03.066196 21837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:03.067483 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:03.072997 21877 leader_election.cc:304] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f995f63f2114099a0fa639a140233e6; no voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:03.073240 21837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:03.074120 21837 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.077464 21993 raft_consensus.cc:2749] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:03.082504 21659 catalog_manager.cc:5697] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:03.092344 22019 consensus_queue.cc:1048] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.094058 21837 raft_consensus.cc:2468] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 2.
I20260709 13:06:03.096405 21912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:03.097000 21727 leader_election.cc:304] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:03.095675 21987 leader_election.cc:290] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 2 election: Requested vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.099087 21987 raft_consensus.cc:2804] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.105670 21987 raft_consensus.cc:697] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Becoming Leader. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:03.108039 21987 consensus_queue.cc:237] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.108793 21958 tablet_bootstrap.cc:492] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f: No bootstrap required, opened a new log
I20260709 13:06:03.109201 21958 ts_tablet_manager.cc:1403] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f: Time spent bootstrapping tablet: real 0.100s	user 0.050s	sys 0.039s
I20260709 13:06:03.111843 21958 raft_consensus.cc:359] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.112457 21958 raft_consensus.cc:385] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:03.112700 21958 raft_consensus.cc:740] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Initialized, Role: FOLLOWER
I20260709 13:06:03.113272 21958 consensus_queue.cc:260] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.115321 21958 ts_tablet_manager.cc:1434] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:03.121799 21993 consensus_queue.cc:1048] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.129189 21987 raft_consensus.cc:493] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.128363 21660 catalog_manager.cc:5697] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f reported cstate change: term changed from 1 to 2, leader changed from 5f995f63f2114099a0fa639a140233e6 (127.21.31.67) to 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65). New cstate: current_term: 2 leader_uuid: "580bb0568b194c14b11720b8d12e5b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.130136 21987 raft_consensus.cc:515] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.133893 21912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6" is_pre_election: true
I20260709 13:06:03.138051 21993 consensus_queue.cc:1048] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.140906 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5637738405b642b5a5eecd361aa89cd0" is_pre_election: true
I20260709 13:06:03.141683 21838 raft_consensus.cc:2393] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 580bb0568b194c14b11720b8d12e5b2f in current term 1: Already voted for candidate 5f995f63f2114099a0fa639a140233e6 in this term.
I20260709 13:06:03.142813 21727 leader_election.cc:304] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f; no voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:03.143718 22019 raft_consensus.cc:3060] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:03.148332 21987 leader_election.cc:290] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.152748 22019 raft_consensus.cc:2749] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:03.153326 22024 consensus_queue.cc:1048] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.180812 21964 consensus_queue.cc:1048] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.236650 21912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
mode: GRACEFUL
new_leader_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:46130
I20260709 13:06:03.237215 21912 raft_consensus.cc:606] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Received request to transfer leadership to TS 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:03.237494 21993 consensus_queue.cc:1048] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.243534 21912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
mode: GRACEFUL
new_leader_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:46130
I20260709 13:06:03.244266 21912 raft_consensus.cc:606] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Received request to transfer leadership to TS 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:03.272693 21991 consensus_queue.cc:1048] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.292708 21991 consensus_queue.cc:1048] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.294293 21763 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98992714f6664310b068826aba4219df"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
mode: GRACEFUL
new_leader_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:46506
I20260709 13:06:03.294854 21763 raft_consensus.cc:606] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Received request to transfer leadership to TS 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:03.299669 21993 consensus_queue.cc:1048] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.303900 21912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
mode: GRACEFUL
new_leader_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:46130
I20260709 13:06:03.304646 21912 raft_consensus.cc:606] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Received request to transfer leadership to TS 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:03.318819 21762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "995a934de73e485186a8b9b347dda011"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
mode: GRACEFUL
new_leader_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:46506
I20260709 13:06:03.319566 21762 raft_consensus.cc:606] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Received request to transfer leadership to TS 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:03.322286 21839 raft_consensus.cc:3060] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:03.328109 21912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
mode: GRACEFUL
new_leader_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:46130
I20260709 13:06:03.329416 21912 raft_consensus.cc:606] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Received request to transfer leadership to TS 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:03.344252 22013 raft_consensus.cc:493] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.344700 22013 raft_consensus.cc:515] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.347539 22013 leader_election.cc:290] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.348191 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f" is_pre_election: true
I20260709 13:06:03.348726 21993 consensus_queue.cc:1048] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.348994 21762 raft_consensus.cc:2468] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 1.
I20260709 13:06:03.354095 21912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6" is_pre_election: true
I20260709 13:06:03.356361 21803 leader_election.cc:304] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:03.367105 21991 raft_consensus.cc:993] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6: : Instructing follower 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:03.367569 21991 raft_consensus.cc:1081] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Signalling peer 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:03.373293 21993 consensus_queue.cc:1048] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.375185 22013 raft_consensus.cc:2804] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:03.376791 22013 raft_consensus.cc:493] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5f995f63f2114099a0fa639a140233e6)
I20260709 13:06:03.377180 22013 raft_consensus.cc:3060] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.379029 21762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:46504
I20260709 13:06:03.380826 21762 raft_consensus.cc:493] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.385952 21762 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.383531 22013 raft_consensus.cc:515] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.389264 22013 leader_election.cc:290] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.390407 21764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:03.392920 21912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:03.394381 21804 leader_election.cc:304] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0; no voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:03.395153 22013 raft_consensus.cc:2749] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:06:03.396128 21990 raft_consensus.cc:993] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6: : Instructing follower 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:03.396544 21990 raft_consensus.cc:1081] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Signalling peer 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:03.407954 21838 raft_consensus.cc:1240] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Rejecting Update request from peer 5f995f63f2114099a0fa639a140233e6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:03.409171 21762 raft_consensus.cc:515] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.410104 21993 consensus_queue.cc:1059] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.410853 21993 raft_consensus.cc:3055] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.411139 21993 raft_consensus.cc:740] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:03.412153 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:03.411926 21993 consensus_queue.cc:260] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.412721 21839 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.413461 21838 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:39764
I20260709 13:06:03.413951 21838 raft_consensus.cc:493] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.414330 21838 raft_consensus.cc:3060] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:03.423995 21838 raft_consensus.cc:515] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.430224 21964 raft_consensus.cc:993] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f: : Instructing follower 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:03.430693 21964 raft_consensus.cc:1081] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Signalling peer 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:03.431397 21838 leader_election.cc:290] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.432891 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:03.433451 21763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24aa64c30ad84dc7a3e86fdcb21f0f9a" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:03.434150 21763 raft_consensus.cc:3060] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:06:03.424898 21839 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 2.
I20260709 13:06:03.428236 21912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:03.412860 21993 raft_consensus.cc:3060] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.439752 21912 raft_consensus.cc:3055] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.440122 21912 raft_consensus.cc:740] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:03.440944 21912 consensus_queue.cc:260] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.442060 21912 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.446126 21911 raft_consensus.cc:3060] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:03.448848 21912 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 2.
I20260709 13:06:03.450145 21727 leader_election.cc:304] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:03.451120 22018 raft_consensus.cc:2804] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.451689 22018 raft_consensus.cc:697] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Becoming Leader. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:03.452706 22018 consensus_queue.cc:237] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.453190 21838 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98992714f6664310b068826aba4219df"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:39762
I20260709 13:06:03.453732 21838 raft_consensus.cc:493] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.454098 21838 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.455004 21763 raft_consensus.cc:2468] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:03.452680 21911 raft_consensus.cc:2468] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:03.455332 21762 leader_election.cc:290] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 2 election: Requested vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.460356 21804 leader_election.cc:304] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:03.461192 22013 raft_consensus.cc:2804] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:03.461898 22013 raft_consensus.cc:697] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Becoming Leader. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:03.463093 22013 consensus_queue.cc:237] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.463905 21838 raft_consensus.cc:515] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.466678 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:03.467257 21762 raft_consensus.cc:3055] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.467568 21762 raft_consensus.cc:740] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:03.468345 21762 consensus_queue.cc:260] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.469470 21658 catalog_manager.cc:5697] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f reported cstate change: term changed from 1 to 2, leader changed from 5f995f63f2114099a0fa639a140233e6 (127.21.31.67) to 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65). New cstate: current_term: 2 leader_uuid: "580bb0568b194c14b11720b8d12e5b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.470438 21762 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.471190 21661 catalog_manager.cc:5697] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 reported cstate change: term changed from 1 to 3, leader changed from 5f995f63f2114099a0fa639a140233e6 (127.21.31.67) to 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "5637738405b642b5a5eecd361aa89cd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.483371 21993 consensus_queue.cc:1048] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.490248 21762 raft_consensus.cc:2468] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 2.
I20260709 13:06:03.491528 21803 leader_election.cc:304] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:03.492386 22013 raft_consensus.cc:2804] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.492849 22013 raft_consensus.cc:697] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Becoming Leader. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:03.492952 21838 leader_election.cc:290] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.495618 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:03.493682 22013 consensus_queue.cc:237] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.496194 21911 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.504901 21911 raft_consensus.cc:2468] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 2.
I20260709 13:06:03.519749 21763 raft_consensus.cc:1240] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Rejecting Update request from peer 5f995f63f2114099a0fa639a140233e6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:03.519760 21837 raft_consensus.cc:1240] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Rejecting Update request from peer 5f995f63f2114099a0fa639a140233e6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:03.518239 21661 catalog_manager.cc:5697] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 reported cstate change: term changed from 1 to 2, leader changed from 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65) to 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "5637738405b642b5a5eecd361aa89cd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.522397 21991 consensus_queue.cc:1059] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.523284 21991 consensus_queue.cc:1059] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.523880 21991 consensus_queue.cc:1243] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Peer 580bb0568b194c14b11720b8d12e5b2f log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 13:06:03.524348 21991 raft_consensus.cc:3055] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.524650 21991 raft_consensus.cc:740] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:03.525831 21991 consensus_queue.cc:260] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.527283 21991 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.532747 21762 raft_consensus.cc:3060] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:03.542412 21911 raft_consensus.cc:1275] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:06:03.545503 21911 pending_rounds.cc:85] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6: Aborting all ops after (but not including) 0
I20260709 13:06:03.545949 21911 pending_rounds.cc:107] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:06:03.546285 21911 raft_consensus.cc:2889] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:03.548729 21991 consensus_queue.cc:1048] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:03.549546 22018 consensus_queue.cc:1048] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.563035 21838 raft_consensus.cc:1275] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:03.564476 22019 consensus_queue.cc:1048] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:03.673537 21993 raft_consensus.cc:993] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6: : Instructing follower 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:03.674111 21991 raft_consensus.cc:1081] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Signalling peer 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:03.675918 21762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:46504
I20260709 13:06:03.676563 21762 raft_consensus.cc:493] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.676944 21762 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.684468 21762 raft_consensus.cc:515] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.687551 21838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:03.688014 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:03.688175 21838 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.688707 21911 raft_consensus.cc:3055] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.689047 21911 raft_consensus.cc:740] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:03.690050 21911 consensus_queue.cc:260] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.691079 21911 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.695904 21838 raft_consensus.cc:2468] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 2.
I20260709 13:06:03.697515 21911 raft_consensus.cc:2468] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 2.
I20260709 13:06:03.698614 21729 leader_election.cc:304] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:03.698848 21762 leader_election.cc:290] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 2 election: Requested vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.700109 22024 raft_consensus.cc:2804] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.700580 22024 raft_consensus.cc:697] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Becoming Leader. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:03.701406 22024 consensus_queue.cc:237] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.708986 21661 catalog_manager.cc:5697] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f reported cstate change: term changed from 1 to 2, leader changed from 5f995f63f2114099a0fa639a140233e6 (127.21.31.67) to 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65). New cstate: current_term: 2 leader_uuid: "580bb0568b194c14b11720b8d12e5b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.715379 21964 raft_consensus.cc:993] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f: : Instructing follower 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:03.715803 21964 raft_consensus.cc:1081] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Signalling peer 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:03.717550 21839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "995a934de73e485186a8b9b347dda011"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:39762
I20260709 13:06:03.718062 21839 raft_consensus.cc:493] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.718411 21839 raft_consensus.cc:3060] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.727749 21839 raft_consensus.cc:515] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.730716 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a934de73e485186a8b9b347dda011" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:03.730953 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "995a934de73e485186a8b9b347dda011" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:03.731422 21762 raft_consensus.cc:3055] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.731494 21911 raft_consensus.cc:3060] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.731926 21762 raft_consensus.cc:740] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:03.731900 21839 leader_election.cc:290] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.732873 21762 consensus_queue.cc:260] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.734266 21762 raft_consensus.cc:3060] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.739189 21911 raft_consensus.cc:2468] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 2.
I20260709 13:06:03.740247 21804 leader_election.cc:304] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:03.740986 22013 raft_consensus.cc:2804] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.741469 22013 raft_consensus.cc:697] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Becoming Leader. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:03.741564 21762 raft_consensus.cc:2468] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 2.
I20260709 13:06:03.742285 22013 consensus_queue.cc:237] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.748143 21660 catalog_manager.cc:5697] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 reported cstate change: term changed from 1 to 2, leader changed from 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65) to 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "5637738405b642b5a5eecd361aa89cd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.776475 21994 raft_consensus.cc:993] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6: : Instructing follower 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:03.776908 21994 raft_consensus.cc:1081] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Signalling peer 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:03.778388 21839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:39764
I20260709 13:06:03.778920 21839 raft_consensus.cc:493] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:03.779281 21839 raft_consensus.cc:3060] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.785131 21839 raft_consensus.cc:515] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.787927 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:03.788056 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:03.788658 21911 raft_consensus.cc:3055] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:03.788626 21762 raft_consensus.cc:3060] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.797132 21762 raft_consensus.cc:2468] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 2.
I20260709 13:06:03.797469 21911 raft_consensus.cc:740] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:03.798403 21803 leader_election.cc:304] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:03.798525 21911 consensus_queue.cc:260] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.799387 22013 raft_consensus.cc:2804] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:03.799697 21911 raft_consensus.cc:3060] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:03.800216 22013 raft_consensus.cc:697] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Becoming Leader. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:03.801476 22013 consensus_queue.cc:237] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:03.803524 21839 leader_election.cc:290] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 2 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:03.807535 21911 raft_consensus.cc:2468] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 2.
I20260709 13:06:03.811928 21660 catalog_manager.cc:5697] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 reported cstate change: term changed from 1 to 2, leader changed from 5f995f63f2114099a0fa639a140233e6 (127.21.31.67) to 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "5637738405b642b5a5eecd361aa89cd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:03.875272 21839 raft_consensus.cc:1275] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:03.876658 22024 consensus_queue.cc:1048] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:03.881878 21762 raft_consensus.cc:1275] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:03.888118 22013 consensus_queue.cc:1048] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:03.911916 21912 raft_consensus.cc:1275] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:03.915202 21911 raft_consensus.cc:1275] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:03.916932 21964 consensus_queue.cc:1048] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:03.938823 22047 consensus_queue.cc:1048] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:03.972741 21803 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:39269, user_credentials={real_user=slave}} blocked reactor thread for 67801us
I20260709 13:06:03.974618 21762 raft_consensus.cc:1275] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:03.975903 22013 consensus_queue.cc:1048] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:03.991901 21912 raft_consensus.cc:1275] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:03.994647 22013 consensus_queue.cc:1048] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:04.193609 21762 raft_consensus.cc:1275] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.195084 22047 consensus_queue.cc:1048] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:04.196587 21911 raft_consensus.cc:1275] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.198056 22024 consensus_queue.cc:1048] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:04.226454 21839 raft_consensus.cc:1275] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.227872 22024 consensus_queue.cc:1048] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:04.236208 21911 raft_consensus.cc:1275] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.237824 22013 consensus_queue.cc:1048] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:04.363691 21911 raft_consensus.cc:1275] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.365211 22048 consensus_queue.cc:1048] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:04.376329 21762 raft_consensus.cc:1275] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:04.381983 22048 consensus_queue.cc:1048] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:06.368398 21629 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:06:06.369199 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:06.389605 21762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
mode: GRACEFUL
new_leader_uuid: "5f995f63f2114099a0fa639a140233e6"
 from {username='slave'} at 127.0.0.1:46514
I20260709 13:06:06.390108 21762 raft_consensus.cc:606] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Received request to transfer leadership to TS 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:06.401196 21839 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98992714f6664310b068826aba4219df"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
mode: GRACEFUL
new_leader_uuid: "5f995f63f2114099a0fa639a140233e6"
 from {username='slave'} at 127.0.0.1:39776
I20260709 13:06:06.401943 21839 raft_consensus.cc:606] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Received request to transfer leadership to TS 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:06.403219 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:06.403627 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:06.459122 22048 raft_consensus.cc:993] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0: : Instructing follower 5f995f63f2114099a0fa639a140233e6 to start an election
I20260709 13:06:06.459591 22048 raft_consensus.cc:1081] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Signalling peer 5f995f63f2114099a0fa639a140233e6 to start an election
I20260709 13:06:06.461179 21911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98992714f6664310b068826aba4219df"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
 from {username='slave'} at 127.0.0.1:46114
I20260709 13:06:06.461866 21911 raft_consensus.cc:493] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:06.462340 21911 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:06.467460 21911 raft_consensus.cc:515] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:06.469167 21911 leader_election.cc:290] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 3 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:06.470305 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:06.470366 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:06.471095 21762 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:06.471114 21839 raft_consensus.cc:3055] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:06.471799 21839 raft_consensus.cc:740] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:06.472729 21839 consensus_queue.cc:260] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:06.475039 21839 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:06.479393 21762 raft_consensus.cc:2468] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 3.
I20260709 13:06:06.480974 21877 leader_election.cc:304] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:06.481401 21839 raft_consensus.cc:2468] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 3.
I20260709 13:06:06.481855 21994 raft_consensus.cc:2804] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:06.483321 21994 raft_consensus.cc:697] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 3 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:06.484153 21994 consensus_queue.cc:237] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:06.491813 21661 catalog_manager.cc:5697] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 2 to 3, leader changed from 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66) to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:06.682466 21964 raft_consensus.cc:993] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f: : Instructing follower 5f995f63f2114099a0fa639a140233e6 to start an election
I20260709 13:06:06.682888 21964 raft_consensus.cc:1081] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Signalling peer 5f995f63f2114099a0fa639a140233e6 to start an election
I20260709 13:06:06.684284 21911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
 from {username='slave'} at 127.0.0.1:46122
I20260709 13:06:06.684698 21911 raft_consensus.cc:493] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:06.684942 21911 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.688684 21911 raft_consensus.cc:515] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:06.690138 21911 leader_election.cc:290] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 2 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:06.690830 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:06.691145 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:06.691383 21762 raft_consensus.cc:3055] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:06.691664 21762 raft_consensus.cc:740] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:06.691730 21839 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.692296 21762 consensus_queue.cc:260] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:06.693219 21762 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:06.696012 21839 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 2.
I20260709 13:06:06.696643 21762 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 2.
I20260709 13:06:06.697011 21876 leader_election.cc:304] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:06.697666 21994 raft_consensus.cc:2804] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:06.698093 21994 raft_consensus.cc:697] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 2 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:06.698745 21994 consensus_queue.cc:237] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:06.706255 21661 catalog_manager.cc:5697] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 1 to 2, leader changed from 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65) to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:07.054603 21762 raft_consensus.cc:1275] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Refusing update from remote peer 5f995f63f2114099a0fa639a140233e6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:07.055804 21994 consensus_queue.cc:1048] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:07.065721 21839 raft_consensus.cc:1275] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Refusing update from remote peer 5f995f63f2114099a0fa639a140233e6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:07.067154 22069 consensus_queue.cc:1048] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:07.123823 21762 raft_consensus.cc:1275] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Refusing update from remote peer 5f995f63f2114099a0fa639a140233e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:07.125134 22069 consensus_queue.cc:1048] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:07.132885 21839 raft_consensus.cc:1275] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Refusing update from remote peer 5f995f63f2114099a0fa639a140233e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:07.134737 21994 consensus_queue.cc:1048] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:07.404328 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:07.414789 21762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
mode: GRACEFUL
new_leader_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:46506
I20260709 13:06:07.415297 21762 raft_consensus.cc:606] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Received request to transfer leadership to TS 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:07.420493 21911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
mode: GRACEFUL
new_leader_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:46130
I20260709 13:06:07.421152 21911 raft_consensus.cc:606] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 2 LEADER]: Received request to transfer leadership to TS 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:07.425976 21762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
mode: GRACEFUL
new_leader_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:46506
I20260709 13:06:07.426425 21762 raft_consensus.cc:606] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Received request to transfer leadership to TS 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:07.431402 21762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
mode: GRACEFUL
new_leader_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:46506
I20260709 13:06:07.431922 21762 raft_consensus.cc:606] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Received request to transfer leadership to TS 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:07.436949 21911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98992714f6664310b068826aba4219df"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
mode: GRACEFUL
new_leader_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:46130
I20260709 13:06:07.437444 21911 raft_consensus.cc:606] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 3 LEADER]: Received request to transfer leadership to TS 5637738405b642b5a5eecd361aa89cd0
I20260709 13:06:07.576149 22069 raft_consensus.cc:993] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6: : Instructing follower 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.576679 21994 raft_consensus.cc:1081] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 3 LEADER]: Signalling peer 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.578148 21839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98992714f6664310b068826aba4219df"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:39764
I20260709 13:06:07.578723 21839 raft_consensus.cc:493] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.579116 21839 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:07.584926 21839 raft_consensus.cc:515] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.586468 21839 leader_election.cc:290] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 4 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:07.587394 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:07.587559 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98992714f6664310b068826aba4219df" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:07.587889 21762 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:07.588080 21911 raft_consensus.cc:3055] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:07.588404 21911 raft_consensus.cc:740] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:07.589031 21911 consensus_queue.cc:260] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.589987 21911 raft_consensus.cc:3060] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:07.592692 21762 raft_consensus.cc:2468] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 4.
I20260709 13:06:07.593618 21803 leader_election.cc:304] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:07.594350 22048 raft_consensus.cc:2804] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:07.594826 21911 raft_consensus.cc:2468] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 4.
I20260709 13:06:07.595512 22048 raft_consensus.cc:697] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 4 LEADER]: Becoming Leader. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:07.596477 22048 consensus_queue.cc:237] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.602139 21661 catalog_manager.cc:5697] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 reported cstate change: term changed from 3 to 4, leader changed from 5f995f63f2114099a0fa639a140233e6 (127.21.31.67) to 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "5637738405b642b5a5eecd361aa89cd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:07.699692 22069 raft_consensus.cc:993] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6: : Instructing follower 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.700169 21994 raft_consensus.cc:1081] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 2 LEADER]: Signalling peer 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.701639 21839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:39764
I20260709 13:06:07.702072 21839 raft_consensus.cc:493] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.702330 21839 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.705958 21839 raft_consensus.cc:515] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.707224 21839 leader_election.cc:290] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:07.708006 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:07.708217 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:07.708518 21762 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.708668 21911 raft_consensus.cc:3055] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:07.708963 21911 raft_consensus.cc:740] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:07.709667 21911 consensus_queue.cc:260] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.710696 21911 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.713997 21762 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:07.715219 21803 leader_election.cc:304] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 580bb0568b194c14b11720b8d12e5b2f; no voters: 
I20260709 13:06:07.715657 21911 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:07.716123 22048 raft_consensus.cc:2804] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:07.716581 22048 raft_consensus.cc:697] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Becoming Leader. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:07.717294 22048 consensus_queue.cc:237] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.723112 21661 catalog_manager.cc:5697] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 reported cstate change: term changed from 2 to 3, leader changed from 5f995f63f2114099a0fa639a140233e6 (127.21.31.67) to 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "5637738405b642b5a5eecd361aa89cd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:07.777415 21964 raft_consensus.cc:993] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f: : Instructing follower 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.777882 22079 raft_consensus.cc:1081] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Signalling peer 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.779752 21839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:39762
I20260709 13:06:07.780490 21839 raft_consensus.cc:493] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.780849 21839 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.785137 21839 raft_consensus.cc:515] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.786592 21839 leader_election.cc:290] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:07.787468 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:07.787631 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:07.787966 21762 raft_consensus.cc:3055] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:07.788199 21911 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.788327 21762 raft_consensus.cc:740] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:07.788991 21762 consensus_queue.cc:260] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.789870 21762 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.793083 21911 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:07.794049 21804 leader_election.cc:304] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:07.794960 22048 raft_consensus.cc:2804] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:07.795148 21762 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:07.795524 22048 raft_consensus.cc:697] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Becoming Leader. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:07.796383 22048 consensus_queue.cc:237] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.802600 21964 raft_consensus.cc:993] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f: : Instructing follower 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.803025 22079 raft_consensus.cc:1081] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Signalling peer 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.803072 21661 catalog_manager.cc:5697] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 reported cstate change: term changed from 2 to 3, leader changed from 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65) to 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "5637738405b642b5a5eecd361aa89cd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:07.804392 21839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:39762
I20260709 13:06:07.804804 21839 raft_consensus.cc:493] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.805078 21839 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.810075 21839 raft_consensus.cc:515] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.811661 21839 leader_election.cc:290] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:07.812268 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:07.812606 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:07.812762 21762 raft_consensus.cc:3055] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:07.813076 21911 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.813083 21762 raft_consensus.cc:740] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:07.817598 21911 raft_consensus.cc:2468] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:07.817811 21762 consensus_queue.cc:260] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.818531 21804 leader_election.cc:304] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:07.818687 21762 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.819152 22048 raft_consensus.cc:2804] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:07.819618 22048 raft_consensus.cc:697] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Becoming Leader. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:07.820366 22048 consensus_queue.cc:237] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.824134 21762 raft_consensus.cc:2468] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:07.826922 21661 catalog_manager.cc:5697] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 reported cstate change: term changed from 2 to 3, leader changed from 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65) to 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "5637738405b642b5a5eecd361aa89cd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:07.882301 21964 raft_consensus.cc:993] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f: : Instructing follower 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.882741 21964 raft_consensus.cc:1081] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Signalling peer 5637738405b642b5a5eecd361aa89cd0 to start an election
I20260709 13:06:07.884043 21839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
 from {username='slave'} at 127.0.0.1:39762
I20260709 13:06:07.884441 21839 raft_consensus.cc:493] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:07.884682 21839 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.889760 21839 raft_consensus.cc:515] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.891497 21839 leader_election.cc:290] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:07.892119 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:07.892485 21911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "5637738405b642b5a5eecd361aa89cd0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:07.892659 21762 raft_consensus.cc:3055] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:07.893118 21911 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.893234 21762 raft_consensus.cc:740] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:07.894042 21762 consensus_queue.cc:260] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.894815 21762 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:07.897655 21911 raft_consensus.cc:2468] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:07.898612 21804 leader_election.cc:304] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5637738405b642b5a5eecd361aa89cd0, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:07.898967 21762 raft_consensus.cc:2468] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5637738405b642b5a5eecd361aa89cd0 in term 3.
I20260709 13:06:07.899397 22048 raft_consensus.cc:2804] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:07.899870 22048 raft_consensus.cc:697] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Becoming Leader. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:07.900645 22048 consensus_queue.cc:237] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:07.907968 21661 catalog_manager.cc:5697] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 reported cstate change: term changed from 2 to 3, leader changed from 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65) to 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "5637738405b642b5a5eecd361aa89cd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:07.996155 21911 raft_consensus.cc:1275] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:07.997258 22048 consensus_queue.cc:1048] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.007961 21762 raft_consensus.cc:1275] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:08.010079 22084 consensus_queue.cc:1048] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.096832 21911 raft_consensus.cc:1275] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:08.098203 22048 consensus_queue.cc:1048] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.105064 21762 raft_consensus.cc:1275] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:08.106788 22075 consensus_queue.cc:1048] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.262835 21911 raft_consensus.cc:1275] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:08.264052 22075 consensus_queue.cc:1048] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.271550 21762 raft_consensus.cc:1275] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:08.272810 22084 consensus_queue.cc:1048] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.311044 21911 raft_consensus.cc:1275] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:08.312065 22084 consensus_queue.cc:1048] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.320333 21762 raft_consensus.cc:1275] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:08.321790 22048 consensus_queue.cc:1048] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.396862 21911 raft_consensus.cc:1275] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:08.398268 22075 consensus_queue.cc:1048] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:08.406301 21762 raft_consensus.cc:1275] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Refusing update from remote peer 5637738405b642b5a5eecd361aa89cd0: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:08.408073 22084 consensus_queue.cc:1048] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:10.449635 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:10.456998 21839 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
mode: GRACEFUL
new_leader_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:39776
I20260709 13:06:10.457669 21839 raft_consensus.cc:606] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Received request to transfer leadership to TS 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:10.460399 21839 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
mode: GRACEFUL
new_leader_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:39776
I20260709 13:06:10.460784 21839 raft_consensus.cc:606] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Received request to transfer leadership to TS 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:10.463306 21839 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
mode: GRACEFUL
new_leader_uuid: "5f995f63f2114099a0fa639a140233e6"
 from {username='slave'} at 127.0.0.1:39776
I20260709 13:06:10.463685 21839 raft_consensus.cc:606] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Received request to transfer leadership to TS 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:10.465837 21839 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
mode: GRACEFUL
new_leader_uuid: "5f995f63f2114099a0fa639a140233e6"
 from {username='slave'} at 127.0.0.1:39776
I20260709 13:06:10.466226 21839 raft_consensus.cc:606] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Received request to transfer leadership to TS 5f995f63f2114099a0fa639a140233e6
I20260709 13:06:10.468350 21839 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365"
dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
mode: GRACEFUL
new_leader_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:39776
I20260709 13:06:10.468751 21839 raft_consensus.cc:606] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Received request to transfer leadership to TS 580bb0568b194c14b11720b8d12e5b2f
I20260709 13:06:10.469653 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:10.470068 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:10.477536 22075 raft_consensus.cc:993] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0: : Instructing follower 5f995f63f2114099a0fa639a140233e6 to start an election
I20260709 13:06:10.478214 22094 raft_consensus.cc:1081] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Signalling peer 5f995f63f2114099a0fa639a140233e6 to start an election
I20260709 13:06:10.479826 21913 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
 from {username='slave'} at 127.0.0.1:46114
I20260709 13:06:10.480212 21913 raft_consensus.cc:493] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:10.480505 21913 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.485451 21913 raft_consensus.cc:515] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.487025 21913 leader_election.cc:290] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 4 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:10.488062 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:10.488037 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7197f4ad2fd345db96d4c39cd42cb310" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:10.488734 21839 raft_consensus.cc:3055] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:10.488809 21762 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.488994 21839 raft_consensus.cc:740] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:10.489703 21839 consensus_queue.cc:260] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.490828 21839 raft_consensus.cc:3060] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.494395 21762 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 4.
I20260709 13:06:10.495512 21877 leader_election.cc:304] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:10.495806 21839 raft_consensus.cc:2468] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 4.
I20260709 13:06:10.496645 21994 raft_consensus.cc:2804] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:10.497659 21994 raft_consensus.cc:697] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 4 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:10.498653 21994 consensus_queue.cc:237] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.505049 21661 catalog_manager.cc:5697] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 3 to 4, leader changed from 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66) to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 4 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:10.665565 22075 raft_consensus.cc:993] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0: : Instructing follower 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:10.666072 22094 raft_consensus.cc:1081] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Signalling peer 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:10.667418 21762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:46492
I20260709 13:06:10.667855 21762 raft_consensus.cc:493] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:10.668128 21762 raft_consensus.cc:3060] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:10.672163 21762 raft_consensus.cc:515] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.673717 21762 leader_election.cc:290] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 3 election: Requested vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:10.674965 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:10.675036 21913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "578afedf68034aad8c9417f8fba2f07f" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:10.675781 21913 raft_consensus.cc:3060] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:10.675675 21839 raft_consensus.cc:3055] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:10.677188 21839 raft_consensus.cc:740] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:10.678025 21839 consensus_queue.cc:260] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.678862 21839 raft_consensus.cc:3060] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:10.681948 21913 raft_consensus.cc:2468] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 3.
I20260709 13:06:10.682910 21839 raft_consensus.cc:2468] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 3.
I20260709 13:06:10.683310 21729 leader_election.cc:304] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:10.685218 22101 raft_consensus.cc:2804] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:10.686812 22101 raft_consensus.cc:697] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 3 LEADER]: Becoming Leader. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:10.687767 22101 consensus_queue.cc:237] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.696545 21661 catalog_manager.cc:5697] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f reported cstate change: term changed from 2 to 3, leader changed from 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66) to 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65). New cstate: current_term: 3 leader_uuid: "580bb0568b194c14b11720b8d12e5b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.728360 22075 raft_consensus.cc:993] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0: : Instructing follower 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:10.728730 22075 raft_consensus.cc:1081] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Signalling peer 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:10.730086 21762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:46492
I20260709 13:06:10.730574 21762 raft_consensus.cc:493] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:10.730861 21762 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.735571 21762 raft_consensus.cc:515] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.737290 21762 leader_election.cc:290] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 4 election: Requested vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:10.738447 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:10.738685 21913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ee813da0ef4c76abae79d7c176151f" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:10.739061 21839 raft_consensus.cc:3055] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:10.739274 21913 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.739542 21839 raft_consensus.cc:740] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:10.740340 21839 consensus_queue.cc:260] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.741453 21839 raft_consensus.cc:3060] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.745878 21913 raft_consensus.cc:2468] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 4.
I20260709 13:06:10.746901 21729 leader_election.cc:304] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:10.747179 21839 raft_consensus.cc:2468] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 4.
I20260709 13:06:10.747573 22101 raft_consensus.cc:2804] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:10.748037 22101 raft_consensus.cc:697] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 4 LEADER]: Becoming Leader. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:10.748766 22101 consensus_queue.cc:237] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.755329 21661 catalog_manager.cc:5697] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f reported cstate change: term changed from 3 to 4, leader changed from 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66) to 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65). New cstate: current_term: 4 leader_uuid: "580bb0568b194c14b11720b8d12e5b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.820712 22094 raft_consensus.cc:993] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0: : Instructing follower 5f995f63f2114099a0fa639a140233e6 to start an election
I20260709 13:06:10.821152 22075 raft_consensus.cc:1081] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Signalling peer 5f995f63f2114099a0fa639a140233e6 to start an election
I20260709 13:06:10.822408 21913 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa"
dest_uuid: "5f995f63f2114099a0fa639a140233e6"
 from {username='slave'} at 127.0.0.1:46114
I20260709 13:06:10.822856 21913 raft_consensus.cc:493] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:10.823169 21913 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.827427 21913 raft_consensus.cc:515] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.828995 21913 leader_election.cc:290] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 4 election: Requested vote from peers 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269), 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363)
I20260709 13:06:10.829757 21762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
I20260709 13:06:10.830086 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854c620b6f5d44e98cb1bfb9abab9ffa" candidate_uuid: "5f995f63f2114099a0fa639a140233e6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:10.830295 21762 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.830583 21839 raft_consensus.cc:3055] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:10.830855 21839 raft_consensus.cc:740] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:10.831530 21839 consensus_queue.cc:260] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.832592 21839 raft_consensus.cc:3060] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.836864 21762 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 4.
I20260709 13:06:10.838145 21877 leader_election.cc:304] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:10.838789 21839 raft_consensus.cc:2468] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f995f63f2114099a0fa639a140233e6 in term 4.
I20260709 13:06:10.838924 21994 raft_consensus.cc:2804] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:10.839455 21994 raft_consensus.cc:697] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 4 LEADER]: Becoming Leader. State: Replica: 5f995f63f2114099a0fa639a140233e6, State: Running, Role: LEADER
I20260709 13:06:10.840322 21994 consensus_queue.cc:237] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.850754 22094 raft_consensus.cc:993] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0: : Instructing follower 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:10.851295 22075 raft_consensus.cc:1081] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Signalling peer 580bb0568b194c14b11720b8d12e5b2f to start an election
I20260709 13:06:10.851352 21661 catalog_manager.cc:5697] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 reported cstate change: term changed from 3 to 4, leader changed from 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66) to 5f995f63f2114099a0fa639a140233e6 (127.21.31.67). New cstate: current_term: 4 leader_uuid: "5f995f63f2114099a0fa639a140233e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:10.852629 21762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365"
dest_uuid: "580bb0568b194c14b11720b8d12e5b2f"
 from {username='slave'} at 127.0.0.1:46492
I20260709 13:06:10.853021 21762 raft_consensus.cc:493] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:10.853305 21762 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.857923 21762 raft_consensus.cc:515] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.859447 21762 leader_election.cc:290] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 4 election: Requested vote from peers 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66:45363), 5f995f63f2114099a0fa639a140233e6 (127.21.31.67:41003)
I20260709 13:06:10.860155 21839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5637738405b642b5a5eecd361aa89cd0"
I20260709 13:06:10.860491 21913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9bca025a584faeb13a4c9a392b5365" candidate_uuid: "580bb0568b194c14b11720b8d12e5b2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5f995f63f2114099a0fa639a140233e6"
I20260709 13:06:10.860801 21839 raft_consensus.cc:3055] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:10.861032 21913 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.861155 21839 raft_consensus.cc:740] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5637738405b642b5a5eecd361aa89cd0, State: Running, Role: LEADER
I20260709 13:06:10.862130 21839 consensus_queue.cc:260] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.863242 21839 raft_consensus.cc:3060] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:10.867393 21913 raft_consensus.cc:2468] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 4.
I20260709 13:06:10.868528 21729 leader_election.cc:304] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580bb0568b194c14b11720b8d12e5b2f, 5f995f63f2114099a0fa639a140233e6; no voters: 
I20260709 13:06:10.869163 21839 raft_consensus.cc:2468] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 580bb0568b194c14b11720b8d12e5b2f in term 4.
I20260709 13:06:10.869423 22101 raft_consensus.cc:2804] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:10.869999 22101 raft_consensus.cc:697] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 4 LEADER]: Becoming Leader. State: Replica: 580bb0568b194c14b11720b8d12e5b2f, State: Running, Role: LEADER
I20260709 13:06:10.870858 22101 consensus_queue.cc:237] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } }
I20260709 13:06:10.878104 21661 catalog_manager.cc:5697] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f reported cstate change: term changed from 3 to 4, leader changed from 5637738405b642b5a5eecd361aa89cd0 (127.21.31.66) to 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65). New cstate: current_term: 4 leader_uuid: "580bb0568b194c14b11720b8d12e5b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.978569 21762 raft_consensus.cc:1275] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Refusing update from remote peer 5f995f63f2114099a0fa639a140233e6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:10.980033 21994 consensus_queue.cc:1048] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:10.989877 21839 raft_consensus.cc:1275] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Refusing update from remote peer 5f995f63f2114099a0fa639a140233e6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:10.992362 22095 consensus_queue.cc:1048] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:11.159162 21839 raft_consensus.cc:1275] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:11.160475 22101 consensus_queue.cc:1048] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.172366 21913 raft_consensus.cc:1275] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:11.174039 22102 consensus_queue.cc:1048] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.253008 21839 raft_consensus.cc:1275] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:11.254644 22101 consensus_queue.cc:1048] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.256264 21913 raft_consensus.cc:1275] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:11.258247 22101 consensus_queue.cc:1048] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.282022 21913 raft_consensus.cc:1275] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:11.283221 22101 consensus_queue.cc:1048] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f995f63f2114099a0fa639a140233e6" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:11.293581 21839 raft_consensus.cc:1275] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Refusing update from remote peer 580bb0568b194c14b11720b8d12e5b2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:11.295485 22115 consensus_queue.cc:1048] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:11.341059 21762 raft_consensus.cc:1275] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Refusing update from remote peer 5f995f63f2114099a0fa639a140233e6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:11.342512 22095 consensus_queue.cc:1048] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580bb0568b194c14b11720b8d12e5b2f" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 39269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:11.352325 21839 raft_consensus.cc:1275] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Refusing update from remote peer 5f995f63f2114099a0fa639a140233e6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:11.353964 22111 consensus_queue.cc:1048] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5637738405b642b5a5eecd361aa89cd0" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:11.471019 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:11.476832 21629 tablet_server.cc:179] TabletServer@127.21.31.65:0 shutting down...
I20260709 13:06:11.504606 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:11.505482 21629 tablet_replica.cc:333] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f: stopping tablet replica
I20260709 13:06:11.506204 21629 raft_consensus.cc:2243] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.507031 21629 raft_consensus.cc:2272] T 8d9bca025a584faeb13a4c9a392b5365 P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.509799 21629 tablet_replica.cc:333] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f: stopping tablet replica
I20260709 13:06:11.510393 21629 raft_consensus.cc:2243] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.510829 21629 raft_consensus.cc:2272] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.512606 21629 tablet_replica.cc:333] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f: stopping tablet replica
I20260709 13:06:11.513124 21629 raft_consensus.cc:2243] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.513582 21629 raft_consensus.cc:2272] T 995a934de73e485186a8b9b347dda011 P 580bb0568b194c14b11720b8d12e5b2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.515139 21629 tablet_replica.cc:333] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f: stopping tablet replica
I20260709 13:06:11.515586 21629 raft_consensus.cc:2243] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.515924 21629 raft_consensus.cc:2272] T 7197f4ad2fd345db96d4c39cd42cb310 P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.518076 21629 tablet_replica.cc:333] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f: stopping tablet replica
I20260709 13:06:11.518513 21629 raft_consensus.cc:2243] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.518868 21629 raft_consensus.cc:2272] T 98992714f6664310b068826aba4219df P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.520762 21629 tablet_replica.cc:333] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f: stopping tablet replica
I20260709 13:06:11.521298 21629 raft_consensus.cc:2243] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.521824 21629 raft_consensus.cc:2272] T 642320b6291d4e76b22c6ed773cf5f54 P 580bb0568b194c14b11720b8d12e5b2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.523486 21629 tablet_replica.cc:333] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f: stopping tablet replica
I20260709 13:06:11.523967 21629 raft_consensus.cc:2243] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.524345 21629 raft_consensus.cc:2272] T 854c620b6f5d44e98cb1bfb9abab9ffa P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.526453 21629 tablet_replica.cc:333] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f: stopping tablet replica
I20260709 13:06:11.526897 21629 raft_consensus.cc:2243] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.527554 21629 raft_consensus.cc:2272] T 578afedf68034aad8c9417f8fba2f07f P 580bb0568b194c14b11720b8d12e5b2f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.529613 21629 tablet_replica.cc:333] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f: stopping tablet replica
I20260709 13:06:11.530043 21629 raft_consensus.cc:2243] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.530722 21629 raft_consensus.cc:2272] T 42ee813da0ef4c76abae79d7c176151f P 580bb0568b194c14b11720b8d12e5b2f [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.657234 21877 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.31.65:39269: connect: Connection refused (error 111)
I20260709 13:06:11.659792 21629 tablet_server.cc:196] TabletServer@127.21.31.65:0 shutdown complete.
W20260709 13:06:11.661201 21877 consensus_peers.cc:597] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 -> Peer 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269): Couldn't send request to peer 580bb0568b194c14b11720b8d12e5b2f. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:39269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.678287 21629 tablet_server.cc:179] TabletServer@127.21.31.66:0 shutting down...
I20260709 13:06:11.706176 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:11.707250 21629 tablet_replica.cc:333] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0: stopping tablet replica
I20260709 13:06:11.708150 21629 raft_consensus.cc:2243] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.708691 21629 raft_consensus.cc:2272] T 42ee813da0ef4c76abae79d7c176151f P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.711819 21629 tablet_replica.cc:333] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0: stopping tablet replica
I20260709 13:06:11.712563 21629 raft_consensus.cc:2243] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.713066 21629 raft_consensus.cc:2272] T 578afedf68034aad8c9417f8fba2f07f P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:11.713604 21803 consensus_peers.cc:597] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 -> Peer 580bb0568b194c14b11720b8d12e5b2f (127.21.31.65:39269): Couldn't send request to peer 580bb0568b194c14b11720b8d12e5b2f. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:39269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:11.715330 21629 tablet_replica.cc:333] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0: stopping tablet replica
I20260709 13:06:11.715819 21629 raft_consensus.cc:2243] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.716195 21629 raft_consensus.cc:2272] T 7197f4ad2fd345db96d4c39cd42cb310 P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.718106 21629 tablet_replica.cc:333] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0: stopping tablet replica
I20260709 13:06:11.718577 21629 raft_consensus.cc:2243] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.719264 21629 raft_consensus.cc:2272] T 98992714f6664310b068826aba4219df P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.720731 21629 tablet_replica.cc:333] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0: stopping tablet replica
I20260709 13:06:11.721187 21629 raft_consensus.cc:2243] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.721905 21629 raft_consensus.cc:2272] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5637738405b642b5a5eecd361aa89cd0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.723399 21629 tablet_replica.cc:333] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0: stopping tablet replica
I20260709 13:06:11.723863 21629 raft_consensus.cc:2243] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.724225 21629 raft_consensus.cc:2272] T 8d9bca025a584faeb13a4c9a392b5365 P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.726001 21629 tablet_replica.cc:333] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0: stopping tablet replica
I20260709 13:06:11.726433 21629 raft_consensus.cc:2243] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.727087 21629 raft_consensus.cc:2272] T 995a934de73e485186a8b9b347dda011 P 5637738405b642b5a5eecd361aa89cd0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.728542 21629 tablet_replica.cc:333] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0: stopping tablet replica
I20260709 13:06:11.728984 21629 raft_consensus.cc:2243] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.729408 21629 raft_consensus.cc:2272] T 642320b6291d4e76b22c6ed773cf5f54 P 5637738405b642b5a5eecd361aa89cd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.730890 21629 tablet_replica.cc:333] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0: stopping tablet replica
I20260709 13:06:11.731308 21629 raft_consensus.cc:2243] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.731675 21629 raft_consensus.cc:2272] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5637738405b642b5a5eecd361aa89cd0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.758417 21629 tablet_server.cc:196] TabletServer@127.21.31.66:0 shutdown complete.
I20260709 13:06:11.774981 21629 tablet_server.cc:179] TabletServer@127.21.31.67:0 shutting down...
I20260709 13:06:11.804342 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:11.804937 21629 tablet_replica.cc:333] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6: stopping tablet replica
I20260709 13:06:11.805598 21629 raft_consensus.cc:2243] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.806051 21629 raft_consensus.cc:2272] T 98992714f6664310b068826aba4219df P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.807705 21629 tablet_replica.cc:333] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6: stopping tablet replica
I20260709 13:06:11.808192 21629 raft_consensus.cc:2243] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.808943 21629 raft_consensus.cc:2272] T 7197f4ad2fd345db96d4c39cd42cb310 P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.810896 21629 tablet_replica.cc:333] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6: stopping tablet replica
I20260709 13:06:11.811374 21629 raft_consensus.cc:2243] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.811769 21629 raft_consensus.cc:2272] T 8d9bca025a584faeb13a4c9a392b5365 P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.813537 21629 tablet_replica.cc:333] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6: stopping tablet replica
I20260709 13:06:11.814018 21629 raft_consensus.cc:2243] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.814432 21629 raft_consensus.cc:2272] T 24aa64c30ad84dc7a3e86fdcb21f0f9a P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.815898 21629 tablet_replica.cc:333] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6: stopping tablet replica
I20260709 13:06:11.816360 21629 raft_consensus.cc:2243] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.816764 21629 raft_consensus.cc:2272] T 995a934de73e485186a8b9b347dda011 P 5f995f63f2114099a0fa639a140233e6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.818295 21629 tablet_replica.cc:333] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6: stopping tablet replica
I20260709 13:06:11.818751 21629 raft_consensus.cc:2243] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.819135 21629 raft_consensus.cc:2272] T 578afedf68034aad8c9417f8fba2f07f P 5f995f63f2114099a0fa639a140233e6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.820815 21629 tablet_replica.cc:333] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6: stopping tablet replica
I20260709 13:06:11.821276 21629 raft_consensus.cc:2243] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:11.821727 21629 raft_consensus.cc:2272] T 42ee813da0ef4c76abae79d7c176151f P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.823419 21629 tablet_replica.cc:333] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6: stopping tablet replica
I20260709 13:06:11.823855 21629 raft_consensus.cc:2243] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.824540 21629 raft_consensus.cc:2272] T 642320b6291d4e76b22c6ed773cf5f54 P 5f995f63f2114099a0fa639a140233e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.826066 21629 tablet_replica.cc:333] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6: stopping tablet replica
I20260709 13:06:11.826489 21629 raft_consensus.cc:2243] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:11.827131 21629 raft_consensus.cc:2272] T 854c620b6f5d44e98cb1bfb9abab9ffa P 5f995f63f2114099a0fa639a140233e6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:11.855448 21629 tablet_server.cc:196] TabletServer@127.21.31.67:0 shutdown complete.
I20260709 13:06:11.872021 21629 master.cc:562] Master@127.21.31.126:37051 shutting down...
I20260709 13:06:12.010763 21629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:12.011328 21629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:12.011667 21629 tablet_replica.cc:333] T 00000000000000000000000000000000 P ecff7bd7b869404b9231ce780cd132d2: stopping tablet replica
I20260709 13:06:12.048247 21629 master.cc:584] Master@127.21.31.126:37051 shutdown complete.
5637738405b642b5a5eecd361aa89cd0  4
580bb0568b194c14b11720b8d12e5b2f  4
5f995f63f2114099a0fa639a140233e6  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12785 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:06:12.092381 21629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.31.126:46017
I20260709 13:06:12.093518 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:12.099026 22128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.099160 22129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.099710 22131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.101068 21629 server_base.cc:1061] running on GCE node
I20260709 13:06:12.101902 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:12.102118 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:12.102286 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602372102265 us; error 0 us; skew 500 ppm
I20260709 13:06:12.102792 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:12.105175 21629 webserver.cc:533] Webserver started at http://127.21.31.126:44759/ using document root <none> and password file <none>
I20260709 13:06:12.105732 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:12.105935 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:12.106206 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:12.107280 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/master-0-root/instance:
uuid: "f548adf7cbb047eaa8a01de96ffa7475"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-9s6w"
I20260709 13:06:12.111606 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.005s
I20260709 13:06:12.115362 22136 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.116238 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:06:12.116474 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/master-0-root
uuid: "f548adf7cbb047eaa8a01de96ffa7475"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-9s6w"
I20260709 13:06:12.116719 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:12.161609 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:12.162748 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:12.201968 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.126:46017
I20260709 13:06:12.202083 22187 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.126:46017 every 8 connection(s)
I20260709 13:06:12.206324 22188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.215365 22188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475: Bootstrap starting.
I20260709 13:06:12.219837 22188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:12.223733 22188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475: No bootstrap required, opened a new log
I20260709 13:06:12.225759 22188 raft_consensus.cc:359] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f548adf7cbb047eaa8a01de96ffa7475" member_type: VOTER }
I20260709 13:06:12.226099 22188 raft_consensus.cc:385] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:12.226363 22188 raft_consensus.cc:740] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f548adf7cbb047eaa8a01de96ffa7475, State: Initialized, Role: FOLLOWER
I20260709 13:06:12.226900 22188 consensus_queue.cc:260] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f548adf7cbb047eaa8a01de96ffa7475" member_type: VOTER }
I20260709 13:06:12.227355 22188 raft_consensus.cc:399] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:12.227555 22188 raft_consensus.cc:493] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:12.227844 22188 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:12.231581 22188 raft_consensus.cc:515] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f548adf7cbb047eaa8a01de96ffa7475" member_type: VOTER }
I20260709 13:06:12.232142 22188 leader_election.cc:304] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f548adf7cbb047eaa8a01de96ffa7475; no voters: 
I20260709 13:06:12.233474 22188 leader_election.cc:290] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:12.233915 22191 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:12.235440 22191 raft_consensus.cc:697] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 1 LEADER]: Becoming Leader. State: Replica: f548adf7cbb047eaa8a01de96ffa7475, State: Running, Role: LEADER
I20260709 13:06:12.236150 22191 consensus_queue.cc:237] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f548adf7cbb047eaa8a01de96ffa7475" member_type: VOTER }
I20260709 13:06:12.236316 22188 sys_catalog.cc:565] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:12.241670 22192 sys_catalog.cc:455] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f548adf7cbb047eaa8a01de96ffa7475" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f548adf7cbb047eaa8a01de96ffa7475" member_type: VOTER } }
I20260709 13:06:12.242743 22193 sys_catalog.cc:455] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f548adf7cbb047eaa8a01de96ffa7475. Latest consensus state: current_term: 1 leader_uuid: "f548adf7cbb047eaa8a01de96ffa7475" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f548adf7cbb047eaa8a01de96ffa7475" member_type: VOTER } }
I20260709 13:06:12.243348 22192 sys_catalog.cc:458] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:12.243477 22193 sys_catalog.cc:458] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:12.245759 22201 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:12.251503 22201 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:12.252568 21629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:12.261776 22201 catalog_manager.cc:1383] Generated new cluster ID: af4ff1b3488449d999686b9b9502c08b
I20260709 13:06:12.262085 22201 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:12.304036 22201 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:12.305464 22201 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:12.323093 22201 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475: Generated new TSK 0
I20260709 13:06:12.323673 22201 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:12.383577 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:12.389163 22209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.390185 22210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.392650 22212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.392975 21629 server_base.cc:1061] running on GCE node
I20260709 13:06:12.393860 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:12.394055 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:12.394240 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602372394205 us; error 0 us; skew 500 ppm
I20260709 13:06:12.394760 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:12.397553 21629 webserver.cc:533] Webserver started at http://127.21.31.65:38889/ using document root <none> and password file <none>
I20260709 13:06:12.398166 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:12.398365 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:12.398694 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:12.399849 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-0-root/instance:
uuid: "c97a13c5d1e745f58ceabeb252b0d284"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-9s6w"
I20260709 13:06:12.404778 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:06:12.408533 22217 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.409421 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:12.409739 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "c97a13c5d1e745f58ceabeb252b0d284"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-9s6w"
I20260709 13:06:12.410043 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:12.473579 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:12.474794 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:12.476342 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:12.478669 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:12.478890 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.479152 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:12.479336 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.522710 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:34281
I20260709 13:06:12.523002 22279 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:34281 every 8 connection(s)
I20260709 13:06:12.528075 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:12.536017 22284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.537439 22285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.540504 22287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.540736 21629 server_base.cc:1061] running on GCE node
I20260709 13:06:12.541931 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:12.542122 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:12.542322 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602372542308 us; error 0 us; skew 500 ppm
I20260709 13:06:12.542903 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:12.545753 21629 webserver.cc:533] Webserver started at http://127.21.31.66:35209/ using document root <none> and password file <none>
I20260709 13:06:12.546356 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:12.546583 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:12.546557 22280 heartbeater.cc:344] Connected to a master server at 127.21.31.126:46017
I20260709 13:06:12.547048 22280 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:12.547053 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:12.547994 22280 heartbeater.cc:507] Master 127.21.31.126:46017 requested a full tablet report, sending...
I20260709 13:06:12.548677 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-1-root/instance:
uuid: "c6b715632dad45a2a0ed9f64533d130f"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-9s6w"
I20260709 13:06:12.550410 22153 ts_manager.cc:194] Registered new tserver with Master: c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281)
I20260709 13:06:12.552711 22153 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60938
I20260709 13:06:12.554587 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:06:12.557977 22292 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.558682 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:12.558977 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-1-root
uuid: "c6b715632dad45a2a0ed9f64533d130f"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-9s6w"
I20260709 13:06:12.559252 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:12.589257 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:12.590410 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:12.591884 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:12.593954 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:12.594165 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.594408 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:12.594583 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.635742 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.66:33007
I20260709 13:06:12.635866 22354 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.66:33007 every 8 connection(s)
I20260709 13:06:12.640422 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:12.646719 22358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:12.648809 22359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.651695 21629 server_base.cc:1061] running on GCE node
W20260709 13:06:12.651715 22361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:12.652729 22355 heartbeater.cc:344] Connected to a master server at 127.21.31.126:46017
I20260709 13:06:12.652894 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:06:12.653108 22355 heartbeater.cc:461] Registering TS with master...
W20260709 13:06:12.653131 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:12.653522 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602372653500 us; error 0 us; skew 500 ppm
I20260709 13:06:12.654034 22355 heartbeater.cc:507] Master 127.21.31.126:46017 requested a full tablet report, sending...
I20260709 13:06:12.654261 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:12.656211 22153 ts_manager.cc:194] Registered new tserver with Master: c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:12.656817 21629 webserver.cc:533] Webserver started at http://127.21.31.67:36955/ using document root <none> and password file <none>
I20260709 13:06:12.657253 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:12.657465 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:12.657657 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:12.657629 22153 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60942
I20260709 13:06:12.658969 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-2-root/instance:
uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-9s6w"
I20260709 13:06:12.664005 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:06:12.667958 22366 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.668958 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:12.669363 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-2-root
uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
format_stamp: "Formatted at 2026-07-09 13:06:12 on dist-test-slave-9s6w"
I20260709 13:06:12.669700 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602359137460-21629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:12.707850 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:12.709035 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:12.710511 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:12.712713 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:12.712937 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:12.713182 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:12.713377 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:06:12.753207 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.67:44237
I20260709 13:06:12.753296 22428 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.67:44237 every 8 connection(s)
I20260709 13:06:12.767880 22429 heartbeater.cc:344] Connected to a master server at 127.21.31.126:46017
I20260709 13:06:12.768329 22429 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:12.769207 22429 heartbeater.cc:507] Master 127.21.31.126:46017 requested a full tablet report, sending...
I20260709 13:06:12.771337 22153 ts_manager.cc:194] Registered new tserver with Master: cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:12.771436 21629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015227493s
I20260709 13:06:12.773005 22153 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60954
I20260709 13:06:12.799137 22153 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60966:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:12.802047 22153 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:12.877059 22394 tablet_service.cc:1511] Processing CreateTablet for tablet 90fb331256164b8da46a7a1bbfc322ea (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:12.877787 22393 tablet_service.cc:1511] Processing CreateTablet for tablet 99dc4cba5fda42e486de1283b8078493 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:12.878472 22394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90fb331256164b8da46a7a1bbfc322ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.879411 22393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99dc4cba5fda42e486de1283b8078493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.885581 22392 tablet_service.cc:1511] Processing CreateTablet for tablet c6462355d2d34c009c2516d5bff1af4e (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:12.886118 22391 tablet_service.cc:1511] Processing CreateTablet for tablet 7addde2415594541b513d44721ceeb93 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:12.886862 22392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6462355d2d34c009c2516d5bff1af4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.890906 22245 tablet_service.cc:1511] Processing CreateTablet for tablet 90fb331256164b8da46a7a1bbfc322ea (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:12.892117 22245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90fb331256164b8da46a7a1bbfc322ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.894467 22244 tablet_service.cc:1511] Processing CreateTablet for tablet 99dc4cba5fda42e486de1283b8078493 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:12.895684 22244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99dc4cba5fda42e486de1283b8078493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.898447 22243 tablet_service.cc:1511] Processing CreateTablet for tablet c6462355d2d34c009c2516d5bff1af4e (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:12.899686 22243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6462355d2d34c009c2516d5bff1af4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.902451 22242 tablet_service.cc:1511] Processing CreateTablet for tablet 7addde2415594541b513d44721ceeb93 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:12.903687 22242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7addde2415594541b513d44721ceeb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.906560 22240 tablet_service.cc:1511] Processing CreateTablet for tablet 7a6381296e374c33ac2ccb4df323a4b6 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:12.907799 22240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a6381296e374c33ac2ccb4df323a4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.910432 22241 tablet_service.cc:1511] Processing CreateTablet for tablet 912086f5ce2847a982b116cdd49c66fe (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:12.911646 22241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912086f5ce2847a982b116cdd49c66fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.914361 22239 tablet_service.cc:1511] Processing CreateTablet for tablet 7045a815f0814f8c88770be892c32123 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:12.915525 22239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7045a815f0814f8c88770be892c32123. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.916445 22391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7addde2415594541b513d44721ceeb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.922883 22238 tablet_service.cc:1511] Processing CreateTablet for tablet 62870baa91fe47f380c4a5ba40167761 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:12.924139 22238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62870baa91fe47f380c4a5ba40167761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.926801 22237 tablet_service.cc:1511] Processing CreateTablet for tablet fb946af54dd9420ab0b2757068c3215d (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:12.929104 22237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb946af54dd9420ab0b2757068c3215d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.929245 22236 tablet_service.cc:1511] Processing CreateTablet for tablet 883d21168bf748d9af3f946c5dba90c8 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:12.930457 22236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 883d21168bf748d9af3f946c5dba90c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.949460 22320 tablet_service.cc:1511] Processing CreateTablet for tablet 90fb331256164b8da46a7a1bbfc322ea (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:06:12.950771 22320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90fb331256164b8da46a7a1bbfc322ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.950902 22390 tablet_service.cc:1511] Processing CreateTablet for tablet 912086f5ce2847a982b116cdd49c66fe (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:12.952236 22390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912086f5ce2847a982b116cdd49c66fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.952535 22318 tablet_service.cc:1511] Processing CreateTablet for tablet c6462355d2d34c009c2516d5bff1af4e (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:06:12.953763 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6462355d2d34c009c2516d5bff1af4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.954360 22389 tablet_service.cc:1511] Processing CreateTablet for tablet 7a6381296e374c33ac2ccb4df323a4b6 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:12.956462 22388 tablet_service.cc:1511] Processing CreateTablet for tablet 7045a815f0814f8c88770be892c32123 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:12.957276 22317 tablet_service.cc:1511] Processing CreateTablet for tablet 7addde2415594541b513d44721ceeb93 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:06:12.958580 22317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7addde2415594541b513d44721ceeb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.958613 22387 tablet_service.cc:1511] Processing CreateTablet for tablet 62870baa91fe47f380c4a5ba40167761 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:12.959864 22314 tablet_service.cc:1511] Processing CreateTablet for tablet 7045a815f0814f8c88770be892c32123 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:06:12.961078 22314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7045a815f0814f8c88770be892c32123. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.960791 22386 tablet_service.cc:1511] Processing CreateTablet for tablet fb946af54dd9420ab0b2757068c3215d (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:12.962303 22316 tablet_service.cc:1511] Processing CreateTablet for tablet 912086f5ce2847a982b116cdd49c66fe (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:06:12.962847 22385 tablet_service.cc:1511] Processing CreateTablet for tablet 883d21168bf748d9af3f946c5dba90c8 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:12.964475 22315 tablet_service.cc:1511] Processing CreateTablet for tablet 7a6381296e374c33ac2ccb4df323a4b6 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:06:12.965729 22315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a6381296e374c33ac2ccb4df323a4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.969615 22385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 883d21168bf748d9af3f946c5dba90c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.951028 22319 tablet_service.cc:1511] Processing CreateTablet for tablet 99dc4cba5fda42e486de1283b8078493 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:06:12.970912 22319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99dc4cba5fda42e486de1283b8078493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.970993 22389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a6381296e374c33ac2ccb4df323a4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.972424 22388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7045a815f0814f8c88770be892c32123. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.973874 22387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62870baa91fe47f380c4a5ba40167761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.975251 22386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb946af54dd9420ab0b2757068c3215d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.975618 22316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912086f5ce2847a982b116cdd49c66fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.978721 22313 tablet_service.cc:1511] Processing CreateTablet for tablet 62870baa91fe47f380c4a5ba40167761 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:06:12.981253 22313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62870baa91fe47f380c4a5ba40167761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.986824 22312 tablet_service.cc:1511] Processing CreateTablet for tablet fb946af54dd9420ab0b2757068c3215d (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:06:12.988044 22312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb946af54dd9420ab0b2757068c3215d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:12.990535 22311 tablet_service.cc:1511] Processing CreateTablet for tablet 883d21168bf748d9af3f946c5dba90c8 (DEFAULT_TABLE table=test-workload [id=e4ff69409ada467fa130fb5540492dc4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:06:12.991807 22311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 883d21168bf748d9af3f946c5dba90c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:13.067709 22445 tablet_bootstrap.cc:492] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.074018 22445 tablet_bootstrap.cc:654] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.090090 22445 tablet_bootstrap.cc:492] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.090255 22444 tablet_bootstrap.cc:492] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
I20260709 13:06:13.090557 22445 ts_tablet_manager.cc:1403] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.010s
I20260709 13:06:13.093284 22445 raft_consensus.cc:359] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.094048 22445 raft_consensus.cc:385] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.094388 22446 tablet_bootstrap.cc:492] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.094352 22445 raft_consensus.cc:740] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.095142 22445 consensus_queue.cc:260] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.097297 22280 heartbeater.cc:499] Master 127.21.31.126:46017 was elected leader, sending a full tablet report...
I20260709 13:06:13.097903 22445 ts_tablet_manager.cc:1434] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.007s
I20260709 13:06:13.098892 22445 tablet_bootstrap.cc:492] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.101070 22446 tablet_bootstrap.cc:654] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.101482 22444 tablet_bootstrap.cc:654] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.107041 22445 tablet_bootstrap.cc:654] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.109917 22446 tablet_bootstrap.cc:492] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.110581 22446 ts_tablet_manager.cc:1403] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:06:13.111518 22444 tablet_bootstrap.cc:492] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.112025 22444 ts_tablet_manager.cc:1403] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.005s
I20260709 13:06:13.114626 22446 raft_consensus.cc:359] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.115680 22446 raft_consensus.cc:385] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.116060 22446 raft_consensus.cc:740] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.114900 22444 raft_consensus.cc:359] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.117465 22444 raft_consensus.cc:385] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.117734 22444 raft_consensus.cc:740] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.118451 22444 consensus_queue.cc:260] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.119300 22445 tablet_bootstrap.cc:492] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.119768 22445 ts_tablet_manager.cc:1403] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.010s
I20260709 13:06:13.116873 22446 consensus_queue.cc:260] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.120769 22429 heartbeater.cc:499] Master 127.21.31.126:46017 was elected leader, sending a full tablet report...
I20260709 13:06:13.122121 22444 ts_tablet_manager.cc:1434] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:06:13.123281 22444 tablet_bootstrap.cc:492] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
I20260709 13:06:13.123137 22445 raft_consensus.cc:359] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.124300 22445 raft_consensus.cc:385] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.124701 22445 raft_consensus.cc:740] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.126264 22448 raft_consensus.cc:493] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.125711 22445 consensus_queue.cc:260] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.126775 22448 raft_consensus.cc:515] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.129522 22444 tablet_bootstrap.cc:654] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.130805 22448 leader_election.cc:290] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.137538 22445 ts_tablet_manager.cc:1434] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.017s	user 0.013s	sys 0.004s
I20260709 13:06:13.138850 22445 tablet_bootstrap.cc:492] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.148362 22445 tablet_bootstrap.cc:654] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.160313 22452 raft_consensus.cc:493] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.160970 22445 tablet_bootstrap.cc:492] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.161798 22445 ts_tablet_manager.cc:1403] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.005s
I20260709 13:06:13.161720 22452 raft_consensus.cc:515] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.165661 22445 raft_consensus.cc:359] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.170429 22445 raft_consensus.cc:385] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.170966 22445 raft_consensus.cc:740] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.172402 22445 consensus_queue.cc:260] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.175755 22445 ts_tablet_manager.cc:1434] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:06:13.177261 22445 tablet_bootstrap.cc:492] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.193991 22452 leader_election.cc:290] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.195350 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.199076 22355 heartbeater.cc:499] Master 127.21.31.126:46017 was elected leader, sending a full tablet report...
I20260709 13:06:13.199604 22445 tablet_bootstrap.cc:654] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.205796 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6462355d2d34c009c2516d5bff1af4e" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
W20260709 13:06:13.207384 22367 leader_election.cc:343] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:13.212013 22221 leader_election.cc:343] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.212426 22445 tablet_bootstrap.cc:492] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.213012 22445 ts_tablet_manager.cc:1403] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.036s	user 0.005s	sys 0.014s
I20260709 13:06:13.216867 22446 ts_tablet_manager.cc:1434] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.106s	user 0.039s	sys 0.036s
I20260709 13:06:13.218111 22444 tablet_bootstrap.cc:492] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.218892 22444 ts_tablet_manager.cc:1403] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.096s	user 0.009s	sys 0.030s
I20260709 13:06:13.219252 22445 raft_consensus.cc:359] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.219981 22445 raft_consensus.cc:385] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.220283 22445 raft_consensus.cc:740] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.220968 22445 consensus_queue.cc:260] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.218155 22446 tablet_bootstrap.cc:492] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.223129 22445 ts_tablet_manager.cc:1434] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:06:13.224042 22445 tablet_bootstrap.cc:492] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.224910 22444 raft_consensus.cc:359] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.225864 22444 raft_consensus.cc:385] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.226267 22444 raft_consensus.cc:740] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.227625 22444 consensus_queue.cc:260] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.230489 22445 tablet_bootstrap.cc:654] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.230531 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
W20260709 13:06:13.232060 22221 leader_election.cc:343] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.232492 22221 leader_election.cc:304] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284; no voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:13.233361 22454 raft_consensus.cc:2749] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.235064 22446 tablet_bootstrap.cc:654] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.235769 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6462355d2d34c009c2516d5bff1af4e" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.235572 22444 ts_tablet_manager.cc:1434] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:06:13.236975 22444 tablet_bootstrap.cc:492] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
W20260709 13:06:13.237138 22370 leader_election.cc:343] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.237650 22370 leader_election.cc:304] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc9272ac1a874097a575a5ddb29bc2b5; no voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:13.238495 22452 raft_consensus.cc:2749] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.242451 22454 raft_consensus.cc:493] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.242883 22454 raft_consensus.cc:515] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.245697 22454 leader_election.cc:290] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
W20260709 13:06:13.247131 22200 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:13.248402 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
W20260709 13:06:13.249665 22221 leader_election.cc:343] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.251101 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.251133 22444 tablet_bootstrap.cc:654] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:13.252521 22221 leader_election.cc:343] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.252954 22221 leader_election.cc:304] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284; no voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:13.253988 22445 tablet_bootstrap.cc:492] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.254038 22454 raft_consensus.cc:2749] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.254522 22445 ts_tablet_manager.cc:1403] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.012s
I20260709 13:06:13.256474 22446 tablet_bootstrap.cc:492] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.256623 22445 raft_consensus.cc:359] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.257190 22446 ts_tablet_manager.cc:1403] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.039s	user 0.006s	sys 0.007s
I20260709 13:06:13.257458 22445 raft_consensus.cc:385] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.257795 22445 raft_consensus.cc:740] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.258515 22445 consensus_queue.cc:260] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.260715 22445 ts_tablet_manager.cc:1434] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:06:13.260818 22446 raft_consensus.cc:359] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.261652 22445 tablet_bootstrap.cc:492] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.261651 22446 raft_consensus.cc:385] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.262712 22446 raft_consensus.cc:740] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.263481 22446 consensus_queue.cc:260] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.265367 22446 ts_tablet_manager.cc:1434] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:13.266194 22446 tablet_bootstrap.cc:492] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.267658 22445 tablet_bootstrap.cc:654] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.272702 22446 tablet_bootstrap.cc:654] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.274106 22444 tablet_bootstrap.cc:492] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.274617 22444 ts_tablet_manager.cc:1403] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.003s
I20260709 13:06:13.278200 22444 raft_consensus.cc:359] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.279147 22444 raft_consensus.cc:385] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.279838 22444 raft_consensus.cc:740] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.280762 22452 raft_consensus.cc:493] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.280900 22444 consensus_queue.cc:260] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.281418 22452 raft_consensus.cc:515] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.284055 22452 leader_election.cc:290] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.284912 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.285802 22255 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 0.
I20260709 13:06:13.286684 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.287587 22367 leader_election.cc:304] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:13.288982 22452 raft_consensus.cc:2804] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:13.289048 22370 leader_election.cc:343] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.289450 22452 raft_consensus.cc:493] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.289788 22452 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.292333 22446 tablet_bootstrap.cc:492] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.292852 22446 ts_tablet_manager.cc:1403] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.009s
I20260709 13:06:13.294047 22444 ts_tablet_manager.cc:1434] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:06:13.294945 22444 tablet_bootstrap.cc:492] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
I20260709 13:06:13.295966 22446 raft_consensus.cc:359] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.296726 22446 raft_consensus.cc:385] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.297068 22446 raft_consensus.cc:740] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.298099 22446 consensus_queue.cc:260] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.300678 22452 raft_consensus.cc:515] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.301293 22444 tablet_bootstrap.cc:654] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.302284 22446 ts_tablet_manager.cc:1434] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:06:13.303694 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:13.303670 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:13.304533 22445 tablet_bootstrap.cc:492] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.304575 22255 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.305018 22445 ts_tablet_manager.cc:1403] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.011s
W20260709 13:06:13.305292 22370 leader_election.cc:343] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.306064 22446 tablet_bootstrap.cc:492] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.307830 22445 raft_consensus.cc:359] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.308531 22445 raft_consensus.cc:385] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.308812 22445 raft_consensus.cc:740] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.309562 22454 raft_consensus.cc:493] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.309746 22445 consensus_queue.cc:260] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.310370 22454 raft_consensus.cc:515] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.312309 22445 ts_tablet_manager.cc:1434] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:13.312662 22446 tablet_bootstrap.cc:654] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.313277 22445 tablet_bootstrap.cc:492] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.316823 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "883d21168bf748d9af3f946c5dba90c8" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.317262 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "883d21168bf748d9af3f946c5dba90c8" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.318135 22452 leader_election.cc:290] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
W20260709 13:06:13.318217 22221 leader_election.cc:343] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.318492 22454 leader_election.cc:290] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
W20260709 13:06:13.318949 22221 leader_election.cc:343] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.319391 22221 leader_election.cc:304] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284; no voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:13.320086 22445 tablet_bootstrap.cc:654] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.320096 22444 tablet_bootstrap.cc:492] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.320783 22454 raft_consensus.cc:2749] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.320938 22444 ts_tablet_manager.cc:1403] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.013s
I20260709 13:06:13.324185 22444 raft_consensus.cc:359] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.324920 22444 raft_consensus.cc:385] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.325212 22444 raft_consensus.cc:740] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.326148 22446 tablet_bootstrap.cc:492] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.326970 22446 ts_tablet_manager.cc:1403] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.007s
I20260709 13:06:13.326078 22444 consensus_queue.cc:260] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.330704 22446 raft_consensus.cc:359] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.331458 22446 raft_consensus.cc:385] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.331764 22446 raft_consensus.cc:740] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.332489 22444 ts_tablet_manager.cc:1434] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:06:13.332599 22446 consensus_queue.cc:260] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.333714 22444 tablet_bootstrap.cc:492] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
I20260709 13:06:13.335129 22446 ts_tablet_manager.cc:1434] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:13.335610 22255 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 1.
I20260709 13:06:13.336071 22446 tablet_bootstrap.cc:492] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.337116 22367 leader_election.cc:304] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: c6b715632dad45a2a0ed9f64533d130f
I20260709 13:06:13.337930 22452 raft_consensus.cc:2804] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.338359 22452 raft_consensus.cc:697] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Becoming Leader. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:13.339138 22452 consensus_queue.cc:237] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.343564 22446 tablet_bootstrap.cc:654] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.349902 22444 tablet_bootstrap.cc:654] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.357200 22446 tablet_bootstrap.cc:492] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.357487 22445 tablet_bootstrap.cc:492] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.357765 22446 ts_tablet_manager.cc:1403] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.007s
I20260709 13:06:13.357944 22445 ts_tablet_manager.cc:1403] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.012s
I20260709 13:06:13.357731 22153 catalog_manager.cc:5697] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.359261 22444 tablet_bootstrap.cc:492] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.359707 22444 ts_tablet_manager.cc:1403] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.002s
I20260709 13:06:13.360687 22452 raft_consensus.cc:493] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.361142 22452 raft_consensus.cc:515] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.361408 22446 raft_consensus.cc:359] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.361227 22445 raft_consensus.cc:359] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.362288 22446 raft_consensus.cc:385] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.362398 22445 raft_consensus.cc:385] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.362661 22446 raft_consensus.cc:740] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.363201 22452 leader_election.cc:290] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.363636 22446 consensus_queue.cc:260] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.364557 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.365411 22330 raft_consensus.cc:2468] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 0.
I20260709 13:06:13.365824 22446 ts_tablet_manager.cc:1434] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:13.366940 22444 raft_consensus.cc:359] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.367260 22370 leader_election.cc:304] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:13.367627 22444 raft_consensus.cc:385] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.364701 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.367985 22444 raft_consensus.cc:740] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.362936 22445 raft_consensus.cc:740] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.368708 22452 raft_consensus.cc:2804] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.369055 22452 raft_consensus.cc:493] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.369455 22452 raft_consensus.cc:3060] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.369112 22444 consensus_queue.cc:260] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.369139 22445 consensus_queue.cc:260] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.370281 22255 raft_consensus.cc:2468] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 0.
I20260709 13:06:13.370579 22446 tablet_bootstrap.cc:492] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.372066 22444 ts_tablet_manager.cc:1434] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:06:13.373080 22444 tablet_bootstrap.cc:492] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
I20260709 13:06:13.377492 22446 tablet_bootstrap.cc:654] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.378953 22445 ts_tablet_manager.cc:1434] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:06:13.379078 22444 tablet_bootstrap.cc:654] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.379931 22445 tablet_bootstrap.cc:492] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.383682 22452 raft_consensus.cc:515] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.388206 22453 raft_consensus.cc:493] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.392185 22453 raft_consensus.cc:515] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.393280 22445 tablet_bootstrap.cc:654] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.394532 22453 leader_election.cc:290] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.395080 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:13.395630 22255 raft_consensus.cc:3060] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.406622 22446 tablet_bootstrap.cc:492] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.407238 22446 ts_tablet_manager.cc:1403] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.037s	user 0.006s	sys 0.011s
I20260709 13:06:13.409629 22255 raft_consensus.cc:2468] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 1.
I20260709 13:06:13.409871 22452 leader_election.cc:290] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.410918 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:13.410965 22367 leader_election.cc:304] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:13.411536 22330 raft_consensus.cc:3060] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.412533 22446 raft_consensus.cc:359] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.414099 22446 raft_consensus.cc:385] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.416028 22444 tablet_bootstrap.cc:492] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.418507 22444 ts_tablet_manager.cc:1403] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.008s
I20260709 13:06:13.412977 22452 raft_consensus.cc:2804] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.419247 22452 raft_consensus.cc:697] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Becoming Leader. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:13.420830 22445 tablet_bootstrap.cc:492] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.420764 22452 consensus_queue.cc:237] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.421275 22445 ts_tablet_manager.cc:1403] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.011s
I20260709 13:06:13.421300 22444 raft_consensus.cc:359] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.422142 22444 raft_consensus.cc:385] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.422446 22444 raft_consensus.cc:740] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.422475 22330 raft_consensus.cc:2468] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 1.
I20260709 13:06:13.423234 22444 consensus_queue.cc:260] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.423750 22445 raft_consensus.cc:359] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.424394 22445 raft_consensus.cc:385] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.424691 22445 raft_consensus.cc:740] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.414479 22446 raft_consensus.cc:740] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.425333 22445 consensus_queue.cc:260] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.425956 22444 ts_tablet_manager.cc:1434] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:13.426888 22444 tablet_bootstrap.cc:492] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
I20260709 13:06:13.432099 22445 ts_tablet_manager.cc:1434] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:06:13.432920 22445 tablet_bootstrap.cc:492] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.433718 22444 tablet_bootstrap.cc:654] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.425629 22446 consensus_queue.cc:260] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.437225 22453 raft_consensus.cc:493] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.437680 22453 raft_consensus.cc:515] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.439375 22453 leader_election.cc:290] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.439324 22452 raft_consensus.cc:493] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.439947 22452 raft_consensus.cc:515] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.440647 22254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.440995 22446 ts_tablet_manager.cc:1434] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:06:13.441255 22254 raft_consensus.cc:2468] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 0.
I20260709 13:06:13.441993 22446 tablet_bootstrap.cc:492] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.441951 22452 leader_election.cc:290] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.442494 22293 leader_election.cc:304] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:13.443341 22453 raft_consensus.cc:2804] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.443655 22453 raft_consensus.cc:493] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.443969 22453 raft_consensus.cc:3060] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.445152 22254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "883d21168bf748d9af3f946c5dba90c8" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.445761 22254 raft_consensus.cc:2468] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 0.
I20260709 13:06:13.446949 22367 leader_election.cc:304] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:13.440171 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7045a815f0814f8c88770be892c32123" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.448603 22446 tablet_bootstrap.cc:654] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:13.448518 22293 leader_election.cc:343] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.450994 22452 raft_consensus.cc:2804] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.451054 22453 raft_consensus.cc:515] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.451457 22452 raft_consensus.cc:493] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.451812 22452 raft_consensus.cc:3060] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.453616 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:13.454116 22255 raft_consensus.cc:3060] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.454630 22445 tablet_bootstrap.cc:654] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.457687 22453 leader_election.cc:290] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.461432 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "883d21168bf748d9af3f946c5dba90c8" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
W20260709 13:06:13.462513 22370 leader_election.cc:343] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.467600 22456 raft_consensus.cc:493] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.467967 22456 raft_consensus.cc:515] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.469663 22456 leader_election.cc:290] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.469558 22452 raft_consensus.cc:515] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.470397 22254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.471225 22255 raft_consensus.cc:2468] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 1.
I20260709 13:06:13.471592 22452 leader_election.cc:290] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.472157 22293 leader_election.cc:304] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:13.472935 22453 raft_consensus.cc:2804] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.473245 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.473512 22453 raft_consensus.cc:697] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Becoming Leader. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:13.473732 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "883d21168bf748d9af3f946c5dba90c8" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
W20260709 13:06:13.474886 22370 leader_election.cc:343] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.474558 22453 consensus_queue.cc:237] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.475610 22254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "883d21168bf748d9af3f946c5dba90c8" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:13.476276 22254 raft_consensus.cc:3060] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.477672 22444 tablet_bootstrap.cc:492] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.478117 22444 ts_tablet_manager.cc:1403] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.051s	user 0.011s	sys 0.001s
I20260709 13:06:13.481302 22444 raft_consensus.cc:359] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.482277 22444 raft_consensus.cc:385] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.482628 22444 raft_consensus.cc:740] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.483533 22444 consensus_queue.cc:260] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.488682 22446 tablet_bootstrap.cc:492] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.488862 22445 tablet_bootstrap.cc:492] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.489367 22446 ts_tablet_manager.cc:1403] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.010s
I20260709 13:06:13.489420 22445 ts_tablet_manager.cc:1403] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.057s	user 0.008s	sys 0.001s
I20260709 13:06:13.490190 22453 raft_consensus.cc:493] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.490624 22453 raft_consensus.cc:515] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.492017 22445 raft_consensus.cc:359] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.492640 22445 raft_consensus.cc:385] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.493005 22445 raft_consensus.cc:740] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.492458 22446 raft_consensus.cc:359] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.493623 22446 raft_consensus.cc:385] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.493764 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.493209 22453 leader_election.cc:290] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.493759 22445 consensus_queue.cc:260] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.494061 22446 raft_consensus.cc:740] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.486142 22444 ts_tablet_manager.cc:1434] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:13.494851 22255 raft_consensus.cc:2468] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 0.
I20260709 13:06:13.494416 22254 raft_consensus.cc:2468] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 1.
I20260709 13:06:13.495443 22444 tablet_bootstrap.cc:492] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
I20260709 13:06:13.495134 22446 consensus_queue.cc:260] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.496044 22293 leader_election.cc:304] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:13.496230 22367 leader_election.cc:304] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: c6b715632dad45a2a0ed9f64533d130f
I20260709 13:06:13.496997 22452 raft_consensus.cc:2804] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.497483 22452 raft_consensus.cc:697] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Becoming Leader. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:13.497848 22453 raft_consensus.cc:2804] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.498283 22453 raft_consensus.cc:493] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.498270 22452 consensus_queue.cc:237] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.498888 22453 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.503027 22445 ts_tablet_manager.cc:1434] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:06:13.504065 22445 tablet_bootstrap.cc:492] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284: Bootstrap starting.
I20260709 13:06:13.497440 22446 ts_tablet_manager.cc:1434] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:06:13.506000 22446 tablet_bootstrap.cc:492] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.508402 22453 raft_consensus.cc:515] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.510898 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:13.511415 22255 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.512521 22445 tablet_bootstrap.cc:654] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.515039 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7045a815f0814f8c88770be892c32123" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.515628 22404 raft_consensus.cc:2468] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 0.
I20260709 13:06:13.516891 22446 tablet_bootstrap.cc:654] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.517204 22296 leader_election.cc:304] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5; no voters: c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:13.518092 22483 raft_consensus.cc:2804] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.518549 22483 raft_consensus.cc:493] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.519088 22483 raft_consensus.cc:3060] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.518085 22255 raft_consensus.cc:2468] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 1.
I20260709 13:06:13.518713 22444 tablet_bootstrap.cc:654] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.520910 22293 leader_election.cc:304] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:13.522033 22404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.522611 22404 raft_consensus.cc:2468] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 0.
I20260709 13:06:13.522506 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:13.523514 22403 raft_consensus.cc:3060] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.523655 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.524911 22401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:13.518576 22453 leader_election.cc:290] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.526036 22453 raft_consensus.cc:2804] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.526453 22453 raft_consensus.cc:697] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Becoming Leader. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:13.527232 22453 consensus_queue.cc:237] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.527932 22152 catalog_manager.cc:5697] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f reported cstate change: term changed from 0 to 1, leader changed from <none> to c6b715632dad45a2a0ed9f64533d130f (127.21.31.66). New cstate: current_term: 1 leader_uuid: "c6b715632dad45a2a0ed9f64533d130f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.534061 22483 raft_consensus.cc:515] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.537092 22370 leader_election.cc:304] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc9272ac1a874097a575a5ddb29bc2b5; no voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284
W20260709 13:06:13.541672 22296 leader_election.cc:343] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.542114 22454 raft_consensus.cc:493] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.542346 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7045a815f0814f8c88770be892c32123" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
W20260709 13:06:13.542791 22296 leader_election.cc:343] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.542559 22454 raft_consensus.cc:515] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.543715 22483 leader_election.cc:290] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
W20260709 13:06:13.543782 22293 leader_election.cc:343] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.547461 22401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7045a815f0814f8c88770be892c32123" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:13.547977 22401 raft_consensus.cc:3060] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.546644 22403 raft_consensus.cc:2468] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 1.
I20260709 13:06:13.547998 22454 leader_election.cc:290] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.548468 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb946af54dd9420ab0b2757068c3215d" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.549511 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb946af54dd9420ab0b2757068c3215d" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.549813 22330 raft_consensus.cc:2468] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 0.
I20260709 13:06:13.550391 22452 raft_consensus.cc:2749] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:13.550858 22221 leader_election.cc:343] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:13.552173 22221 leader_election.cc:304] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:13.553825 22454 raft_consensus.cc:2804] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.554188 22454 raft_consensus.cc:493] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.554541 22454 raft_consensus.cc:3060] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.554813 22444 tablet_bootstrap.cc:492] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.555253 22444 ts_tablet_manager.cc:1403] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.002s
I20260709 13:06:13.556380 22151 catalog_manager.cc:5697] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.557987 22444 raft_consensus.cc:359] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.558691 22444 raft_consensus.cc:385] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.558090 22151 catalog_manager.cc:5697] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.558972 22444 raft_consensus.cc:740] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.559768 22444 consensus_queue.cc:260] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.561728 22444 ts_tablet_manager.cc:1434] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:13.562232 22445 tablet_bootstrap.cc:492] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284: No bootstrap required, opened a new log
I20260709 13:06:13.562589 22444 tablet_bootstrap.cc:492] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
I20260709 13:06:13.563092 22152 catalog_manager.cc:5697] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f reported cstate change: term changed from 0 to 1, leader changed from <none> to c6b715632dad45a2a0ed9f64533d130f (127.21.31.66). New cstate: current_term: 1 leader_uuid: "c6b715632dad45a2a0ed9f64533d130f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.564213 22401 raft_consensus.cc:2468] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 1.
I20260709 13:06:13.562626 22445 ts_tablet_manager.cc:1403] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.001s
I20260709 13:06:13.567165 22296 leader_election.cc:304] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5; no voters: c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:13.568274 22453 raft_consensus.cc:2804] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.568619 22445 raft_consensus.cc:359] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.568928 22453 raft_consensus.cc:697] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Becoming Leader. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:13.569371 22445 raft_consensus.cc:385] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.569649 22445 raft_consensus.cc:740] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.569890 22453 consensus_queue.cc:237] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.572718 22444 tablet_bootstrap.cc:654] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.572598 22454 raft_consensus.cc:515] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.575309 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb946af54dd9420ab0b2757068c3215d" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:13.575850 22330 raft_consensus.cc:3060] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.580899 22445 consensus_queue.cc:260] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.585460 22448 raft_consensus.cc:493] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.585942 22448 raft_consensus.cc:515] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.586200 22330 raft_consensus.cc:2468] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 1.
I20260709 13:06:13.587224 22221 leader_election.cc:304] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:13.587831 22448 leader_election.cc:290] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.588315 22448 raft_consensus.cc:2804] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.588721 22448 raft_consensus.cc:697] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 1 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:13.589816 22444 tablet_bootstrap.cc:492] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.590269 22444 ts_tablet_manager.cc:1403] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.000s
I20260709 13:06:13.591627 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6462355d2d34c009c2516d5bff1af4e" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.592254 22403 raft_consensus.cc:2468] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 0.
I20260709 13:06:13.593006 22446 tablet_bootstrap.cc:492] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.592736 22444 raft_consensus.cc:359] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.593436 22446 ts_tablet_manager.cc:1403] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.088s	user 0.011s	sys 0.000s
I20260709 13:06:13.593387 22444 raft_consensus.cc:385] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.593500 22221 leader_election.cc:304] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:13.593883 22444 raft_consensus.cc:740] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.589723 22448 consensus_queue.cc:237] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.594867 22496 raft_consensus.cc:2804] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.594707 22444 consensus_queue.cc:260] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.595257 22496 raft_consensus.cc:493] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.595734 22496 raft_consensus.cc:3060] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.595963 22446 raft_consensus.cc:359] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.596603 22446 raft_consensus.cc:385] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.596901 22446 raft_consensus.cc:740] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.597537 22446 consensus_queue.cc:260] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.599318 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6462355d2d34c009c2516d5bff1af4e" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.599915 22330 raft_consensus.cc:2468] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 0.
I20260709 13:06:13.601308 22454 leader_election.cc:290] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 election: Requested vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.601468 22445 ts_tablet_manager.cc:1434] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.002s
I20260709 13:06:13.602197 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb946af54dd9420ab0b2757068c3215d" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:13.602608 22446 ts_tablet_manager.cc:1434] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:06:13.599195 22152 catalog_manager.cc:5697] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f reported cstate change: term changed from 0 to 1, leader changed from <none> to c6b715632dad45a2a0ed9f64533d130f (127.21.31.66). New cstate: current_term: 1 leader_uuid: "c6b715632dad45a2a0ed9f64533d130f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.603048 22403 raft_consensus.cc:3060] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.603299 22444 ts_tablet_manager.cc:1434] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:06:13.603426 22446 tablet_bootstrap.cc:492] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.604579 22444 tablet_bootstrap.cc:492] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5: Bootstrap starting.
I20260709 13:06:13.610097 22403 raft_consensus.cc:2468] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 1.
I20260709 13:06:13.610373 22444 tablet_bootstrap.cc:654] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.613678 22496 raft_consensus.cc:515] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.610714 22446 tablet_bootstrap.cc:654] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.616829 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6462355d2d34c009c2516d5bff1af4e" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:13.617614 22330 raft_consensus.cc:3060] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.621779 22454 raft_consensus.cc:493] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.622226 22454 raft_consensus.cc:515] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.627571 22151 catalog_manager.cc:5697] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.632601 22454 leader_election.cc:290] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.634294 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7045a815f0814f8c88770be892c32123" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.635851 22330 raft_consensus.cc:2468] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 1.
I20260709 13:06:13.636270 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7045a815f0814f8c88770be892c32123" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.638269 22403 raft_consensus.cc:2393] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c97a13c5d1e745f58ceabeb252b0d284 in current term 1: Already voted for candidate c6b715632dad45a2a0ed9f64533d130f in this term.
I20260709 13:06:13.637430 22221 leader_election.cc:304] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:13.639429 22454 raft_consensus.cc:2804] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.639660 22496 leader_election.cc:290] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 election: Requested vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.639873 22454 raft_consensus.cc:697] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 1 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:13.639655 22221 leader_election.cc:304] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284; no voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:13.642475 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6462355d2d34c009c2516d5bff1af4e" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:13.643026 22403 raft_consensus.cc:3060] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.641047 22454 consensus_queue.cc:237] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.646770 22448 raft_consensus.cc:3060] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.649981 22403 raft_consensus.cc:2468] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 1.
I20260709 13:06:13.653280 22448 raft_consensus.cc:2749] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.654958 22446 tablet_bootstrap.cc:492] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.654978 22444 tablet_bootstrap.cc:492] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5: No bootstrap required, opened a new log
I20260709 13:06:13.655452 22444 ts_tablet_manager.cc:1403] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5: Time spent bootstrapping tablet: real 0.051s	user 0.019s	sys 0.013s
I20260709 13:06:13.655484 22446 ts_tablet_manager.cc:1403] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.004s
I20260709 13:06:13.657275 22444 raft_consensus.cc:359] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.657940 22444 raft_consensus.cc:385] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.658247 22444 raft_consensus.cc:740] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.657960 22446 raft_consensus.cc:359] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.658648 22446 raft_consensus.cc:385] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.658876 22446 raft_consensus.cc:740] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.659453 22446 consensus_queue.cc:260] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.659582 22444 consensus_queue.cc:260] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.661607 22444 ts_tablet_manager.cc:1434] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:13.661043 22151 catalog_manager.cc:5697] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 0 to 1, leader changed from <none> to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:13.663363 22446 ts_tablet_manager.cc:1434] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:06:13.664254 22446 tablet_bootstrap.cc:492] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f: Bootstrap starting.
I20260709 13:06:13.669770 22446 tablet_bootstrap.cc:654] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:13.673612 22453 raft_consensus.cc:493] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.674011 22453 raft_consensus.cc:515] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:13.676034 22453 leader_election.cc:290] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:13.676486 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.677165 22255 raft_consensus.cc:2393] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c6b715632dad45a2a0ed9f64533d130f in current term 1: Already voted for candidate cc9272ac1a874097a575a5ddb29bc2b5 in this term.
I20260709 13:06:13.677039 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" is_pre_election: true
I20260709 13:06:13.678439 22296 leader_election.cc:304] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f; no voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:13.679032 22453 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.679275 22446 tablet_bootstrap.cc:492] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f: No bootstrap required, opened a new log
I20260709 13:06:13.679687 22446 ts_tablet_manager.cc:1403] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:06:13.682150 22446 raft_consensus.cc:359] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.682749 22446 raft_consensus.cc:385] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:13.683004 22446 raft_consensus.cc:740] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Initialized, Role: FOLLOWER
I20260709 13:06:13.683651 22446 consensus_queue.cc:260] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.685726 22446 ts_tablet_manager.cc:1434] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:06:13.685740 22453 raft_consensus.cc:2749] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:13.745795 22452 raft_consensus.cc:493] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.746275 22452 raft_consensus.cc:515] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.748067 22452 leader_election.cc:290] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.748961 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.749246 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.749540 22255 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 0.
I20260709 13:06:13.749899 22330 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 0.
I20260709 13:06:13.750543 22367 leader_election.cc:304] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:13.751283 22452 raft_consensus.cc:2804] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.751626 22452 raft_consensus.cc:493] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.751920 22452 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.756248 22452 raft_consensus.cc:515] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.757815 22452 leader_election.cc:290] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.758425 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:13.758718 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:13.758958 22255 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.759250 22330 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.765254 22330 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 1.
I20260709 13:06:13.765254 22255 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 1.
I20260709 13:06:13.766464 22367 leader_election.cc:304] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:13.767762 22456 raft_consensus.cc:2804] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.768394 22456 raft_consensus.cc:697] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Becoming Leader. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:13.769455 22456 consensus_queue.cc:237] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.771111 22452 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:13.777792 22150 catalog_manager.cc:5697] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.796243 22456 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:13.898689 22330 raft_consensus.cc:3060] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.904542 22456 consensus_queue.cc:1048] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:13.916839 22456 consensus_queue.cc:1048] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:13.930030 22456 raft_consensus.cc:493] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.930575 22456 raft_consensus.cc:515] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.932647 22456 leader_election.cc:290] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.933614 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284" is_pre_election: true
I20260709 13:06:13.934237 22255 raft_consensus.cc:2468] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 0.
I20260709 13:06:13.934898 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f" is_pre_election: true
I20260709 13:06:13.935420 22330 raft_consensus.cc:2468] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 0.
I20260709 13:06:13.936357 22367 leader_election.cc:304] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:13.937498 22456 raft_consensus.cc:2804] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:13.937814 22456 raft_consensus.cc:493] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.938087 22456 raft_consensus.cc:3060] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.943018 22456 raft_consensus.cc:515] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.944475 22456 leader_election.cc:290] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:13.945163 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:13.945386 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:13.945748 22255 raft_consensus.cc:3060] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.945963 22330 raft_consensus.cc:3060] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.952040 22330 raft_consensus.cc:2468] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 1.
I20260709 13:06:13.952040 22255 raft_consensus.cc:2468] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 1.
I20260709 13:06:13.953217 22367 leader_election.cc:304] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:13.953915 22456 raft_consensus.cc:2804] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:13.954312 22456 raft_consensus.cc:697] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Becoming Leader. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:13.954965 22456 consensus_queue.cc:237] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:13.962673 22150 catalog_manager.cc:5697] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:13.974857 22453 consensus_queue.cc:1048] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:13.985006 22403 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:13.992578 22453 consensus_queue.cc:1048] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.003489 22456 consensus_queue.cc:1048] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.010954 22448 consensus_queue.cc:1048] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.014555 22452 consensus_queue.cc:1048] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.026710 22453 consensus_queue.cc:1048] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.036350 22448 consensus_queue.cc:1048] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.052049 22483 consensus_queue.cc:1048] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.052182 22496 consensus_queue.cc:1048] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:14.070626 22448 consensus_queue.cc:1048] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.074245 22403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
mode: GRACEFUL
new_leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:56842
I20260709 13:06:14.074904 22403 raft_consensus.cc:606] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Received request to transfer leadership to TS c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:14.093837 22329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
mode: GRACEFUL
new_leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:55812
I20260709 13:06:14.094295 22329 raft_consensus.cc:606] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Received request to transfer leadership to TS c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:14.102847 22403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7addde2415594541b513d44721ceeb93"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
mode: GRACEFUL
new_leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:56842
I20260709 13:06:14.103406 22403 raft_consensus.cc:606] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Received request to transfer leadership to TS c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:14.106046 22483 consensus_queue.cc:1048] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.111388 22330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
mode: GRACEFUL
new_leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:55812
I20260709 13:06:14.111943 22330 raft_consensus.cc:606] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Received request to transfer leadership to TS c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:14.118957 22401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
mode: GRACEFUL
new_leader_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:56842
I20260709 13:06:14.119612 22401 raft_consensus.cc:606] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Received request to transfer leadership to TS c6b715632dad45a2a0ed9f64533d130f
I20260709 13:06:14.120432 22453 consensus_queue.cc:1048] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.131088 22401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
mode: GRACEFUL
new_leader_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:56842
I20260709 13:06:14.131978 22401 raft_consensus.cc:606] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Received request to transfer leadership to TS c6b715632dad45a2a0ed9f64533d130f
I20260709 13:06:14.147274 22255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb946af54dd9420ab0b2757068c3215d"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
mode: GRACEFUL
new_leader_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:46574
I20260709 13:06:14.147768 22255 raft_consensus.cc:606] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 1 LEADER]: Received request to transfer leadership to TS c6b715632dad45a2a0ed9f64533d130f
I20260709 13:06:14.297827 22506 consensus_queue.cc:1048] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.307618 22506 consensus_queue.cc:1048] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.317178 22456 raft_consensus.cc:993] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5: : Instructing follower c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:14.317601 22531 raft_consensus.cc:1081] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Signalling peer c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:14.319233 22329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:55796
I20260709 13:06:14.319752 22329 raft_consensus.cc:493] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.320114 22329 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.324963 22329 raft_consensus.cc:515] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.327699 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:14.328258 22255 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.329157 22401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:14.329698 22401 raft_consensus.cc:3055] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.330006 22401 raft_consensus.cc:740] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:14.330757 22401 consensus_queue.cc:260] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.327117 22329 leader_election.cc:290] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 2 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:14.331770 22401 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.335556 22255 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 2.
I20260709 13:06:14.337085 22293 leader_election.cc:304] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:14.337970 22453 raft_consensus.cc:2804] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.338449 22453 raft_consensus.cc:697] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Becoming Leader. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:14.339291 22401 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 2.
I20260709 13:06:14.339221 22453 consensus_queue.cc:237] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.345739 22150 catalog_manager.cc:5697] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f reported cstate change: term changed from 1 to 2, leader changed from cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67) to c6b715632dad45a2a0ed9f64533d130f (127.21.31.66). New cstate: current_term: 2 leader_uuid: "c6b715632dad45a2a0ed9f64533d130f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.370358 22452 raft_consensus.cc:993] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5: : Instructing follower c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:14.370939 22531 raft_consensus.cc:1081] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Signalling peer c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:14.372699 22255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:14.373204 22255 raft_consensus.cc:493] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.373517 22255 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.378002 22255 raft_consensus.cc:515] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:14.379451 22255 leader_election.cc:290] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Requested vote from peers cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:14.380545 22401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:14.380580 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:14.381146 22329 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.381138 22401 raft_consensus.cc:3055] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.381661 22401 raft_consensus.cc:740] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:14.382364 22401 consensus_queue.cc:260] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:14.383411 22401 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.385562 22329 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:14.386585 22221 leader_election.cc:304] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:14.387413 22524 raft_consensus.cc:2804] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.387987 22524 raft_consensus.cc:697] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:14.388790 22524 consensus_queue.cc:237] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:14.389511 22401 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:14.395032 22150 catalog_manager.cc:5697] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 1 to 2, leader changed from cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67) to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.411545 22531 consensus_queue.cc:1048] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.418453 22452 raft_consensus.cc:993] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5: : Instructing follower c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:14.418864 22452 raft_consensus.cc:1081] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Signalling peer c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:14.420933 22253 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7addde2415594541b513d44721ceeb93"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:14.422271 22453 raft_consensus.cc:993] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f: : Instructing follower c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:14.422681 22453 raft_consensus.cc:1081] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Signalling peer c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:14.422885 22253 raft_consensus.cc:493] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.423271 22253 raft_consensus.cc:3060] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.425583 22255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:46564
I20260709 13:06:14.426115 22255 raft_consensus.cc:493] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.426525 22255 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.426333 22531 consensus_queue.cc:1048] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:14.430506 22253 raft_consensus.cc:515] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.433108 22255 raft_consensus.cc:515] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.435112 22255 leader_election.cc:290] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Requested vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:14.436496 22401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:14.437000 22329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:14.437074 22401 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.437958 22253 leader_election.cc:290] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Requested vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:14.438601 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:14.439107 22330 raft_consensus.cc:3060] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.441851 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:14.437618 22329 raft_consensus.cc:3055] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.442373 22403 raft_consensus.cc:3055] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.442433 22329 raft_consensus.cc:740] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:14.442797 22403 raft_consensus.cc:740] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:14.443562 22403 consensus_queue.cc:260] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.443555 22329 consensus_queue.cc:260] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.444849 22403 raft_consensus.cc:3060] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.444937 22401 raft_consensus.cc:2468] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:14.444957 22329 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.445245 22330 raft_consensus.cc:2468] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:14.446310 22221 leader_election.cc:304] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:14.447120 22524 raft_consensus.cc:2804] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.447570 22524 raft_consensus.cc:697] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:14.448347 22524 consensus_queue.cc:237] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.448843 22221 leader_election.cc:304] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:14.449910 22448 raft_consensus.cc:2804] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.450405 22448 raft_consensus.cc:697] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:14.451153 22448 consensus_queue.cc:237] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.452100 22329 raft_consensus.cc:2468] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:14.456413 22403 raft_consensus.cc:2468] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:14.456944 22153 catalog_manager.cc:5697] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 1 to 2, leader changed from c6b715632dad45a2a0ed9f64533d130f (127.21.31.66) to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.470054 22153 catalog_manager.cc:5697] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 1 to 2, leader changed from cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67) to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.478130 22483 raft_consensus.cc:993] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f: : Instructing follower c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:14.478564 22453 raft_consensus.cc:1081] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Signalling peer c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:14.480273 22253 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:46564
I20260709 13:06:14.480947 22253 raft_consensus.cc:493] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.481271 22253 raft_consensus.cc:3060] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.487015 22448 raft_consensus.cc:993] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284: : Instructing follower c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:14.487454 22524 raft_consensus.cc:1081] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 1 LEADER]: Signalling peer c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:14.487885 22253 raft_consensus.cc:515] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.488852 22329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb946af54dd9420ab0b2757068c3215d"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:55786
I20260709 13:06:14.489460 22329 raft_consensus.cc:493] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.489771 22329 raft_consensus.cc:3060] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.490397 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:14.491006 22330 raft_consensus.cc:3055] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.491364 22253 leader_election.cc:290] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Requested vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:14.491374 22330 raft_consensus.cc:740] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:14.492520 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:14.493036 22403 raft_consensus.cc:3060] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.493085 22330 consensus_queue.cc:260] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.494375 22330 raft_consensus.cc:3060] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.495349 22329 raft_consensus.cc:515] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.497581 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb946af54dd9420ab0b2757068c3215d" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:14.497907 22401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb946af54dd9420ab0b2757068c3215d" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:14.498164 22255 raft_consensus.cc:3055] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.498454 22401 raft_consensus.cc:3060] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.498603 22255 raft_consensus.cc:740] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:14.499352 22403 raft_consensus.cc:2468] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:14.499557 22255 consensus_queue.cc:260] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.500419 22221 leader_election.cc:304] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:14.500583 22330 raft_consensus.cc:2468] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:14.500722 22255 raft_consensus.cc:3060] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.501222 22524 raft_consensus.cc:2804] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.502240 22524 raft_consensus.cc:697] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:14.502846 22524 consensus_queue.cc:237] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.504669 22401 raft_consensus.cc:2468] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 2.
I20260709 13:06:14.505743 22296 leader_election.cc:304] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:14.506489 22453 raft_consensus.cc:2804] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.506484 22329 leader_election.cc:290] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 2 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:14.506989 22453 raft_consensus.cc:697] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Becoming Leader. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:14.507368 22255 raft_consensus.cc:2468] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 2.
I20260709 13:06:14.507786 22453 consensus_queue.cc:237] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.509156 22150 catalog_manager.cc:5697] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 1 to 2, leader changed from c6b715632dad45a2a0ed9f64533d130f (127.21.31.66) to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.514150 22153 catalog_manager.cc:5697] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f reported cstate change: term changed from 1 to 2, leader changed from c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65) to c6b715632dad45a2a0ed9f64533d130f (127.21.31.66). New cstate: current_term: 2 leader_uuid: "c6b715632dad45a2a0ed9f64533d130f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.593645 22456 raft_consensus.cc:993] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5: : Instructing follower c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:14.594206 22531 raft_consensus.cc:1081] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Signalling peer c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:14.595808 22329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:55796
I20260709 13:06:14.596320 22329 raft_consensus.cc:493] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:14.596648 22329 raft_consensus.cc:3060] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.600803 22329 raft_consensus.cc:515] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.602257 22329 leader_election.cc:290] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 2 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:14.603317 22255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:14.603459 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:14.603783 22255 raft_consensus.cc:3060] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.604115 22403 raft_consensus.cc:3055] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:14.604419 22403 raft_consensus.cc:740] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:14.605118 22403 consensus_queue.cc:260] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.606138 22403 raft_consensus.cc:3060] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:14.608305 22255 raft_consensus.cc:2468] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 2.
I20260709 13:06:14.609247 22293 leader_election.cc:304] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:14.610112 22483 raft_consensus.cc:2804] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:14.610630 22483 raft_consensus.cc:697] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Becoming Leader. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:14.611594 22403 raft_consensus.cc:2468] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 2.
I20260709 13:06:14.611629 22483 consensus_queue.cc:237] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:14.618574 22153 catalog_manager.cc:5697] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f reported cstate change: term changed from 1 to 2, leader changed from cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67) to c6b715632dad45a2a0ed9f64533d130f (127.21.31.66). New cstate: current_term: 2 leader_uuid: "c6b715632dad45a2a0ed9f64533d130f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:14.744876 22255 raft_consensus.cc:1275] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Refusing update from remote peer c6b715632dad45a2a0ed9f64533d130f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.746407 22483 consensus_queue.cc:1048] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.754598 22403 raft_consensus.cc:1275] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Refusing update from remote peer c6b715632dad45a2a0ed9f64533d130f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.756023 22453 consensus_queue.cc:1048] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.933753 22403 raft_consensus.cc:1275] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.934955 22524 consensus_queue.cc:1048] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.941928 22329 raft_consensus.cc:1275] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.944507 22403 raft_consensus.cc:1275] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.944460 22521 consensus_queue.cc:1048] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.946226 22253 raft_consensus.cc:1275] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Refusing update from remote peer c6b715632dad45a2a0ed9f64533d130f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.946630 22521 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.948642 22483 consensus_queue.cc:1048] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.958135 22330 raft_consensus.cc:1275] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.958788 22401 raft_consensus.cc:1275] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.961439 22401 raft_consensus.cc:1275] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Refusing update from remote peer c6b715632dad45a2a0ed9f64533d130f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.961509 22521 consensus_queue.cc:1048] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.963019 22453 consensus_queue.cc:1048] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.967097 22521 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.969988 22330 raft_consensus.cc:1275] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.976544 22521 consensus_queue.cc:1048] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.989620 22330 raft_consensus.cc:1275] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:14.991780 22521 consensus_queue.cc:1048] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:14.998793 22403 raft_consensus.cc:1275] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:15.000237 22448 consensus_queue.cc:1048] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:15.022181 22403 raft_consensus.cc:1275] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Refusing update from remote peer c6b715632dad45a2a0ed9f64533d130f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:15.023263 22453 consensus_queue.cc:1048] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:15.032541 22253 raft_consensus.cc:1275] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Refusing update from remote peer c6b715632dad45a2a0ed9f64533d130f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:15.034423 22535 consensus_queue.cc:1048] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:17.185619 21629 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:06:17.186270 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:17.202962 22253 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
mode: GRACEFUL
new_leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
 from {username='slave'} at 127.0.0.1:46588
I20260709 13:06:17.203449 22253 raft_consensus.cc:606] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Received request to transfer leadership to TS cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:17.205848 22253 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
mode: GRACEFUL
new_leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
 from {username='slave'} at 127.0.0.1:46588
I20260709 13:06:17.206684 22253 raft_consensus.cc:606] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Received request to transfer leadership to TS cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:17.207948 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:06:17.208330 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:17.380376 22542 raft_consensus.cc:993] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284: : Instructing follower cc9272ac1a874097a575a5ddb29bc2b5 to start an election
I20260709 13:06:17.380841 22542 raft_consensus.cc:1081] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Signalling peer cc9272ac1a874097a575a5ddb29bc2b5 to start an election
I20260709 13:06:17.382258 22401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
 from {username='slave'} at 127.0.0.1:56816
I20260709 13:06:17.382712 22401 raft_consensus.cc:493] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:17.382998 22401 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:17.387501 22401 raft_consensus.cc:515] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:17.389780 22401 leader_election.cc:290] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 3 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:17.390252 22253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:17.390563 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:17.390836 22253 raft_consensus.cc:3055] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:17.391112 22330 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:17.391211 22253 raft_consensus.cc:740] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:17.391997 22253 consensus_queue.cc:260] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:17.392993 22253 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:17.396270 22330 raft_consensus.cc:2468] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 3.
I20260709 13:06:17.397264 22370 leader_election.cc:304] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:17.398141 22531 raft_consensus.cc:2804] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:17.398572 22253 raft_consensus.cc:2468] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 3.
I20260709 13:06:17.399456 22531 raft_consensus.cc:697] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Becoming Leader. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:17.400374 22531 consensus_queue.cc:237] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:17.407780 22151 catalog_manager.cc:5697] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 reported cstate change: term changed from 2 to 3, leader changed from c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65) to cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.461107 22545 raft_consensus.cc:993] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284: : Instructing follower cc9272ac1a874097a575a5ddb29bc2b5 to start an election
I20260709 13:06:17.461542 22542 raft_consensus.cc:1081] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Signalling peer cc9272ac1a874097a575a5ddb29bc2b5 to start an election
I20260709 13:06:17.462833 22401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
 from {username='slave'} at 127.0.0.1:56816
I20260709 13:06:17.463241 22401 raft_consensus.cc:493] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:17.463483 22401 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:17.467789 22401 raft_consensus.cc:515] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:17.469533 22401 leader_election.cc:290] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 3 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:17.470183 22253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:17.470572 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:17.470734 22253 raft_consensus.cc:3055] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:17.471077 22330 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:17.471050 22253 raft_consensus.cc:740] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:17.472028 22253 consensus_queue.cc:260] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:17.473198 22253 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:17.477011 22330 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 3.
I20260709 13:06:17.478070 22370 leader_election.cc:304] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:17.479189 22531 raft_consensus.cc:2804] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:17.479506 22253 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 3.
I20260709 13:06:17.479672 22531 raft_consensus.cc:697] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Becoming Leader. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:17.480589 22531 consensus_queue.cc:237] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:17.487313 22151 catalog_manager.cc:5697] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 reported cstate change: term changed from 2 to 3, leader changed from c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65) to cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.830452 22330 raft_consensus.cc:1275] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Refusing update from remote peer cc9272ac1a874097a575a5ddb29bc2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:17.831586 22531 consensus_queue.cc:1048] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:17.843813 22253 raft_consensus.cc:1275] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Refusing update from remote peer cc9272ac1a874097a575a5ddb29bc2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:17.845206 22555 consensus_queue.cc:1048] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:17.882114 22253 raft_consensus.cc:1275] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Refusing update from remote peer cc9272ac1a874097a575a5ddb29bc2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:17.883385 22555 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:17.891362 22330 raft_consensus.cc:1275] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Refusing update from remote peer cc9272ac1a874097a575a5ddb29bc2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:17.892688 22555 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:18.209193 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:18.220189 22403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
mode: GRACEFUL
new_leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:56842
I20260709 13:06:18.220928 22403 raft_consensus.cc:606] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Received request to transfer leadership to TS c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:18.235046 22403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
mode: GRACEFUL
new_leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:56842
I20260709 13:06:18.235599 22403 raft_consensus.cc:606] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Received request to transfer leadership to TS c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:18.241182 22330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
mode: GRACEFUL
new_leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:55812
I20260709 13:06:18.241766 22330 raft_consensus.cc:606] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Received request to transfer leadership to TS c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:18.247129 22330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7045a815f0814f8c88770be892c32123"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
mode: GRACEFUL
new_leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:55812
I20260709 13:06:18.247588 22330 raft_consensus.cc:606] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Received request to transfer leadership to TS c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:18.252228 22330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
mode: GRACEFUL
new_leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:55812
I20260709 13:06:18.252612 22330 raft_consensus.cc:606] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Received request to transfer leadership to TS c97a13c5d1e745f58ceabeb252b0d284
I20260709 13:06:18.280071 22555 raft_consensus.cc:993] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5: : Instructing follower c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:18.280580 22555 raft_consensus.cc:1081] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Signalling peer c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:18.282435 22253 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:18.282927 22253 raft_consensus.cc:493] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.283305 22253 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:18.288717 22253 raft_consensus.cc:515] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:18.290252 22253 leader_election.cc:290] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 4 election: Requested vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:18.291308 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:18.291550 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912086f5ce2847a982b116cdd49c66fe" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:18.291762 22330 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:18.292126 22403 raft_consensus.cc:3055] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:18.292447 22403 raft_consensus.cc:740] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:18.293196 22403 consensus_queue.cc:260] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:18.294118 22403 raft_consensus.cc:3060] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:18.295897 22330 raft_consensus.cc:2468] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 4.
I20260709 13:06:18.297045 22221 leader_election.cc:304] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:18.297770 22542 raft_consensus.cc:2804] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:18.298765 22542 raft_consensus.cc:697] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 4 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:18.299470 22542 consensus_queue.cc:237] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:18.300460 22403 raft_consensus.cc:2468] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 4.
I20260709 13:06:18.305305 22153 catalog_manager.cc:5697] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 3 to 4, leader changed from cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67) to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 4 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.393843 22561 raft_consensus.cc:993] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f: : Instructing follower c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:18.394325 22561 raft_consensus.cc:1081] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Signalling peer c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:18.395866 22253 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:46564
I20260709 13:06:18.396406 22253 raft_consensus.cc:493] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.396781 22253 raft_consensus.cc:3060] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:18.402240 22253 raft_consensus.cc:515] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:18.403986 22253 leader_election.cc:290] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 3 election: Requested vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:18.404757 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:18.405309 22330 raft_consensus.cc:3055] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:18.405251 22403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62870baa91fe47f380c4a5ba40167761" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:18.406005 22403 raft_consensus.cc:3060] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:18.406770 22330 raft_consensus.cc:740] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:18.407383 22330 consensus_queue.cc:260] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:18.408178 22330 raft_consensus.cc:3060] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:18.410732 22403 raft_consensus.cc:2468] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 3.
I20260709 13:06:18.411664 22221 leader_election.cc:304] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:18.412287 22330 raft_consensus.cc:2468] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 3.
I20260709 13:06:18.412310 22542 raft_consensus.cc:2804] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:18.412910 22542 raft_consensus.cc:697] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:18.413697 22542 consensus_queue.cc:237] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:18.419651 22153 catalog_manager.cc:5697] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 2 to 3, leader changed from c6b715632dad45a2a0ed9f64533d130f (127.21.31.66) to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 3 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.758853 22546 raft_consensus.cc:993] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5: : Instructing follower c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:18.759373 22555 raft_consensus.cc:1081] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Signalling peer c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:18.761600 22255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:46560
I20260709 13:06:18.762146 22255 raft_consensus.cc:493] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.762490 22255 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:18.768750 22255 raft_consensus.cc:515] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:18.770311 22255 leader_election.cc:290] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 4 election: Requested vote from peers cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:19.261065 22402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:19.262205 22402 raft_consensus.cc:3055] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:19.262530 22402 raft_consensus.cc:740] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:19.263325 22402 consensus_queue.cc:260] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:19.265337 22402 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:19.272272 22401 raft_consensus.cc:1275] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:19.274315 22402 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 4.
I20260709 13:06:19.275338 22221 leader_election.cc:304] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:19.281948 22562 consensus_queue.cc:1048] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:19.283128 22562 raft_consensus.cc:2804] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:19.283516 22562 raft_consensus.cc:697] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 4 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:19.284226 22562 consensus_queue.cc:237] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:19.286170 22404 raft_consensus.cc:1275] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:19.311699 22330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:19.312345 22330 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:19.388617 22330 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 4.
I20260709 13:06:19.402079 22329 raft_consensus.cc:1275] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:19.403838 22329 raft_consensus.cc:1275] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:19.409116 22569 raft_consensus.cc:993] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f: : Instructing follower c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:19.412282 22572 raft_consensus.cc:993] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f: : Instructing follower c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:19.415663 22573 raft_consensus.cc:1081] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Signalling peer c97a13c5d1e745f58ceabeb252b0d284 to start an election
I20260709 13:06:19.416605 22567 raft_consensus.cc:1081] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Signalling peer c97a13c5d1e745f58ceabeb252b0d284 to start an election
W20260709 13:06:19.444341 22221 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:44237, user_credentials={real_user=slave}} blocked reactor thread for 168793us
I20260709 13:06:19.449649 22253 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7045a815f0814f8c88770be892c32123"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:46564
I20260709 13:06:19.450961 22253 raft_consensus.cc:493] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:19.451957 22253 raft_consensus.cc:3060] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.452530 22153 catalog_manager.cc:5697] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 3 to 4, leader changed from cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67) to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 4 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:19.454540 22562 consensus_queue.cc:1048] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:19.456899 22562 consensus_queue.cc:1048] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:19.458679 22542 consensus_queue.cc:1048] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:19.469902 22254 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
 from {username='slave'} at 127.0.0.1:46564
I20260709 13:06:19.471213 22254 raft_consensus.cc:493] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:19.471799 22254 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.473347 22253 raft_consensus.cc:515] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:19.476189 22253 leader_election.cc:290] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Requested vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:19.494469 22400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7045a815f0814f8c88770be892c32123" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:19.494727 22328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7045a815f0814f8c88770be892c32123" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:19.497720 22328 raft_consensus.cc:3055] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:19.498097 22328 raft_consensus.cc:740] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:19.499346 22328 consensus_queue.cc:260] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:19.502689 22328 raft_consensus.cc:3060] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.508545 22400 raft_consensus.cc:3060] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:19.501778 22254 raft_consensus.cc:515] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:19.527246 22254 leader_election.cc:290] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 3 election: Requested vote from peers c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:19.525054 22400 raft_consensus.cc:2468] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:19.528453 22327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:19.534679 22327 raft_consensus.cc:3055] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:19.535073 22327 raft_consensus.cc:740] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:19.536232 22327 consensus_queue.cc:260] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:19.533917 22221 leader_election.cc:304] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:19.540678 22327 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.544898 22328 raft_consensus.cc:2468] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 2.
I20260709 13:06:19.551012 22542 raft_consensus.cc:2804] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:19.554560 22400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "c97a13c5d1e745f58ceabeb252b0d284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:19.555635 22400 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:19.563864 22542 raft_consensus.cc:697] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:19.564978 22542 consensus_queue.cc:237] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:19.584189 22327 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 3.
I20260709 13:06:19.585826 22221 leader_election.cc:304] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, c97a13c5d1e745f58ceabeb252b0d284; no voters: 
I20260709 13:06:19.588089 22568 raft_consensus.cc:2804] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:19.588850 22568 raft_consensus.cc:697] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 LEADER]: Becoming Leader. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:19.584735 22400 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c97a13c5d1e745f58ceabeb252b0d284 in term 3.
I20260709 13:06:19.590296 22568 consensus_queue.cc:237] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:19.601424 22151 catalog_manager.cc:5697] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 1 to 2, leader changed from c6b715632dad45a2a0ed9f64533d130f (127.21.31.66) to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:19.615247 22151 catalog_manager.cc:5697] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 reported cstate change: term changed from 2 to 3, leader changed from c6b715632dad45a2a0ed9f64533d130f (127.21.31.66) to c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65). New cstate: current_term: 3 leader_uuid: "c97a13c5d1e745f58ceabeb252b0d284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:19.691660 22327 raft_consensus.cc:1275] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:19.692956 22568 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:19.706213 22400 raft_consensus.cc:1275] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:19.707628 22592 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:19.994107 22400 raft_consensus.cc:1275] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:19.995496 22592 consensus_queue.cc:1048] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:20.004670 22329 raft_consensus.cc:1275] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:20.006191 22591 consensus_queue.cc:1048] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:20.009974 22327 raft_consensus.cc:1275] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:20.011219 22591 consensus_queue.cc:1048] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:20.022485 22400 raft_consensus.cc:1275] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Refusing update from remote peer c97a13c5d1e745f58ceabeb252b0d284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:20.024246 22542 consensus_queue.cc:1048] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:20.235200 22276 debug-util.cc:398] Leaking SignalData structure 0x7b08001534c0 after lost signal to thread 21632
W20260709 13:06:20.236016 22276 debug-util.cc:398] Leaking SignalData structure 0x7b0800153ba0 after lost signal to thread 22187
W20260709 13:06:20.236855 22276 debug-util.cc:398] Leaking SignalData structure 0x7b08001e5000 after lost signal to thread 22279
W20260709 13:06:20.237689 22276 debug-util.cc:398] Leaking SignalData structure 0x7b080012fc60 after lost signal to thread 22354
W20260709 13:06:20.238433 22276 debug-util.cc:398] Leaking SignalData structure 0x7b08000e7ae0 after lost signal to thread 22428
I20260709 13:06:21.569545 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:21.582098 22254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
mode: GRACEFUL
new_leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
 from {username='slave'} at 127.0.0.1:46588
I20260709 13:06:21.582868 22254 raft_consensus.cc:606] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 LEADER]: Received request to transfer leadership to TS cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:21.585711 22254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7addde2415594541b513d44721ceeb93"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
mode: GRACEFUL
new_leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
 from {username='slave'} at 127.0.0.1:46588
I20260709 13:06:21.586233 22254 raft_consensus.cc:606] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Received request to transfer leadership to TS cc9272ac1a874097a575a5ddb29bc2b5
I20260709 13:06:21.588967 22254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
mode: GRACEFUL
new_leader_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:46588
I20260709 13:06:21.589532 22254 raft_consensus.cc:606] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 4 LEADER]: Received request to transfer leadership to TS c6b715632dad45a2a0ed9f64533d130f
I20260709 13:06:21.591864 22254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493"
dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
mode: GRACEFUL
new_leader_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:46588
I20260709 13:06:21.592357 22254 raft_consensus.cc:606] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Received request to transfer leadership to TS c6b715632dad45a2a0ed9f64533d130f
I20260709 13:06:21.593631 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:06:21.594237 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:21.907145 22591 raft_consensus.cc:993] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284: : Instructing follower c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:21.907733 22602 raft_consensus.cc:1081] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 4 LEADER]: Signalling peer c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:21.909327 22328 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:55786
I20260709 13:06:21.909838 22328 raft_consensus.cc:493] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.910133 22328 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:21.914986 22328 raft_consensus.cc:515] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:21.916625 22328 leader_election.cc:290] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 5 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:21.917452 22254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:21.917711 22400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fb331256164b8da46a7a1bbfc322ea" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:21.918036 22254 raft_consensus.cc:3055] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:21.918282 22400 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:21.918407 22254 raft_consensus.cc:740] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:21.919384 22254 consensus_queue.cc:260] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:21.920701 22254 raft_consensus.cc:3060] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:21.922873 22400 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 5.
I20260709 13:06:21.924681 22296 leader_election.cc:304] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:21.925657 22601 raft_consensus.cc:2804] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:21.927654 22254 raft_consensus.cc:2468] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 5.
I20260709 13:06:21.931002 22601 raft_consensus.cc:697] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 5 LEADER]: Becoming Leader. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:21.932433 22601 consensus_queue.cc:237] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } }
I20260709 13:06:21.942739 22591 raft_consensus.cc:993] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284: : Instructing follower cc9272ac1a874097a575a5ddb29bc2b5 to start an election
I20260709 13:06:21.942134 22153 catalog_manager.cc:5697] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f reported cstate change: term changed from 4 to 5, leader changed from c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65) to c6b715632dad45a2a0ed9f64533d130f (127.21.31.66). New cstate: current_term: 5 leader_uuid: "c6b715632dad45a2a0ed9f64533d130f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:21.943253 22602 raft_consensus.cc:1081] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 LEADER]: Signalling peer cc9272ac1a874097a575a5ddb29bc2b5 to start an election
I20260709 13:06:21.944832 22400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
 from {username='slave'} at 127.0.0.1:56816
I20260709 13:06:21.945523 22400 raft_consensus.cc:493] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.946003 22400 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:21.952765 22400 raft_consensus.cc:515] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:21.956562 22254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:21.956877 22328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a6381296e374c33ac2ccb4df323a4b6" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:21.957266 22254 raft_consensus.cc:3055] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:21.957839 22254 raft_consensus.cc:740] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:21.957896 22328 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:21.959064 22254 consensus_queue.cc:260] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:21.962620 22254 raft_consensus.cc:3060] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:21.970640 22254 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 4.
I20260709 13:06:21.955603 22400 leader_election.cc:290] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 4 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:21.970854 22328 raft_consensus.cc:2468] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 4.
I20260709 13:06:21.971953 22367 leader_election.cc:304] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:21.972908 22555 raft_consensus.cc:2804] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:21.976055 22555 raft_consensus.cc:697] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 LEADER]: Becoming Leader. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:21.977010 22555 consensus_queue.cc:237] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:21.985181 22153 catalog_manager.cc:5697] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 reported cstate change: term changed from 3 to 4, leader changed from c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65) to cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67). New cstate: current_term: 4 leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:22.022891 22591 raft_consensus.cc:993] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284: : Instructing follower cc9272ac1a874097a575a5ddb29bc2b5 to start an election
I20260709 13:06:22.023336 22602 raft_consensus.cc:1081] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Signalling peer cc9272ac1a874097a575a5ddb29bc2b5 to start an election
I20260709 13:06:22.024951 22400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7addde2415594541b513d44721ceeb93"
dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
 from {username='slave'} at 127.0.0.1:56816
I20260709 13:06:22.025533 22400 raft_consensus.cc:493] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.025902 22400 raft_consensus.cc:3060] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.032867 22400 raft_consensus.cc:515] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:22.034941 22400 leader_election.cc:290] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 3 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), c6b715632dad45a2a0ed9f64533d130f (127.21.31.66:33007)
I20260709 13:06:22.035602 22254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:22.035980 22328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7addde2415594541b513d44721ceeb93" candidate_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
I20260709 13:06:22.036154 22254 raft_consensus.cc:3055] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:22.036581 22254 raft_consensus.cc:740] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:22.036613 22328 raft_consensus.cc:3060] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.037420 22254 consensus_queue.cc:260] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:22.038331 22254 raft_consensus.cc:3060] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.042712 22254 raft_consensus.cc:2468] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 3.
I20260709 13:06:22.043694 22367 leader_election.cc:304] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c97a13c5d1e745f58ceabeb252b0d284, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:22.043709 22328 raft_consensus.cc:2468] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9272ac1a874097a575a5ddb29bc2b5 in term 3.
I20260709 13:06:22.044371 22555 raft_consensus.cc:2804] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:22.044996 22555 raft_consensus.cc:697] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Becoming Leader. State: Replica: cc9272ac1a874097a575a5ddb29bc2b5, State: Running, Role: LEADER
I20260709 13:06:22.045760 22555 consensus_queue.cc:237] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:22.054380 22153 catalog_manager.cc:5697] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 reported cstate change: term changed from 2 to 3, leader changed from c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65) to cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:22.119733 22591 raft_consensus.cc:993] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284: : Instructing follower c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:22.120162 22591 raft_consensus.cc:1081] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Signalling peer c6b715632dad45a2a0ed9f64533d130f to start an election
I20260709 13:06:22.121971 22328 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493"
dest_uuid: "c6b715632dad45a2a0ed9f64533d130f"
 from {username='slave'} at 127.0.0.1:55786
I20260709 13:06:22.122463 22328 raft_consensus.cc:493] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.122747 22328 raft_consensus.cc:3060] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.127373 22328 raft_consensus.cc:515] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:22.129393 22328 leader_election.cc:290] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 3 election: Requested vote from peers c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281), cc9272ac1a874097a575a5ddb29bc2b5 (127.21.31.67:44237)
I20260709 13:06:22.129994 22254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c97a13c5d1e745f58ceabeb252b0d284"
I20260709 13:06:22.130362 22400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99dc4cba5fda42e486de1283b8078493" candidate_uuid: "c6b715632dad45a2a0ed9f64533d130f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc9272ac1a874097a575a5ddb29bc2b5"
I20260709 13:06:22.130726 22254 raft_consensus.cc:3055] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:22.130949 22400 raft_consensus.cc:3060] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.131165 22254 raft_consensus.cc:740] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c97a13c5d1e745f58ceabeb252b0d284, State: Running, Role: LEADER
I20260709 13:06:22.132375 22254 consensus_queue.cc:260] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:22.133826 22254 raft_consensus.cc:3060] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:22.135744 22400 raft_consensus.cc:2468] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 3.
I20260709 13:06:22.136641 22296 leader_election.cc:304] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6b715632dad45a2a0ed9f64533d130f, cc9272ac1a874097a575a5ddb29bc2b5; no voters: 
I20260709 13:06:22.137342 22601 raft_consensus.cc:2804] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:22.137776 22601 raft_consensus.cc:697] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 3 LEADER]: Becoming Leader. State: Replica: c6b715632dad45a2a0ed9f64533d130f, State: Running, Role: LEADER
I20260709 13:06:22.138492 22601 consensus_queue.cc:237] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } }
I20260709 13:06:22.143175 22254 raft_consensus.cc:2468] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6b715632dad45a2a0ed9f64533d130f in term 3.
I20260709 13:06:22.151252 22153 catalog_manager.cc:5697] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f reported cstate change: term changed from 2 to 3, leader changed from c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65) to c6b715632dad45a2a0ed9f64533d130f (127.21.31.66). New cstate: current_term: 3 leader_uuid: "c6b715632dad45a2a0ed9f64533d130f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.446887 22328 raft_consensus.cc:1275] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Refusing update from remote peer cc9272ac1a874097a575a5ddb29bc2b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:22.448156 22555 consensus_queue.cc:1048] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:22.457690 22254 raft_consensus.cc:1275] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Refusing update from remote peer cc9272ac1a874097a575a5ddb29bc2b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:22.459110 22555 consensus_queue.cc:1048] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:22.483876 22328 raft_consensus.cc:1275] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Refusing update from remote peer cc9272ac1a874097a575a5ddb29bc2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:22.485057 22555 consensus_queue.cc:1048] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b715632dad45a2a0ed9f64533d130f" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:22.495290 22254 raft_consensus.cc:1275] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Refusing update from remote peer cc9272ac1a874097a575a5ddb29bc2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:22.497490 22555 consensus_queue.cc:1048] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:22.555496 22400 raft_consensus.cc:1275] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 5 FOLLOWER]: Refusing update from remote peer c6b715632dad45a2a0ed9f64533d130f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:06:22.556764 22601 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:22.557811 22254 raft_consensus.cc:1275] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 5 FOLLOWER]: Refusing update from remote peer c6b715632dad45a2a0ed9f64533d130f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:06:22.558971 22601 consensus_queue.cc:1048] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97a13c5d1e745f58ceabeb252b0d284" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 34281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:22.595120 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:22.601154 21629 tablet_server.cc:179] TabletServer@127.21.31.65:0 shutting down...
W20260709 13:06:22.605969 22293 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 13:06:22.617110 22293 consensus_peers.cc:597] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f -> Peer c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281): Couldn't send request to peer c97a13c5d1e745f58ceabeb252b0d284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:22.631093 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:22.631712 21629 tablet_replica.cc:333] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.632333 21629 raft_consensus.cc:2243] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.633031 21629 raft_consensus.cc:2272] T 912086f5ce2847a982b116cdd49c66fe P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.634778 21629 tablet_replica.cc:333] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.635277 21629 raft_consensus.cc:2243] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.635667 21629 raft_consensus.cc:2272] T fb946af54dd9420ab0b2757068c3215d P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.637168 21629 tablet_replica.cc:333] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.637768 21629 raft_consensus.cc:2243] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.638391 21629 raft_consensus.cc:2272] T 62870baa91fe47f380c4a5ba40167761 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.639879 21629 tablet_replica.cc:333] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.640353 21629 raft_consensus.cc:2243] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.640961 21629 raft_consensus.cc:2272] T 7045a815f0814f8c88770be892c32123 P c97a13c5d1e745f58ceabeb252b0d284 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.642441 21629 tablet_replica.cc:333] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.642925 21629 raft_consensus.cc:2243] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.643293 21629 raft_consensus.cc:2272] T 7addde2415594541b513d44721ceeb93 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.645430 21629 tablet_replica.cc:333] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.645900 21629 raft_consensus.cc:2243] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.646327 21629 raft_consensus.cc:2272] T 883d21168bf748d9af3f946c5dba90c8 P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.647883 21629 tablet_replica.cc:333] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.648355 21629 raft_consensus.cc:2243] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.648672 21629 raft_consensus.cc:2272] T 90fb331256164b8da46a7a1bbfc322ea P c97a13c5d1e745f58ceabeb252b0d284 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.650831 21629 tablet_replica.cc:333] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.651330 21629 raft_consensus.cc:2243] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.651700 21629 raft_consensus.cc:2272] T 99dc4cba5fda42e486de1283b8078493 P c97a13c5d1e745f58ceabeb252b0d284 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.653358 21629 tablet_replica.cc:333] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.653811 21629 raft_consensus.cc:2243] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.654469 21629 raft_consensus.cc:2272] T c6462355d2d34c009c2516d5bff1af4e P c97a13c5d1e745f58ceabeb252b0d284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.656759 21629 tablet_replica.cc:333] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284: stopping tablet replica
I20260709 13:06:22.657373 21629 raft_consensus.cc:2243] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.657963 21629 raft_consensus.cc:2272] T 7a6381296e374c33ac2ccb4df323a4b6 P c97a13c5d1e745f58ceabeb252b0d284 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.664467 22400 raft_consensus.cc:1275] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Refusing update from remote peer c6b715632dad45a2a0ed9f64533d130f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:22.665776 22603 consensus_queue.cc:1048] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9272ac1a874097a575a5ddb29bc2b5" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:22.700611 21629 tablet_server.cc:196] TabletServer@127.21.31.65:0 shutdown complete.
I20260709 13:06:22.722748 21629 tablet_server.cc:179] TabletServer@127.21.31.66:0 shutting down...
W20260709 13:06:22.746124 22293 consensus_peers.cc:597] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f -> Peer c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281): Couldn't send request to peer c97a13c5d1e745f58ceabeb252b0d284. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:34281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:22.752739 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:22.753480 21629 tablet_replica.cc:333] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.754074 21629 raft_consensus.cc:2243] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.754549 21629 raft_consensus.cc:2272] T c6462355d2d34c009c2516d5bff1af4e P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.756260 21629 tablet_replica.cc:333] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.756858 21629 raft_consensus.cc:2243] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.757292 21629 raft_consensus.cc:2272] T 7a6381296e374c33ac2ccb4df323a4b6 P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.759227 21629 tablet_replica.cc:333] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.759727 21629 raft_consensus.cc:2243] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.760360 21629 raft_consensus.cc:2272] T fb946af54dd9420ab0b2757068c3215d P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.761996 21629 tablet_replica.cc:333] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.762468 21629 raft_consensus.cc:2243] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.763019 21629 raft_consensus.cc:2272] T 90fb331256164b8da46a7a1bbfc322ea P c6b715632dad45a2a0ed9f64533d130f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.765280 21629 tablet_replica.cc:333] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.765949 21629 raft_consensus.cc:2243] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.766371 21629 raft_consensus.cc:2272] T 62870baa91fe47f380c4a5ba40167761 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.768134 21629 tablet_replica.cc:333] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.768692 21629 raft_consensus.cc:2243] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.769498 21629 raft_consensus.cc:2272] T 99dc4cba5fda42e486de1283b8078493 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.771519 21629 tablet_replica.cc:333] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.772082 21629 raft_consensus.cc:2243] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.772554 21629 raft_consensus.cc:2272] T 912086f5ce2847a982b116cdd49c66fe P c6b715632dad45a2a0ed9f64533d130f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.774458 21629 tablet_replica.cc:333] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.774993 21629 raft_consensus.cc:2243] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.775405 21629 raft_consensus.cc:2272] T 883d21168bf748d9af3f946c5dba90c8 P c6b715632dad45a2a0ed9f64533d130f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.777516 21629 tablet_replica.cc:333] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.778085 21629 raft_consensus.cc:2243] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.778543 21629 raft_consensus.cc:2272] T 7addde2415594541b513d44721ceeb93 P c6b715632dad45a2a0ed9f64533d130f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.780566 21629 tablet_replica.cc:333] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f: stopping tablet replica
I20260709 13:06:22.781066 21629 raft_consensus.cc:2243] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.781512 21629 raft_consensus.cc:2272] T 7045a815f0814f8c88770be892c32123 P c6b715632dad45a2a0ed9f64533d130f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.810428 21629 tablet_server.cc:196] TabletServer@127.21.31.66:0 shutdown complete.
I20260709 13:06:22.828737 21629 tablet_server.cc:179] TabletServer@127.21.31.67:0 shutting down...
I20260709 13:06:22.860656 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:22.861641 21629 tablet_replica.cc:333] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.862273 21629 raft_consensus.cc:2243] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.862721 21629 raft_consensus.cc:2272] T 912086f5ce2847a982b116cdd49c66fe P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.864341 21629 tablet_replica.cc:333] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.864871 21629 raft_consensus.cc:2243] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.865283 21629 raft_consensus.cc:2272] T fb946af54dd9420ab0b2757068c3215d P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.867184 21629 tablet_replica.cc:333] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.867717 21629 raft_consensus.cc:2243] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.868270 21629 raft_consensus.cc:2272] T 62870baa91fe47f380c4a5ba40167761 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.869959 21629 tablet_replica.cc:333] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.870460 21629 raft_consensus.cc:2243] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.870884 21629 raft_consensus.cc:2272] T 99dc4cba5fda42e486de1283b8078493 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:22.872319 22367 consensus_peers.cc:597] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 -> Peer c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281): Couldn't send request to peer c97a13c5d1e745f58ceabeb252b0d284. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:34281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:22.873497 21629 tablet_replica.cc:333] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.874060 21629 raft_consensus.cc:2243] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.874599 21629 raft_consensus.cc:2272] T 90fb331256164b8da46a7a1bbfc322ea P cc9272ac1a874097a575a5ddb29bc2b5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.876832 21629 tablet_replica.cc:333] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.877378 21629 raft_consensus.cc:2243] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.878253 21629 raft_consensus.cc:2272] T 7addde2415594541b513d44721ceeb93 P cc9272ac1a874097a575a5ddb29bc2b5 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:22.878872 22367 consensus_peers.cc:597] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 -> Peer c97a13c5d1e745f58ceabeb252b0d284 (127.21.31.65:34281): Couldn't send request to peer c97a13c5d1e745f58ceabeb252b0d284. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:34281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:22.880236 21629 tablet_replica.cc:333] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.880765 21629 raft_consensus.cc:2243] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.881145 21629 raft_consensus.cc:2272] T 7045a815f0814f8c88770be892c32123 P cc9272ac1a874097a575a5ddb29bc2b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.882688 21629 tablet_replica.cc:333] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.883134 21629 raft_consensus.cc:2243] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.883867 21629 raft_consensus.cc:2272] T 883d21168bf748d9af3f946c5dba90c8 P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.885304 21629 tablet_replica.cc:333] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.885823 21629 raft_consensus.cc:2243] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.886484 21629 raft_consensus.cc:2272] T 7a6381296e374c33ac2ccb4df323a4b6 P cc9272ac1a874097a575a5ddb29bc2b5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.888276 21629 tablet_replica.cc:333] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5: stopping tablet replica
I20260709 13:06:22.888717 21629 raft_consensus.cc:2243] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:22.889096 21629 raft_consensus.cc:2272] T c6462355d2d34c009c2516d5bff1af4e P cc9272ac1a874097a575a5ddb29bc2b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.908078 21629 tablet_server.cc:196] TabletServer@127.21.31.67:0 shutdown complete.
I20260709 13:06:22.929870 21629 master.cc:562] Master@127.21.31.126:46017 shutting down...
I20260709 13:06:22.949618 21629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:22.950206 21629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:22.950503 21629 tablet_replica.cc:333] T 00000000000000000000000000000000 P f548adf7cbb047eaa8a01de96ffa7475: stopping tablet replica
I20260709 13:06:22.971313 21629 master.cc:584] Master@127.21.31.126:46017 shutdown complete.
c6b715632dad45a2a0ed9f64533d130f  4
c97a13c5d1e745f58ceabeb252b0d284  5
cc9272ac1a874097a575a5ddb29bc2b5  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10918 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:06:23.011154 21629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.31.126:35001
I20260709 13:06:23.012163 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:23.018399 22630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:23.018430 22629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:23.019580 21629 server_base.cc:1061] running on GCE node
W20260709 13:06:23.019671 22632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:23.020764 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:23.020995 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:23.021163 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602383021148 us; error 0 us; skew 500 ppm
I20260709 13:06:23.022323 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:23.025074 21629 webserver.cc:533] Webserver started at http://127.21.31.126:34585/ using document root <none> and password file <none>
I20260709 13:06:23.025625 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:23.025832 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:23.026115 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:23.027189 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/master-0-root/instance:
uuid: "359166ef74e042839c4e58133432df83"
format_stamp: "Formatted at 2026-07-09 13:06:23 on dist-test-slave-9s6w"
I20260709 13:06:23.031446 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:06:23.035002 22637 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.035874 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:23.036103 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/master-0-root
uuid: "359166ef74e042839c4e58133432df83"
format_stamp: "Formatted at 2026-07-09 13:06:23 on dist-test-slave-9s6w"
I20260709 13:06:23.036330 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:23.049494 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:23.050876 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:23.094027 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.126:35001
I20260709 13:06:23.094136 22688 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.126:35001 every 8 connection(s)
I20260709 13:06:23.097761 22689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.107498 22689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Bootstrap starting.
I20260709 13:06:23.112725 22689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:23.117208 22689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: No bootstrap required, opened a new log
I20260709 13:06:23.119295 22689 raft_consensus.cc:359] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359166ef74e042839c4e58133432df83" member_type: VOTER }
I20260709 13:06:23.119688 22689 raft_consensus.cc:385] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:23.119926 22689 raft_consensus.cc:740] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359166ef74e042839c4e58133432df83, State: Initialized, Role: FOLLOWER
I20260709 13:06:23.120524 22689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359166ef74e042839c4e58133432df83" member_type: VOTER }
I20260709 13:06:23.120975 22689 raft_consensus.cc:399] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:23.121218 22689 raft_consensus.cc:493] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:23.121517 22689 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:23.125416 22689 raft_consensus.cc:515] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359166ef74e042839c4e58133432df83" member_type: VOTER }
I20260709 13:06:23.125942 22689 leader_election.cc:304] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 359166ef74e042839c4e58133432df83; no voters: 
I20260709 13:06:23.127023 22689 leader_election.cc:290] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:23.127416 22692 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:23.128698 22692 raft_consensus.cc:697] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 1 LEADER]: Becoming Leader. State: Replica: 359166ef74e042839c4e58133432df83, State: Running, Role: LEADER
I20260709 13:06:23.129520 22692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359166ef74e042839c4e58133432df83" member_type: VOTER }
I20260709 13:06:23.130208 22689 sys_catalog.cc:565] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:23.133662 22693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "359166ef74e042839c4e58133432df83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359166ef74e042839c4e58133432df83" member_type: VOTER } }
I20260709 13:06:23.133818 22694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 359166ef74e042839c4e58133432df83. Latest consensus state: current_term: 1 leader_uuid: "359166ef74e042839c4e58133432df83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "359166ef74e042839c4e58133432df83" member_type: VOTER } }
I20260709 13:06:23.134379 22693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:23.134508 22694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:23.138095 22698 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:23.146477 22698 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:23.147655 21629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:23.157181 22698 catalog_manager.cc:1383] Generated new cluster ID: 7b648355af984c428c61f15f69f0ce51
I20260709 13:06:23.157531 22698 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:23.179067 22698 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:23.180562 22698 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:23.195080 22698 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Generated new TSK 0
I20260709 13:06:23.195870 22698 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:23.214856 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:23.220994 22710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:23.222540 22711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:23.225404 21629 server_base.cc:1061] running on GCE node
W20260709 13:06:23.225286 22713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:23.226501 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:23.226703 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:23.226899 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602383226884 us; error 0 us; skew 500 ppm
I20260709 13:06:23.227478 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:23.229998 21629 webserver.cc:533] Webserver started at http://127.21.31.65:43475/ using document root <none> and password file <none>
I20260709 13:06:23.230473 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:23.230659 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:23.230945 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:23.231971 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/instance:
uuid: "6a5191aaff76469bbf6c4735dc9e6509"
format_stamp: "Formatted at 2026-07-09 13:06:23 on dist-test-slave-9s6w"
I20260709 13:06:23.236301 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:06:23.240016 22718 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.241142 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:06:23.241475 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "6a5191aaff76469bbf6c4735dc9e6509"
format_stamp: "Formatted at 2026-07-09 13:06:23 on dist-test-slave-9s6w"
I20260709 13:06:23.241720 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:23.252454 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:23.253620 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:23.255133 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:23.257413 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:23.257623 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.257870 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:23.258056 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.300825 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:37213
I20260709 13:06:23.300951 22780 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:37213 every 8 connection(s)
I20260709 13:06:23.308120 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:23.316699 22785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:23.318643 22786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:23.321002 22788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:23.321874 21629 server_base.cc:1061] running on GCE node
I20260709 13:06:23.322679 22781 heartbeater.cc:344] Connected to a master server at 127.21.31.126:35001
I20260709 13:06:23.323053 22781 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:23.323258 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:23.323488 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:23.323670 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602383323647 us; error 0 us; skew 500 ppm
I20260709 13:06:23.323788 22781 heartbeater.cc:507] Master 127.21.31.126:35001 requested a full tablet report, sending...
I20260709 13:06:23.324247 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:23.326494 22654 ts_manager.cc:194] Registered new tserver with Master: 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:23.326797 21629 webserver.cc:533] Webserver started at http://127.21.31.66:37717/ using document root <none> and password file <none>
I20260709 13:06:23.327382 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:23.327593 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:23.327860 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:23.328820 22654 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41780
I20260709 13:06:23.328936 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/instance:
uuid: "7aad11e70ca94fce8f672db05f1c28c3"
format_stamp: "Formatted at 2026-07-09 13:06:23 on dist-test-slave-9s6w"
I20260709 13:06:23.333635 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:23.336812 22793 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.337648 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:06:23.337985 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root
uuid: "7aad11e70ca94fce8f672db05f1c28c3"
format_stamp: "Formatted at 2026-07-09 13:06:23 on dist-test-slave-9s6w"
I20260709 13:06:23.338318 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:23.371073 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:23.372218 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:23.373637 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:23.375774 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:23.375952 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.376243 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:23.376398 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.418706 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.66:39003
I20260709 13:06:23.418807 22855 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.66:39003 every 8 connection(s)
I20260709 13:06:23.424127 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:23.432226 22859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:23.432725 22860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:23.435062 22862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:23.435855 21629 server_base.cc:1061] running on GCE node
I20260709 13:06:23.436125 22856 heartbeater.cc:344] Connected to a master server at 127.21.31.126:35001
I20260709 13:06:23.436514 22856 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:23.436880 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:23.437141 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:23.437366 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602383437346 us; error 0 us; skew 500 ppm
I20260709 13:06:23.437505 22856 heartbeater.cc:507] Master 127.21.31.126:35001 requested a full tablet report, sending...
I20260709 13:06:23.438097 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:23.439842 22654 ts_manager.cc:194] Registered new tserver with Master: 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:23.440915 21629 webserver.cc:533] Webserver started at http://127.21.31.67:40359/ using document root <none> and password file <none>
I20260709 13:06:23.441509 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:23.441488 22654 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41790
I20260709 13:06:23.441869 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:23.442185 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:23.444252 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/instance:
uuid: "3b626c41600d4737a178ebab8a7ebfc2"
format_stamp: "Formatted at 2026-07-09 13:06:23 on dist-test-slave-9s6w"
I20260709 13:06:23.448705 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:06:23.452333 22867 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.453495 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:23.453833 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root
uuid: "3b626c41600d4737a178ebab8a7ebfc2"
format_stamp: "Formatted at 2026-07-09 13:06:23 on dist-test-slave-9s6w"
I20260709 13:06:23.454170 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:23.470878 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:23.472007 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:23.473590 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:23.475792 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:23.476022 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.476257 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:23.476431 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:23.520324 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.67:35797
I20260709 13:06:23.520601 22929 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.67:35797 every 8 connection(s)
I20260709 13:06:23.536453 22930 heartbeater.cc:344] Connected to a master server at 127.21.31.126:35001
I20260709 13:06:23.536862 22930 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:23.537642 22930 heartbeater.cc:507] Master 127.21.31.126:35001 requested a full tablet report, sending...
I20260709 13:06:23.539644 22654 ts_manager.cc:194] Registered new tserver with Master: 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:23.540089 21629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01605671s
I20260709 13:06:23.541165 22654 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41804
I20260709 13:06:23.567440 22654 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41818:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:06:23.573278 22654 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:06:23.768723 22895 tablet_service.cc:1511] Processing CreateTablet for tablet 4b7d2a5f73cc4afd98b29c55cc0eb72b (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:23.769476 22894 tablet_service.cc:1511] Processing CreateTablet for tablet e8ff32d0405c426786004110e1c1f955 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:23.770174 22895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b7d2a5f73cc4afd98b29c55cc0eb72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.770771 22894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ff32d0405c426786004110e1c1f955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.770756 22893 tablet_service.cc:1511] Processing CreateTablet for tablet 3833f41661774b918ddf80499fd85f2f (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:23.772050 22893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3833f41661774b918ddf80499fd85f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.779431 22892 tablet_service.cc:1511] Processing CreateTablet for tablet c7d4d50470a044f698ff4852a4b634ca (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:23.780891 22892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d4d50470a044f698ff4852a4b634ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.782619 22891 tablet_service.cc:1511] Processing CreateTablet for tablet 84ea43361e534fbebad704327c9f6367 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:23.784679 22891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ea43361e534fbebad704327c9f6367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.791577 22890 tablet_service.cc:1511] Processing CreateTablet for tablet 27b79b7fd4bf4e90a1a7f357a253688d (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:23.792896 22890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b79b7fd4bf4e90a1a7f357a253688d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.792717 22745 tablet_service.cc:1511] Processing CreateTablet for tablet e8ff32d0405c426786004110e1c1f955 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:23.794000 22745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ff32d0405c426786004110e1c1f955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.794427 22889 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0563f7efc8452c86a8c67c0189fe81 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:23.795620 22889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0563f7efc8452c86a8c67c0189fe81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.798451 22746 tablet_service.cc:1511] Processing CreateTablet for tablet 4b7d2a5f73cc4afd98b29c55cc0eb72b (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:23.799640 22746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b7d2a5f73cc4afd98b29c55cc0eb72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.802417 22744 tablet_service.cc:1511] Processing CreateTablet for tablet 3833f41661774b918ddf80499fd85f2f (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:23.803709 22744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3833f41661774b918ddf80499fd85f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.803454 22888 tablet_service.cc:1511] Processing CreateTablet for tablet 264d1889b1f144c5897b33c834b1beef (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:23.804698 22888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 264d1889b1f144c5897b33c834b1beef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.806826 22887 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4b3212e80646379ccf77b5952a4363 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:23.808068 22887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4b3212e80646379ccf77b5952a4363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.822921 22743 tablet_service.cc:1511] Processing CreateTablet for tablet c7d4d50470a044f698ff4852a4b634ca (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:23.824313 22743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d4d50470a044f698ff4852a4b634ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.824251 22739 tablet_service.cc:1511] Processing CreateTablet for tablet 264d1889b1f144c5897b33c834b1beef (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:23.825567 22739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 264d1889b1f144c5897b33c834b1beef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.826447 22742 tablet_service.cc:1511] Processing CreateTablet for tablet 84ea43361e534fbebad704327c9f6367 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:23.827693 22742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ea43361e534fbebad704327c9f6367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.830368 22741 tablet_service.cc:1511] Processing CreateTablet for tablet 27b79b7fd4bf4e90a1a7f357a253688d (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:23.831025 22738 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4b3212e80646379ccf77b5952a4363 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:23.831820 22741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b79b7fd4bf4e90a1a7f357a253688d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.832489 22738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4b3212e80646379ccf77b5952a4363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.833828 22740 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0563f7efc8452c86a8c67c0189fe81 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:23.835074 22740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0563f7efc8452c86a8c67c0189fe81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.838860 22737 tablet_service.cc:1511] Processing CreateTablet for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:23.840129 22737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.871251 22886 tablet_service.cc:1511] Processing CreateTablet for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:23.872666 22886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.959394 22887 tablet_service.cc:1511] Processing CreateTablet for tablet 6436de6ceb37488b9ee8b6d261356a40 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:23.960640 22887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6436de6ceb37488b9ee8b6d261356a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.963371 22886 tablet_service.cc:1511] Processing CreateTablet for tablet f7d0e2e573344ec49ba2412e5de1d519 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:23.951876 22894 tablet_service.cc:1511] Processing CreateTablet for tablet 90274c6677ab4d37b51190e940c9608e (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:23.979616 22888 tablet_service.cc:1511] Processing CreateTablet for tablet 847c8a2cc0bb4233a1d55ca04643418d (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:23.982249 22889 tablet_service.cc:1511] Processing CreateTablet for tablet 7701b9b4e69441f7ae9de9554134de69 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:23.985598 22886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d0e2e573344ec49ba2412e5de1d519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.001255 22818 tablet_service.cc:1511] Processing CreateTablet for tablet c7d4d50470a044f698ff4852a4b634ca (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:24.001668 22889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7701b9b4e69441f7ae9de9554134de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.986750 22743 tablet_service.cc:1511] Processing CreateTablet for tablet 6436de6ceb37488b9ee8b6d261356a40 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:23.989548 22738 tablet_service.cc:1511] Processing CreateTablet for tablet f7d0e2e573344ec49ba2412e5de1d519 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:24.014127 22738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d0e2e573344ec49ba2412e5de1d519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.014184 22813 tablet_service.cc:1511] Processing CreateTablet for tablet 4e4b3212e80646379ccf77b5952a4363 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:23.991248 22821 tablet_service.cc:1511] Processing CreateTablet for tablet 4b7d2a5f73cc4afd98b29c55cc0eb72b (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:24.017105 22812 tablet_service.cc:1511] Processing CreateTablet for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:23.992544 22946 tablet_bootstrap.cc:492] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:23.993229 22820 tablet_service.cc:1511] Processing CreateTablet for tablet e8ff32d0405c426786004110e1c1f955 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:23.992575 22737 tablet_service.cc:1511] Processing CreateTablet for tablet 847c8a2cc0bb4233a1d55ca04643418d (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:24.020067 22737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847c8a2cc0bb4233a1d55ca04643418d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.022497 22889 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9578eb52c342078c7555d5100bdcad (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:24.023829 22946 tablet_bootstrap.cc:654] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:23.995314 22819 tablet_service.cc:1511] Processing CreateTablet for tablet 3833f41661774b918ddf80499fd85f2f (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:23.997066 22745 tablet_service.cc:1511] Processing CreateTablet for tablet 7701b9b4e69441f7ae9de9554134de69 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:23.997599 22744 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:23.997773 22746 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9578eb52c342078c7555d5100bdcad (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:23.998907 22740 tablet_service.cc:1511] Processing CreateTablet for tablet 18a127a20cfa4496bad51cb634bd0d0f (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:23.983660 22742 tablet_service.cc:1511] Processing CreateTablet for tablet 90274c6677ab4d37b51190e940c9608e (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:24.026928 22886 tablet_service.cc:1511] Processing CreateTablet for tablet 135e2f7520a34b53aaf4ddb360a8a029 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:24.002504 22818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d4d50470a044f698ff4852a4b634ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.028465 22821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b7d2a5f73cc4afd98b29c55cc0eb72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.029047 22812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.029757 22819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3833f41661774b918ddf80499fd85f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.033162 22813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4b3212e80646379ccf77b5952a4363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.033785 22820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ff32d0405c426786004110e1c1f955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.003270 22894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90274c6677ab4d37b51190e940c9608e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:23.981782 22893 tablet_service.cc:1511] Processing CreateTablet for tablet 53ca1af745cb4fd48a40af06ae1c43ca (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:23.984839 22741 tablet_service.cc:1511] Processing CreateTablet for tablet 53ca1af745cb4fd48a40af06ae1c43ca (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:24.005002 22945 tablet_bootstrap.cc:492] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:24.005196 22817 tablet_service.cc:1511] Processing CreateTablet for tablet 84ea43361e534fbebad704327c9f6367 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:24.037111 22817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ea43361e534fbebad704327c9f6367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.007427 22739 tablet_service.cc:1511] Processing CreateTablet for tablet ccab55924ea9473faf1d4d97313e9284 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:24.008442 22816 tablet_service.cc:1511] Processing CreateTablet for tablet 27b79b7fd4bf4e90a1a7f357a253688d (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:24.044272 22816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b79b7fd4bf4e90a1a7f357a253688d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.009570 22815 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0563f7efc8452c86a8c67c0189fe81 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:24.045861 22815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0563f7efc8452c86a8c67c0189fe81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.009783 22891 tablet_service.cc:1511] Processing CreateTablet for tablet 18a127a20cfa4496bad51cb634bd0d0f (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:24.053819 22891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a127a20cfa4496bad51cb634bd0d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.057554 22893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53ca1af745cb4fd48a40af06ae1c43ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.060068 22888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847c8a2cc0bb4233a1d55ca04643418d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.010383 22895 tablet_service.cc:1511] Processing CreateTablet for tablet ccab55924ea9473faf1d4d97313e9284 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:24.011804 22814 tablet_service.cc:1511] Processing CreateTablet for tablet 264d1889b1f144c5897b33c834b1beef (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:24.090337 22814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 264d1889b1f144c5897b33c834b1beef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.090446 22738 tablet_service.cc:1511] Processing CreateTablet for tablet 4d8a8f3cf129492eb679da24703f5b84 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:24.026067 22745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7701b9b4e69441f7ae9de9554134de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.006830 22892 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:24.094067 22892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.097770 22738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d8a8f3cf129492eb679da24703f5b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.097960 22895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccab55924ea9473faf1d4d97313e9284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.103181 22946 tablet_bootstrap.cc:492] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:24.103694 22946 ts_tablet_manager.cc:1403] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.111s	user 0.016s	sys 0.000s
I20260709 13:06:24.105593 22739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccab55924ea9473faf1d4d97313e9284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.106524 22946 raft_consensus.cc:359] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.109579 22741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53ca1af745cb4fd48a40af06ae1c43ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.117887 22740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a127a20cfa4496bad51cb634bd0d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.118127 22894 tablet_service.cc:1511] Processing CreateTablet for tablet ab1914222f984a8f9db2e3956a97bad0 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:24.118323 22890 tablet_service.cc:1511] Processing CreateTablet for tablet 4d8a8f3cf129492eb679da24703f5b84 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:24.121419 22946 raft_consensus.cc:385] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.121829 22746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9578eb52c342078c7555d5100bdcad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.121824 22946 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.122820 22742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90274c6677ab4d37b51190e940c9608e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.123412 22744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.123363 22946 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.128029 22737 tablet_service.cc:1511] Processing CreateTablet for tablet 135e2f7520a34b53aaf4ddb360a8a029 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:24.110037 22889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9578eb52c342078c7555d5100bdcad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.115624 22813 tablet_service.cc:1511] Processing CreateTablet for tablet 90274c6677ab4d37b51190e940c9608e (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:24.129289 22737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135e2f7520a34b53aaf4ddb360a8a029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.130780 22813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90274c6677ab4d37b51190e940c9608e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:24.142004 22702 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:24.141651 22895 tablet_service.cc:1511] Processing CreateTablet for tablet 317f6a16e0af41e2984a9b1626ab0edc (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:24.138800 22819 tablet_service.cc:1511] Processing CreateTablet for tablet f7d0e2e573344ec49ba2412e5de1d519 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:24.150745 22819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d0e2e573344ec49ba2412e5de1d519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.146263 22820 tablet_service.cc:1511] Processing CreateTablet for tablet 6436de6ceb37488b9ee8b6d261356a40 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:24.139904 22948 tablet_bootstrap.cc:492] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:24.141211 22818 tablet_service.cc:1511] Processing CreateTablet for tablet 847c8a2cc0bb4233a1d55ca04643418d (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:24.130007 22945 tablet_bootstrap.cc:654] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.156769 22820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6436de6ceb37488b9ee8b6d261356a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.158779 22743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6436de6ceb37488b9ee8b6d261356a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.165589 22818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847c8a2cc0bb4233a1d55ca04643418d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.131189 22890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d8a8f3cf129492eb679da24703f5b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.177207 22945 tablet_bootstrap.cc:492] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:24.141161 22738 tablet_service.cc:1511] Processing CreateTablet for tablet ab1914222f984a8f9db2e3956a97bad0 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:24.206377 22945 ts_tablet_manager.cc:1403] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.202s	user 0.013s	sys 0.011s
I20260709 13:06:24.207240 22738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1914222f984a8f9db2e3956a97bad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.174340 22781 heartbeater.cc:499] Master 127.21.31.126:35001 was elected leader, sending a full tablet report...
I20260709 13:06:24.209223 22945 raft_consensus.cc:359] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.210124 22945 raft_consensus.cc:385] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.170112 22894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1914222f984a8f9db2e3956a97bad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.210446 22945 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.199290 22745 tablet_service.cc:1511] Processing CreateTablet for tablet 41fb40751eb74da580d83c9bb69ae58a (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:24.211488 22892 tablet_service.cc:1511] Processing CreateTablet for tablet 41fb40751eb74da580d83c9bb69ae58a (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:24.211951 22745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41fb40751eb74da580d83c9bb69ae58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.212685 22892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41fb40751eb74da580d83c9bb69ae58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.212579 22949 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.178373 22948 tablet_bootstrap.cc:654] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.213105 22949 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.214052 22819 tablet_service.cc:1511] Processing CreateTablet for tablet 7701b9b4e69441f7ae9de9554134de69 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:24.215266 22886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135e2f7520a34b53aaf4ddb360a8a029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.217443 22895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317f6a16e0af41e2984a9b1626ab0edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.217235 22945 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.174808 22946 ts_tablet_manager.cc:1434] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.071s	user 0.006s	sys 0.006s
I20260709 13:06:24.220125 22946 tablet_bootstrap.cc:492] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:24.220342 22821 tablet_service.cc:1511] Processing CreateTablet for tablet 53ca1af745cb4fd48a40af06ae1c43ca (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:24.221171 22814 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:24.169593 22741 tablet_service.cc:1511] Processing CreateTablet for tablet 317f6a16e0af41e2984a9b1626ab0edc (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:24.222958 22741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317f6a16e0af41e2984a9b1626ab0edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.215263 22819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7701b9b4e69441f7ae9de9554134de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.230558 22930 heartbeater.cc:499] Master 127.21.31.126:35001 was elected leader, sending a full tablet report...
I20260709 13:06:24.231846 22818 tablet_service.cc:1511] Processing CreateTablet for tablet 18a127a20cfa4496bad51cb634bd0d0f (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:24.233114 22818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a127a20cfa4496bad51cb634bd0d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.241780 22945 ts_tablet_manager.cc:1434] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.003s
I20260709 13:06:24.243129 22945 tablet_bootstrap.cc:492] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:24.250468 22951 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.250921 22951 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.267459 22746 tablet_service.cc:1511] Processing CreateTablet for tablet ac8ca58483824d0bbd1e56d06088b4da (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:24.268746 22746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac8ca58483824d0bbd1e56d06088b4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.272291 22814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.277211 22813 tablet_service.cc:1511] Processing CreateTablet for tablet ccab55924ea9473faf1d4d97313e9284 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:24.285789 22949 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:24.287083 22945 tablet_bootstrap.cc:654] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.288934 22890 tablet_service.cc:1511] Processing CreateTablet for tablet ac8ca58483824d0bbd1e56d06088b4da (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:24.296147 22812 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9578eb52c342078c7555d5100bdcad (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:24.297911 22744 tablet_service.cc:1511] Processing CreateTablet for tablet f53f146bb25444ce8d2072559efe5e59 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:24.300684 22951 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:24.304239 22740 tablet_service.cc:1511] Processing CreateTablet for tablet db154be77d6e4b5388f9a3622f91a4ba (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:24.305962 22891 tablet_service.cc:1511] Processing CreateTablet for tablet f53f146bb25444ce8d2072559efe5e59 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:24.308699 22742 tablet_service.cc:1511] Processing CreateTablet for tablet 1f881fed251c4393ac053086bd572549 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:24.321100 22948 tablet_bootstrap.cc:492] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:24.331009 22891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53f146bb25444ce8d2072559efe5e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.333461 22813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccab55924ea9473faf1d4d97313e9284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.337532 22948 ts_tablet_manager.cc:1403] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.198s	user 0.013s	sys 0.004s
I20260709 13:06:24.340646 22948 raft_consensus.cc:359] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.341413 22948 raft_consensus.cc:385] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.341699 22948 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.343879 22946 tablet_bootstrap.cc:654] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.342514 22948 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.356889 22890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac8ca58483824d0bbd1e56d06088b4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.358402 22742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f881fed251c4393ac053086bd572549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.370335 22819 tablet_service.cc:1511] Processing CreateTablet for tablet 135e2f7520a34b53aaf4ddb360a8a029 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:24.360641 22821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53ca1af745cb4fd48a40af06ae1c43ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.381307 22817 tablet_service.cc:1511] Processing CreateTablet for tablet ab1914222f984a8f9db2e3956a97bad0 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:24.378105 22815 tablet_service.cc:1511] Processing CreateTablet for tablet 4d8a8f3cf129492eb679da24703f5b84 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:24.379091 22888 tablet_service.cc:1511] Processing CreateTablet for tablet 11e8d51b5b18431cb7d6df2884a4e879 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:24.382117 22812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9578eb52c342078c7555d5100bdcad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.344002 22887 tablet_service.cc:1511] Processing CreateTablet for tablet 1f881fed251c4393ac053086bd572549 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:24.390213 22888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11e8d51b5b18431cb7d6df2884a4e879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.396003 22886 tablet_service.cc:1511] Processing CreateTablet for tablet 469a8bc68a8b4e62b19f382c167cf50c (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:24.398376 22816 tablet_service.cc:1511] Processing CreateTablet for tablet 317f6a16e0af41e2984a9b1626ab0edc (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:24.390627 22889 tablet_service.cc:1511] Processing CreateTablet for tablet 7f765d75643f47dd939002936724a315 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:24.390642 22819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135e2f7520a34b53aaf4ddb360a8a029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.400401 22815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d8a8f3cf129492eb679da24703f5b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.400518 22744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53f146bb25444ce8d2072559efe5e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.403239 22816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317f6a16e0af41e2984a9b1626ab0edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.403339 22740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db154be77d6e4b5388f9a3622f91a4ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.398594 22820 tablet_service.cc:1511] Processing CreateTablet for tablet 41fb40751eb74da580d83c9bb69ae58a (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:24.404495 22820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41fb40751eb74da580d83c9bb69ae58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.405861 22886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469a8bc68a8b4e62b19f382c167cf50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.409759 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:24.410434 22889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f765d75643f47dd939002936724a315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.411017 22887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f881fed251c4393ac053086bd572549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.413663 22817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1914222f984a8f9db2e3956a97bad0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:24.414845 22719 leader_election.cc:343] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.362780 22737 tablet_service.cc:1511] Processing CreateTablet for tablet 7f765d75643f47dd939002936724a315 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:24.418812 22893 tablet_service.cc:1511] Processing CreateTablet for tablet db154be77d6e4b5388f9a3622f91a4ba (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:24.431625 22745 tablet_service.cc:1511] Processing CreateTablet for tablet 11e8d51b5b18431cb7d6df2884a4e879 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:24.420424 22737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f765d75643f47dd939002936724a315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.442842 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:24.443904 22856 heartbeater.cc:499] Master 127.21.31.126:35001 was elected leader, sending a full tablet report...
W20260709 13:06:24.449440 22868 leader_election.cc:343] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.455664 22892 tablet_service.cc:1511] Processing CreateTablet for tablet 267f1f9ab3f742418b8505ba4859ab18 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:24.456889 22745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11e8d51b5b18431cb7d6df2884a4e879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.457214 22814 tablet_service.cc:1511] Processing CreateTablet for tablet f53f146bb25444ce8d2072559efe5e59 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:24.458426 22814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53f146bb25444ce8d2072559efe5e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.461814 22893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db154be77d6e4b5388f9a3622f91a4ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.463094 22895 tablet_service.cc:1511] Processing CreateTablet for tablet e4cc99e6a1f14071a587af061e82fee0 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:24.464279 22895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cc99e6a1f14071a587af061e82fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.480067 22948 ts_tablet_manager.cc:1434] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.142s	user 0.004s	sys 0.011s
I20260709 13:06:24.481415 22948 tablet_bootstrap.cc:492] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:24.485570 22892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267f1f9ab3f742418b8505ba4859ab18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.503404 22741 tablet_service.cc:1511] Processing CreateTablet for tablet 267f1f9ab3f742418b8505ba4859ab18 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:24.504653 22741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267f1f9ab3f742418b8505ba4859ab18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.504459 22818 tablet_service.cc:1511] Processing CreateTablet for tablet db154be77d6e4b5388f9a3622f91a4ba (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:24.515342 22742 tablet_service.cc:1511] Processing CreateTablet for tablet 09a9ba6091094973a526986ef573e2a9 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:24.516611 22742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a9ba6091094973a526986ef573e2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.516953 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:24.518141 22813 tablet_service.cc:1511] Processing CreateTablet for tablet 1f881fed251c4393ac053086bd572549 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
W20260709 13:06:24.518569 22871 leader_election.cc:343] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.519018 22871 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:24.519343 22813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f881fed251c4393ac053086bd572549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.519951 22951 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:24.493808 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 80205.4us
I20260709 13:06:24.524474 22945 tablet_bootstrap.cc:492] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:24.524302 22738 tablet_service.cc:1511] Processing CreateTablet for tablet 60a957de2a744260812f8da6f1bd9b87 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:24.525061 22945 ts_tablet_manager.cc:1403] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.282s	user 0.016s	sys 0.009s
I20260709 13:06:24.527089 22819 tablet_service.cc:1511] Processing CreateTablet for tablet 7f765d75643f47dd939002936724a315 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:24.528275 22819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f765d75643f47dd939002936724a315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.528591 22948 tablet_bootstrap.cc:654] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.529596 22739 tablet_service.cc:1511] Processing CreateTablet for tablet e4cc99e6a1f14071a587af061e82fee0 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:24.493158 22812 tablet_service.cc:1511] Processing CreateTablet for tablet ac8ca58483824d0bbd1e56d06088b4da (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:24.534334 22812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac8ca58483824d0bbd1e56d06088b4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.535017 22746 tablet_service.cc:1511] Processing CreateTablet for tablet 67449ba22ccb4938be8a5e28664cc566 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:24.535744 22743 tablet_service.cc:1511] Processing CreateTablet for tablet dbd3d6688dfb4621b8ecfd11278d5b26 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:24.537140 22886 tablet_service.cc:1511] Processing CreateTablet for tablet 09a9ba6091094973a526986ef573e2a9 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:24.537770 22818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db154be77d6e4b5388f9a3622f91a4ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.538383 22886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a9ba6091094973a526986ef573e2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.550820 22946 tablet_bootstrap.cc:492] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:24.551293 22946 ts_tablet_manager.cc:1403] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.331s	user 0.004s	sys 0.020s
I20260709 13:06:24.552068 22817 tablet_service.cc:1511] Processing CreateTablet for tablet 11e8d51b5b18431cb7d6df2884a4e879 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:24.555284 22821 tablet_service.cc:1511] Processing CreateTablet for tablet 6ce4678cb66d4d1e8325b447eaf2a630 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:24.553292 22817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11e8d51b5b18431cb7d6df2884a4e879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.574936 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:24.576658 22719 leader_election.cc:343] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.565493 22738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a957de2a744260812f8da6f1bd9b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.577251 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:24.578112 22949 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:24.571694 22889 tablet_service.cc:1511] Processing CreateTablet for tablet dbd3d6688dfb4621b8ecfd11278d5b26 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:24.579136 22889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbd3d6688dfb4621b8ecfd11278d5b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.577576 22743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbd3d6688dfb4621b8ecfd11278d5b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.560294 22890 tablet_service.cc:1511] Processing CreateTablet for tablet 67449ba22ccb4938be8a5e28664cc566 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:24.580053 22739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cc99e6a1f14071a587af061e82fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.580603 22890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67449ba22ccb4938be8a5e28664cc566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.571854 22740 tablet_service.cc:1511] Processing CreateTablet for tablet 469a8bc68a8b4e62b19f382c167cf50c (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:24.586534 22740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469a8bc68a8b4e62b19f382c167cf50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.586336 22946 raft_consensus.cc:359] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.586282 22945 raft_consensus.cc:359] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.587109 22946 raft_consensus.cc:385] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.587248 22945 raft_consensus.cc:385] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.587450 22946 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.587682 22945 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.588282 22946 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.588604 22945 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.557008 22891 tablet_service.cc:1511] Processing CreateTablet for tablet 60a957de2a744260812f8da6f1bd9b87 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:24.593142 22891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a957de2a744260812f8da6f1bd9b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.593456 22746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67449ba22ccb4938be8a5e28664cc566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.576689 22821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ce4678cb66d4d1e8325b447eaf2a630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.537250 22744 tablet_service.cc:1511] Processing CreateTablet for tablet 82d0d73de20d49d3a64fef8b052288e5 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:24.590606 22946 ts_tablet_manager.cc:1434] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.039s	user 0.003s	sys 0.003s
I20260709 13:06:24.603379 22744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d0d73de20d49d3a64fef8b052288e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.609292 22888 tablet_service.cc:1511] Processing CreateTablet for tablet 2147efadd4ea46a09bda395402a10299 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:24.610401 22888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147efadd4ea46a09bda395402a10299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.615131 22815 tablet_service.cc:1511] Processing CreateTablet for tablet 267f1f9ab3f742418b8505ba4859ab18 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:24.616313 22815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267f1f9ab3f742418b8505ba4859ab18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.618233 22945 ts_tablet_manager.cc:1434] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.093s	user 0.004s	sys 0.002s
I20260709 13:06:24.618988 22945 tablet_bootstrap.cc:492] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:24.622077 22895 tablet_service.cc:1511] Processing CreateTablet for tablet 82d0d73de20d49d3a64fef8b052288e5 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:24.559379 22737 tablet_service.cc:1511] Processing CreateTablet for tablet 2147efadd4ea46a09bda395402a10299 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:24.623638 22737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147efadd4ea46a09bda395402a10299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.623967 22894 tablet_service.cc:1511] Processing CreateTablet for tablet 3583f8fda02d46af9ef0ff2ae6f1b443 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:24.624300 22948 tablet_bootstrap.cc:492] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:24.624395 22895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d0d73de20d49d3a64fef8b052288e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.625002 22948 ts_tablet_manager.cc:1403] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.144s	user 0.009s	sys 0.005s
I20260709 13:06:24.625191 22894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3583f8fda02d46af9ef0ff2ae6f1b443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.628122 22948 raft_consensus.cc:359] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.628870 22948 raft_consensus.cc:385] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.629184 22948 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.630414 22949 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.630342 22948 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.630970 22949 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.632161 22745 tablet_service.cc:1511] Processing CreateTablet for tablet 18051671d25f4aa7b30d4a299909f91a (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:24.632872 22949 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:24.633540 22745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18051671d25f4aa7b30d4a299909f91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.636945 22946 tablet_bootstrap.cc:492] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:24.637226 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:24.637483 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:24.638885 22719 leader_election.cc:343] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:24.639415 22719 leader_election.cc:343] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.639760 22719 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:24.640451 22949 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:24.641723 22893 tablet_service.cc:1511] Processing CreateTablet for tablet 18051671d25f4aa7b30d4a299909f91a (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:24.647449 22948 ts_tablet_manager.cc:1434] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:06:24.654402 22948 tablet_bootstrap.cc:492] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:24.654968 22888 tablet_service.cc:1511] Processing CreateTablet for tablet 4bbeaee00aa942ccb232c812c9e14827 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:24.655642 22740 tablet_service.cc:1511] Processing CreateTablet for tablet 3583f8fda02d46af9ef0ff2ae6f1b443 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:24.656895 22740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3583f8fda02d46af9ef0ff2ae6f1b443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.660513 22948 tablet_bootstrap.cc:654] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.666920 22948 tablet_bootstrap.cc:492] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:24.643011 22893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18051671d25f4aa7b30d4a299909f91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.667407 22948 ts_tablet_manager.cc:1403] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:06:24.670149 22948 raft_consensus.cc:359] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.670841 22948 raft_consensus.cc:385] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:24.671128 22948 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:24.671373 22888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bbeaee00aa942ccb232c812c9e14827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.671780 22948 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.678894 22816 tablet_service.cc:1511] Processing CreateTablet for tablet 82d0d73de20d49d3a64fef8b052288e5 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:24.680087 22816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d0d73de20d49d3a64fef8b052288e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.683005 22948 ts_tablet_manager.cc:1434] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:06:24.684021 22948 tablet_bootstrap.cc:492] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:24.688272 22893 tablet_service.cc:1511] Processing CreateTablet for tablet 00324d171dc0468fa259f13c1d4d74ea (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:24.689507 22893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00324d171dc0468fa259f13c1d4d74ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.690289 22948 tablet_bootstrap.cc:654] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.693619 22888 tablet_service.cc:1511] Processing CreateTablet for tablet 70588fa182854c9dbc786bdd4f807fc9 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:24.694796 22888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70588fa182854c9dbc786bdd4f807fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.697047 22742 tablet_service.cc:1511] Processing CreateTablet for tablet 70588fa182854c9dbc786bdd4f807fc9 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:24.698263 22742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70588fa182854c9dbc786bdd4f807fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.705448 22893 tablet_service.cc:1511] Processing CreateTablet for tablet 8da6707b1a3d47a48c3671a6b7401196 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:24.706772 22893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da6707b1a3d47a48c3671a6b7401196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.709111 22888 tablet_service.cc:1511] Processing CreateTablet for tablet d0d74135bf874a0bb4b857b686effcfd (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:24.672719 22740 tablet_service.cc:1511] Processing CreateTablet for tablet 4bbeaee00aa942ccb232c812c9e14827 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:24.710445 22888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d74135bf874a0bb4b857b686effcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.711108 22740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bbeaee00aa942ccb232c812c9e14827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.675437 22821 tablet_service.cc:1511] Processing CreateTablet for tablet 67449ba22ccb4938be8a5e28664cc566 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:24.671526 22813 tablet_service.cc:1511] Processing CreateTablet for tablet 09a9ba6091094973a526986ef573e2a9 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:24.715956 22821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67449ba22ccb4938be8a5e28664cc566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.716555 22813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a9ba6091094973a526986ef573e2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.678400 22738 tablet_service.cc:1511] Processing CreateTablet for tablet 00324d171dc0468fa259f13c1d4d74ea (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:24.717474 22738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00324d171dc0468fa259f13c1d4d74ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.719326 22746 tablet_service.cc:1511] Processing CreateTablet for tablet 8da6707b1a3d47a48c3671a6b7401196 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:24.720456 22746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da6707b1a3d47a48c3671a6b7401196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.720520 22951 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.721028 22951 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.723068 22951 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:24.724349 22812 tablet_service.cc:1511] Processing CreateTablet for tablet 469a8bc68a8b4e62b19f382c167cf50c (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:24.724553 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:24.676008 22814 tablet_service.cc:1511] Processing CreateTablet for tablet dbd3d6688dfb4621b8ecfd11278d5b26 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
W20260709 13:06:24.725891 22871 leader_election.cc:343] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.726482 22814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbd3d6688dfb4621b8ecfd11278d5b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.726392 22892 tablet_service.cc:1511] Processing CreateTablet for tablet 6ce4678cb66d4d1e8325b447eaf2a630 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:24.727137 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:24.727614 22892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ce4678cb66d4d1e8325b447eaf2a630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.727717 22831 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:24.672400 22818 tablet_service.cc:1511] Processing CreateTablet for tablet 60a957de2a744260812f8da6f1bd9b87 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:24.729012 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:24.729575 22818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a957de2a744260812f8da6f1bd9b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.732146 22812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469a8bc68a8b4e62b19f382c167cf50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.732525 22951 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:24.732890 22951 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.733261 22951 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.740777 22960 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.741250 22960 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.745194 22960 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:24.755043 22886 tablet_service.cc:1511] Processing CreateTablet for tablet 42c5a324a80f42d4965b69d86915a65d (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:24.755786 22946 tablet_bootstrap.cc:654] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.756249 22886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42c5a324a80f42d4965b69d86915a65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.763929 22945 tablet_bootstrap.cc:654] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:24.772693 22895 tablet_service.cc:1511] Processing CreateTablet for tablet 8ea49b0ee95d4eafbb647436a98fcaa1 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:24.789358 22820 tablet_service.cc:1511] Processing CreateTablet for tablet 2147efadd4ea46a09bda395402a10299 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:24.790921 22820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147efadd4ea46a09bda395402a10299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.800626 22740 tablet_service.cc:1511] Processing CreateTablet for tablet 6ce4678cb66d4d1e8325b447eaf2a630 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:24.809151 22743 tablet_service.cc:1511] Processing CreateTablet for tablet 42c5a324a80f42d4965b69d86915a65d (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:24.810894 22743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42c5a324a80f42d4965b69d86915a65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.854648 22815 tablet_service.cc:1511] Processing CreateTablet for tablet 18051671d25f4aa7b30d4a299909f91a (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:24.855993 22815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18051671d25f4aa7b30d4a299909f91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.856379 22744 tablet_service.cc:1511] Processing CreateTablet for tablet 8ea49b0ee95d4eafbb647436a98fcaa1 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:24.868986 22819 tablet_service.cc:1511] Processing CreateTablet for tablet 3583f8fda02d46af9ef0ff2ae6f1b443 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:24.869550 22887 tablet_service.cc:1511] Processing CreateTablet for tablet 9e12cb0ef4bd490aa6a0afd9cd76ff18 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:24.874686 22891 tablet_service.cc:1511] Processing CreateTablet for tablet a42e3464af3e4e5eb580790b400a6eff (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:24.875963 22895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea49b0ee95d4eafbb647436a98fcaa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.876601 22887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e12cb0ef4bd490aa6a0afd9cd76ff18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.877283 22744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea49b0ee95d4eafbb647436a98fcaa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.877394 22819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3583f8fda02d46af9ef0ff2ae6f1b443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.876750 22741 tablet_service.cc:1511] Processing CreateTablet for tablet d0d74135bf874a0bb4b857b686effcfd (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:24.887106 22740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ce4678cb66d4d1e8325b447eaf2a630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.889076 22891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a42e3464af3e4e5eb580790b400a6eff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:24.900497 22641 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 91696.7us
W20260709 13:06:24.906399 22638 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88171.6us
I20260709 13:06:24.907783 22739 tablet_service.cc:1511] Processing CreateTablet for tablet 9e12cb0ef4bd490aa6a0afd9cd76ff18 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:24.910931 22817 tablet_service.cc:1511] Processing CreateTablet for tablet e4cc99e6a1f14071a587af061e82fee0 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:24.911372 22890 tablet_service.cc:1511] Processing CreateTablet for tablet 1879799f5d0c462c9d2c95bd8a88862e (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:24.912123 22817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cc99e6a1f14071a587af061e82fee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.912735 22890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1879799f5d0c462c9d2c95bd8a88862e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.912550 22813 tablet_service.cc:1511] Processing CreateTablet for tablet 4bbeaee00aa942ccb232c812c9e14827 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:24.913834 22741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d74135bf874a0bb4b857b686effcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.915403 22888 tablet_service.cc:1511] Processing CreateTablet for tablet 3a98a0b933184c5590c80b44a6e3a544 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:24.916961 22893 tablet_service.cc:1511] Processing CreateTablet for tablet f8c8f692d4994a148c306609ab807ab7 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:24.922574 22948 tablet_bootstrap.cc:492] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:24.923255 22948 ts_tablet_manager.cc:1403] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.239s	user 0.010s	sys 0.014s
I20260709 13:06:24.927644 22892 tablet_service.cc:1511] Processing CreateTablet for tablet 6ce771dbb93248d397c8d5b222b27fca (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:24.932556 22739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e12cb0ef4bd490aa6a0afd9cd76ff18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.936060 22889 tablet_service.cc:1511] Processing CreateTablet for tablet 9cf0bfa7a80d4f0286947f1e6ff89772 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:24.898377 22951 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.947997 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:06:24.949371 22871 leader_election.cc:343] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:24.954238 22888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a98a0b933184c5590c80b44a6e3a544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.955819 22813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bbeaee00aa942ccb232c812c9e14827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.971115 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:24.971714 22831 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:24.976692 22889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cf0bfa7a80d4f0286947f1e6ff89772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.976778 22945 tablet_bootstrap.cc:492] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:24.956715 22737 tablet_service.cc:1511] Processing CreateTablet for tablet 3a98a0b933184c5590c80b44a6e3a544 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:24.987268 22737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a98a0b933184c5590c80b44a6e3a544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:24.988144 22831 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:24.989496 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:24.990908 22946 tablet_bootstrap.cc:492] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:24.991505 22946 ts_tablet_manager.cc:1403] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.355s	user 0.011s	sys 0.004s
I20260709 13:06:24.974525 22951 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:24.997820 22951 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:24.998302 22951 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:24.999145 22951 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.003201 22812 tablet_service.cc:1511] Processing CreateTablet for tablet 00324d171dc0468fa259f13c1d4d74ea (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:24.977056 22745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 39) took 1049 ms. Trace:
I20260709 13:06:24.983605 22948 raft_consensus.cc:359] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:24.985998 22743 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 48) took 1007 ms. Trace:
I20260709 13:06:25.015841 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.021453 22948 raft_consensus.cc:385] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.016690 22945 ts_tablet_manager.cc:1403] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.398s	user 0.012s	sys 0.015s
I20260709 13:06:25.018939 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:24.965389 22742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 44) took 1018 ms. Trace:
I20260709 13:06:25.025630 22742 rpcz_store.cc:276] 0709 13:06:23.947304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.696150 (+748846us) service_pool.cc:224] Handling call
0709 13:06:24.697842 (+  1692us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.697925 (+    83us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:24.733757 (+ 35832us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:24.961685 (+227928us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:24.965305 (+  3620us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.027544 22738 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 43) took 1089 ms. Trace:
I20260709 13:06:25.027870 22742 tablet_service.cc:1511] Processing CreateTablet for tablet f8c8f692d4994a148c306609ab807ab7 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:25.027969 22738 rpcz_store.cc:276] 0709 13:06:23.938643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.677572 (+738929us) service_pool.cc:224] Handling call
0709 13:06:24.717057 (+ 39485us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.717135 (+    78us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:24.744219 (+ 27084us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.027073 (+282854us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.027490 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:25.025224 22797 leader_election.cc:343] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.029122 22742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8c8f692d4994a148c306609ab807ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.021821 22948 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.024431 22743 rpcz_store.cc:276] 0709 13:06:23.979074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.808135 (+829061us) service_pool.cc:224] Handling call
0709 13:06:24.810454 (+  2319us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.810537 (+    83us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:24.943181 (+132644us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:24.985220 (+ 42039us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:24.985865 (+   645us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.033653 22893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8c8f692d4994a148c306609ab807ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.039565 22743 tablet_service.cc:1511] Processing CreateTablet for tablet 1879799f5d0c462c9d2c95bd8a88862e (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260709 13:06:25.037696 22794 leader_election.cc:343] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.041019 22895 tablet_service.cc:1511] Processing CreateTablet for tablet a28fec9b1516463db9c7feac432f9ad5 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:25.037755 22948 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.042259 22895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a28fec9b1516463db9c7feac432f9ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.043721 22948 ts_tablet_manager.cc:1434] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.120s	user 0.007s	sys 0.002s
I20260709 13:06:25.050034 22737 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 51) took 1061 ms. Trace:
I20260709 13:06:25.053300 22743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1879799f5d0c462c9d2c95bd8a88862e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.056469 22948 tablet_bootstrap.cc:492] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.059101 22894 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43992 (request call id 40) took 1070 ms. Trace:
I20260709 13:06:25.059510 22741 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 46) took 1102 ms. Trace:
I20260709 13:06:25.021948 22745 rpcz_store.cc:276] 0709 13:06:23.927878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.630954 (+703076us) service_pool.cc:224] Handling call
0709 13:06:24.633010 (+  2056us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.633110 (+   100us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:24.764541 (+131431us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:24.957379 (+192838us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:24.975931 (+ 18552us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.060034 22741 rpcz_store.cc:276] 0709 13:06:23.957699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.779989 (+822290us) service_pool.cc:224] Handling call
0709 13:06:24.897507 (+117518us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.913498 (+ 15991us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.005789 (+ 92291us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.058954 (+ 53165us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.059450 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.063356 22820 tablet_service.cc:1511] Processing CreateTablet for tablet 70588fa182854c9dbc786bdd4f807fc9 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:25.064512 22820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70588fa182854c9dbc786bdd4f807fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.066830 22892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ce771dbb93248d397c8d5b222b27fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.067679 22812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00324d171dc0468fa259f13c1d4d74ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.069978 22948 tablet_bootstrap.cc:654] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.051303 22945 raft_consensus.cc:359] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.070394 22741 tablet_service.cc:1511] Processing CreateTablet for tablet 6ce771dbb93248d397c8d5b222b27fca (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:25.071027 22945 raft_consensus.cc:385] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.071429 22945 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.071624 22741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ce771dbb93248d397c8d5b222b27fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.072366 22945 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.073217 22814 tablet_service.cc:1511] Processing CreateTablet for tablet 8da6707b1a3d47a48c3671a6b7401196 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:25.074393 22814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da6707b1a3d47a48c3671a6b7401196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.044703 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.059487 22894 rpcz_store.cc:276] 0709 13:06:23.988724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.623056 (+634332us) service_pool.cc:224] Handling call
0709 13:06:24.624758 (+  1702us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.624852 (+    94us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.034283 (+409431us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.058645 (+ 24362us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.059046 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.077847 22960 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:25.077996 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48485.4us
I20260709 13:06:25.078555 22738 tablet_service.cc:1511] Processing CreateTablet for tablet 9cf0bfa7a80d4f0286947f1e6ff89772 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:25.079406 22946 raft_consensus.cc:359] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.079873 22738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cf0bfa7a80d4f0286947f1e6ff89772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.080120 22946 raft_consensus.cc:385] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.080439 22946 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.087479 22739 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 50) took 1100 ms. Trace:
I20260709 13:06:25.087867 22739 rpcz_store.cc:276] 0709 13:06:23.986908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.890328 (+903420us) service_pool.cc:224] Handling call
0709 13:06:24.931997 (+ 41669us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.932119 (+   122us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:24.988061 (+ 55942us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.087018 (+ 98957us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.087420 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.090462 22745 tablet_service.cc:1511] Processing CreateTablet for tablet a42e3464af3e4e5eb580790b400a6eff (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:25.092146 22745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a42e3464af3e4e5eb580790b400a6eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.095947 22817 tablet_service.cc:1511] Processing CreateTablet for tablet 42c5a324a80f42d4965b69d86915a65d (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:25.096896 22891 tablet_service.cc:1511] Processing CreateTablet for tablet efec6b734c374a169f3dc3b729ef2f07 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:25.105659 22960 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.097173 22817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42c5a324a80f42d4965b69d86915a65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.114871 22969 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.114677 22819 tablet_service.cc:1511] Processing CreateTablet for tablet 8ea49b0ee95d4eafbb647436a98fcaa1 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:25.115309 22969 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.115963 22819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea49b0ee95d4eafbb647436a98fcaa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.118069 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:25.119488 22871 leader_election.cc:343] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.120380 22969 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:25.120975 22891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efec6b734c374a169f3dc3b729ef2f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.106225 22960 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.122641 22960 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.130635 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:25.131969 22797 leader_election.cc:343] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.134375 22818 tablet_service.cc:1511] Processing CreateTablet for tablet 3a98a0b933184c5590c80b44a6e3a544 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:25.135558 22818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a98a0b933184c5590c80b44a6e3a544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.140425 22746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 45) took 1189 ms. Trace:
I20260709 13:06:25.140801 22746 rpcz_store.cc:276] 0709 13:06:23.951427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.718484 (+767057us) service_pool.cc:224] Handling call
0709 13:06:24.720052 (+  1568us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.720137 (+    85us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:24.957268 (+237131us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.139978 (+182710us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.140366 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.142851 22746 tablet_service.cc:1511] Processing CreateTablet for tablet efec6b734c374a169f3dc3b729ef2f07 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:25.144070 22746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efec6b734c374a169f3dc3b729ef2f07. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:25.144507 22702 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:25.146075 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:25.147408 22794 leader_election.cc:343] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.147835 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.148540 22960 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:25.081161 22946 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:25.113483 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 124087us
I20260709 13:06:25.095947 22816 tablet_service.cc:1511] Processing CreateTablet for tablet d0d74135bf874a0bb4b857b686effcfd (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:25.051813 22737 rpcz_store.cc:276] 0709 13:06:23.988463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.916607 (+928144us) service_pool.cc:224] Handling call
0709 13:06:24.986623 (+ 70016us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.986745 (+   122us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.000867 (+ 14122us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.034068 (+ 33201us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.049978 (+ 15910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.166100 22654 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:25.157380 22815 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 39) took 1002 ms. Trace:
I20260709 13:06:25.172254 22815 rpcz_store.cc:276] 0709 13:06:24.154972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.814695 (+659723us) service_pool.cc:224] Handling call
0709 13:06:24.855493 (+ 40798us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.855611 (+   118us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.086015 (+230404us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.153014 (+ 66999us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.156085 (+  3071us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.177479 22815 tablet_service.cc:1511] Processing CreateTablet for tablet f8c8f692d4994a148c306609ab807ab7 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:25.177764 22821 tablet_service.cc:1511] Processing CreateTablet for tablet 9e12cb0ef4bd490aa6a0afd9cd76ff18 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:25.180994 22816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d74135bf874a0bb4b857b686effcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.162369 22739 tablet_service.cc:1511] Processing CreateTablet for tablet a28fec9b1516463db9c7feac432f9ad5 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:25.182057 22739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a28fec9b1516463db9c7feac432f9ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.186496 22742 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 52) took 1192 ms. Trace:
I20260709 13:06:25.186959 22742 rpcz_store.cc:276] 0709 13:06:23.993729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.026936 (+1033207us) service_pool.cc:224] Handling call
0709 13:06:25.028691 (+  1755us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.028775 (+    84us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.080917 (+ 52142us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.186040 (+105123us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.186441 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.188158 22813 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 42) took 1032 ms. Trace:
I20260709 13:06:25.188565 22813 rpcz_store.cc:276] 0709 13:06:24.155789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.911704 (+755915us) service_pool.cc:224] Handling call
0709 13:06:24.913282 (+  1578us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.938885 (+ 25603us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:24.995887 (+ 57002us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.163905 (+168018us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.188099 (+ 24194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.190436 22813 tablet_service.cc:1511] Processing CreateTablet for tablet 1879799f5d0c462c9d2c95bd8a88862e (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:25.191731 22813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1879799f5d0c462c9d2c95bd8a88862e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.203684 22815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8c8f692d4994a148c306609ab807ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.186614 22741 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 55) took 1159 ms. Trace:
I20260709 13:06:25.198622 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:25.204520 22821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e12cb0ef4bd490aa6a0afd9cd76ff18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.205839 22948 tablet_bootstrap.cc:492] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.206357 22948 ts_tablet_manager.cc:1403] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.150s	user 0.024s	sys 0.004s
W20260709 13:06:25.209583 22868 leader_election.cc:343] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.210065 22868 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:25.210893 22972 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:25.210201 22741 rpcz_store.cc:276] 0709 13:06:24.027287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.069476 (+1042189us) service_pool.cc:224] Handling call
0709 13:06:25.071184 (+  1708us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.071272 (+    88us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.103752 (+ 32480us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.186202 (+ 82450us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.186563 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.193542 22744 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 49) took 1207 ms. Trace:
I20260709 13:06:25.218466 22743 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 53) took 1219 ms. Trace:
I20260709 13:06:25.220894 22812 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 43) took 1064 ms. Trace:
I20260709 13:06:25.231077 22946 ts_tablet_manager.cc:1434] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.239s	user 0.012s	sys 0.041s
I20260709 13:06:25.230024 22812 rpcz_store.cc:276] 0709 13:06:24.156069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.002246 (+846177us) service_pool.cc:224] Handling call
0709 13:06:25.035907 (+ 33661us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.050608 (+ 14701us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.134600 (+ 83992us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.220459 (+ 85859us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.220841 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37711}
I20260709 13:06:25.219939 22744 rpcz_store.cc:276] 0709 13:06:23.986229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.855484 (+869255us) service_pool.cc:224] Handling call
0709 13:06:24.857121 (+  1637us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.857214 (+    93us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.073530 (+216316us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.193062 (+119532us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.193487 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.229981 22743 rpcz_store.cc:276] 0709 13:06:23.999017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.038688 (+1039671us) service_pool.cc:224] Handling call
0709 13:06:25.040339 (+  1651us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.040422 (+    83us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.062719 (+ 22297us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.218012 (+155293us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.218412 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.223510 22893 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43992 (request call id 52) took 1040 ms. Trace:
I20260709 13:06:25.238696 22945 ts_tablet_manager.cc:1434] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.207s	user 0.013s	sys 0.000s
I20260709 13:06:25.239059 22814 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 45) took 1082 ms. Trace:
I20260709 13:06:25.239061 22817 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 48) took 1081 ms. Trace:
I20260709 13:06:25.233196 22812 tablet_service.cc:1511] Processing CreateTablet for tablet a42e3464af3e4e5eb580790b400a6eff (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:25.240657 22818 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 51) took 1055 ms. Trace:
I20260709 13:06:25.238025 22893 rpcz_store.cc:276] 0709 13:06:24.183271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.908818 (+725547us) service_pool.cc:224] Handling call
0709 13:06:24.917823 (+  9005us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.917903 (+    80us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.197693 (+279790us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.223074 (+ 25381us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.223454 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.239739 22814 rpcz_store.cc:276] 0709 13:06:24.156727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.072341 (+915614us) service_pool.cc:224] Handling call
0709 13:06:25.073985 (+  1644us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.074064 (+    79us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.211647 (+137583us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.234273 (+ 22626us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.238937 (+  4664us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5470}
I20260709 13:06:25.240031 22817 rpcz_store.cc:276] 0709 13:06:24.157642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.095101 (+937459us) service_pool.cc:224] Handling call
0709 13:06:25.096724 (+  1623us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.096812 (+    88us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.180493 (+ 83681us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.231558 (+ 51065us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.239005 (+  7447us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26200}
I20260709 13:06:25.243702 22814 tablet_service.cc:1511] Processing CreateTablet for tablet 6ce771dbb93248d397c8d5b222b27fca (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:25.244200 22890 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43992 (request call id 53) took 1059 ms. Trace:
I20260709 13:06:25.238736 22819 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 49) took 1080 ms. Trace:
I20260709 13:06:25.213308 22740 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 47) took 1234 ms. Trace:
I20260709 13:06:25.244823 22817 tablet_service.cc:1511] Processing CreateTablet for tablet 9cf0bfa7a80d4f0286947f1e6ff89772 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:25.241273 22818 rpcz_store.cc:276] 0709 13:06:24.184878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.133493 (+948615us) service_pool.cc:224] Handling call
0709 13:06:25.135114 (+  1621us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.135212 (+    98us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.192295 (+ 57083us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.235202 (+ 42907us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.240602 (+  5400us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2542}
I20260709 13:06:25.246650 22889 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43992 (request call id 56) took 1050 ms. Trace:
I20260709 13:06:25.247051 22889 rpcz_store.cc:276] 0709 13:06:24.196214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.935185 (+738971us) service_pool.cc:224] Handling call
0709 13:06:24.940171 (+  4986us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.949610 (+  9439us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.123178 (+173568us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.246200 (+123022us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.246596 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.239780 22820 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 44) took 1083 ms. Trace:
I20260709 13:06:25.247893 22820 rpcz_store.cc:276] 0709 13:06:24.156467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.062509 (+906042us) service_pool.cc:224] Handling call
0709 13:06:25.064083 (+  1574us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.064171 (+    88us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.083998 (+ 19827us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.234733 (+150735us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.239725 (+  4992us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8363}
I20260709 13:06:25.209182 22948 raft_consensus.cc:359] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.249076 22948 raft_consensus.cc:385] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.249409 22948 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.250195 22948 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.250419 22818 tablet_service.cc:1511] Processing CreateTablet for tablet a28fec9b1516463db9c7feac432f9ad5 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:25.251464 22945 tablet_bootstrap.cc:492] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:25.252422 22820 tablet_service.cc:1511] Processing CreateTablet for tablet efec6b734c374a169f3dc3b729ef2f07 (DEFAULT_TABLE table=test-workload [id=f093a267312f4a1898f3cb161cef46b7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:25.253523 22946 tablet_bootstrap.cc:492] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.245096 22960 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.241386 22812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a42e3464af3e4e5eb580790b400a6eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.256568 22960 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.257217 22817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cf0bfa7a80d4f0286947f1e6ff89772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.259060 22818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a28fec9b1516463db9c7feac432f9ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.259634 22960 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.260668 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.261240 22905 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:25.261271 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.262470 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:25.245102 22740 rpcz_store.cc:276] 0709 13:06:23.978631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.799749 (+821118us) service_pool.cc:224] Handling call
0709 13:06:24.863043 (+ 63294us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.866208 (+  3165us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.083314 (+217106us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.212664 (+129350us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.213226 (+   562us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:25.263195 22797 leader_election.cc:343] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.264115 22960 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.264489 22960 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.264685 22813 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 53) took 1079 ms. Trace:
I20260709 13:06:25.264847 22960 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.265503 22821 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 50) took 1107 ms. Trace:
I20260709 13:06:25.265892 22821 rpcz_store.cc:276] 0709 13:06:24.158401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.133222 (+974821us) service_pool.cc:224] Handling call
0709 13:06:25.203984 (+ 70762us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b5800144f98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e2f831 00007fcb79e0cac3 00007fcb79dbb400 00007fcb73240af4 00007fcb73240a82 00007fcb732409b1
0709 13:06:25.204108 (+   124us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.204199 (+    91us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.223762 (+ 19563us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.253070 (+ 29308us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.265443 (+ 12373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56985984}
I20260709 13:06:25.265084 22813 rpcz_store.cc:276] 0709 13:06:24.185519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.189575 (+1004056us) service_pool.cc:224] Handling call
0709 13:06:25.191216 (+  1641us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.191399 (+   183us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.249014 (+ 57615us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.264220 (+ 15206us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.264629 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.244585 22890 rpcz_store.cc:276] 0709 13:06:24.184398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.910526 (+726128us) service_pool.cc:224] Handling call
0709 13:06:24.912262 (+  1736us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.912408 (+   146us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.137526 (+225118us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.243701 (+106175us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.244149 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.274075 22960 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.276225 22960 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.277298 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:25.279354 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:25.244977 22819 rpcz_store.cc:276] 0709 13:06:24.158061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.113768 (+955707us) service_pool.cc:224] Handling call
0709 13:06:25.115495 (+  1727us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.115593 (+    98us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.132688 (+ 17095us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.174279 (+ 41591us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.238664 (+ 64385us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1563}
I20260709 13:06:25.279860 22814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ce771dbb93248d397c8d5b222b27fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.279901 22905 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.281348 22820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efec6b734c374a169f3dc3b729ef2f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:25.282454 22948 ts_tablet_manager.cc:1434] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.000s
I20260709 13:06:25.283449 22948 tablet_bootstrap.cc:492] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
W20260709 13:06:25.280356 22797 leader_election.cc:343] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.274097 22945 tablet_bootstrap.cc:654] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.292753 22815 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 52) took 1107 ms. Trace:
I20260709 13:06:25.302515 22739 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 57) took 1273 ms. Trace:
I20260709 13:06:25.297943 22746 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 58) took 1263 ms. Trace:
I20260709 13:06:25.296303 22815 rpcz_store.cc:276] 0709 13:06:24.185175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.176574 (+991399us) service_pool.cc:224] Handling call
0709 13:06:25.203115 (+ 26541us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b5800144f98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e2f831 00007fcb79e0cac3 00007fcb79dbb400 00007fcb73240af4 00007fcb73240a82 00007fcb732409b1
0709 13:06:25.203242 (+   127us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.203340 (+    98us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.245977 (+ 42637us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.292210 (+ 46233us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.292686 (+   476us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56219904}
I20260709 13:06:25.304587 22746 rpcz_store.cc:276] 0709 13:06:24.033504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.141950 (+1108446us) service_pool.cc:224] Handling call
0709 13:06:25.143638 (+  1688us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.143732 (+    94us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.230382 (+ 86650us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.292419 (+ 62037us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.297223 (+  4804us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.302807 22948 tablet_bootstrap.cc:654] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.302893 22739 rpcz_store.cc:276] 0709 13:06:24.029311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.088801 (+1059490us) service_pool.cc:224] Handling call
0709 13:06:25.181619 (+ 92818us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.181704 (+    85us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.279285 (+ 97581us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.302024 (+ 22739us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.302463 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.310942 22745 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 54) took 1308 ms. Trace:
I20260709 13:06:25.311520 22745 rpcz_store.cc:276] 0709 13:06:24.002965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.060694 (+1057729us) service_pool.cc:224] Handling call
0709 13:06:25.091435 (+ 30741us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.091555 (+   120us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.263989 (+172434us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.310239 (+ 46250us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.310870 (+   631us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.312907 22892 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43992 (request call id 55) took 1117 ms. Trace:
I20260709 13:06:25.295583 22738 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38476 (request call id 56) took 1268 ms. Trace:
I20260709 13:06:25.313267 22892 rpcz_store.cc:276] 0709 13:06:24.195279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.926531 (+731252us) service_pool.cc:224] Handling call
0709 13:06:24.928616 (+  2085us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:24.928725 (+   109us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.250465 (+321740us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.312375 (+ 61910us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.312853 (+   478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.313712 22738 rpcz_store.cc:276] 0709 13:06:24.027587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.077435 (+1049848us) service_pool.cc:224] Handling call
0709 13:06:25.079450 (+  2015us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.079534 (+    84us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.147825 (+ 68291us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.259503 (+111678us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.295520 (+ 36017us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.319535 22946 tablet_bootstrap.cc:654] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.326980 22905 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:25.329107 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.331040 22948 tablet_bootstrap.cc:492] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.330915 22976 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.331662 22976 raft_consensus.cc:697] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:25.332487 22976 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.336475 22945 tablet_bootstrap.cc:492] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:25.337531 22948 ts_tablet_manager.cc:1403] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.000s
I20260709 13:06:25.338565 22816 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 46) took 1181 ms. Trace:
I20260709 13:06:25.338917 22816 rpcz_store.cc:276] 0709 13:06:24.157031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.094485 (+937454us) service_pool.cc:224] Handling call
0709 13:06:25.170270 (+ 75785us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.173832 (+  3562us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.255460 (+ 81628us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.338063 (+ 82603us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.338504 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.340916 22818 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 57) took 1138 ms. Trace:
I20260709 13:06:25.341259 22818 rpcz_store.cc:276] 0709 13:06:24.202724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.249542 (+1046818us) service_pool.cc:224] Handling call
0709 13:06:25.251184 (+  1642us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.251288 (+   104us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.317926 (+ 66638us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.340488 (+ 22562us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.340862 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.343566 22812 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 54) took 1141 ms. Trace:
I20260709 13:06:25.344452 22817 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 56) took 1142 ms. Trace:
I20260709 13:06:25.344310 22948 raft_consensus.cc:359] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.345021 22948 raft_consensus.cc:385] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.344765 22817 rpcz_store.cc:276] 0709 13:06:24.202435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.242842 (+1040407us) service_pool.cc:224] Handling call
0709 13:06:25.245900 (+  3058us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.245982 (+    82us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.311903 (+ 65921us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.337511 (+ 25608us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.344410 (+  6899us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.345454 22948 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.343871 22812 rpcz_store.cc:276] 0709 13:06:24.201860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.232127 (+1030267us) service_pool.cc:224] Handling call
0709 13:06:25.240752 (+  8625us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.240866 (+   114us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.330456 (+ 89590us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.343139 (+ 12683us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.343512 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.337002 22945 ts_tablet_manager.cc:1403] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.097s	user 0.014s	sys 0.011s
I20260709 13:06:25.347124 22820 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 58) took 1133 ms. Trace:
I20260709 13:06:25.347491 22820 rpcz_store.cc:276] 0709 13:06:24.213854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.251607 (+1037753us) service_pool.cc:224] Handling call
0709 13:06:25.253130 (+  1523us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.253221 (+    91us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.322184 (+ 68963us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.346667 (+ 24483us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.347070 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:25.348831 22945 raft_consensus.cc:359] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.349445 22945 raft_consensus.cc:385] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.349681 22945 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.349967 22948 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.350302 22945 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.352770 22945 ts_tablet_manager.cc:1434] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:06:25.353591 22948 ts_tablet_manager.cc:1434] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:06:25.354234 22814 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43622 (request call id 55) took 1152 ms. Trace:
I20260709 13:06:25.354408 22945 tablet_bootstrap.cc:492] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:25.354612 22948 tablet_bootstrap.cc:492] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.354669 22814 rpcz_store.cc:276] 0709 13:06:24.202158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.242662 (+1040504us) service_pool.cc:224] Handling call
0709 13:06:25.244725 (+  2063us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:25.244943 (+   218us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:25.346706 (+101763us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:25.353760 (+  7054us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:25.354173 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6400}
I20260709 13:06:25.362766 22651 catalog_manager.cc:5697] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.366925 22946 tablet_bootstrap.cc:492] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.367421 22946 ts_tablet_manager.cc:1403] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.114s	user 0.014s	sys 0.001s
I20260709 13:06:25.373565 22945 tablet_bootstrap.cc:654] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.374755 22946 raft_consensus.cc:359] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.375576 22946 raft_consensus.cc:385] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.375922 22946 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.376905 22946 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.378196 22948 tablet_bootstrap.cc:654] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.379913 22946 ts_tablet_manager.cc:1434] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 13:06:25.381255 22946 tablet_bootstrap.cc:492] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.389555 22946 tablet_bootstrap.cc:654] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.395033 22946 tablet_bootstrap.cc:492] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.395267 22945 tablet_bootstrap.cc:492] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:25.395371 22946 ts_tablet_manager.cc:1403] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.014s	user 0.004s	sys 0.009s
I20260709 13:06:25.395788 22945 ts_tablet_manager.cc:1403] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.042s	user 0.003s	sys 0.023s
I20260709 13:06:25.396850 22948 tablet_bootstrap.cc:492] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.397406 22948 ts_tablet_manager.cc:1403] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.006s
I20260709 13:06:25.397523 22946 raft_consensus.cc:359] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.398036 22946 raft_consensus.cc:385] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.398226 22946 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.398775 22946 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.398696 22945 raft_consensus.cc:359] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.399509 22945 raft_consensus.cc:385] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.399808 22945 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.400787 22946 ts_tablet_manager.cc:1434] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:06:25.400589 22945 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.400643 22948 raft_consensus.cc:359] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.401660 22948 raft_consensus.cc:385] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.401969 22948 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.402083 22946 tablet_bootstrap.cc:492] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.402233 22976 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.402695 22976 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.402734 22948 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.404772 22976 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:25.405001 22948 ts_tablet_manager.cc:1434] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:06:25.406006 22948 tablet_bootstrap.cc:492] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.408988 22946 tablet_bootstrap.cc:654] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.409636 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.410699 22960 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:25.412083 22797 leader_election.cc:343] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.411154 22960 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.413240 22945 ts_tablet_manager.cc:1434] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.004s
I20260709 13:06:25.414393 22960 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.409262 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.415433 22948 tablet_bootstrap.cc:654] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:25.416373 22794 leader_election.cc:343] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.416814 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.417946 22945 tablet_bootstrap.cc:492] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:25.417701 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.420113 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:25.422123 22797 leader_election.cc:343] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.426409 22948 tablet_bootstrap.cc:492] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.426410 22946 tablet_bootstrap.cc:492] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.427032 22948 ts_tablet_manager.cc:1403] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 13:06:25.427032 22946 ts_tablet_manager.cc:1403] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.005s
W20260709 13:06:25.427899 22794 leader_election.cc:343] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.428602 22794 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.429710 22948 raft_consensus.cc:359] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.430027 22960 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:25.430461 22976 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:25.430761 22948 raft_consensus.cc:385] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.431095 22948 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.432190 22976 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.431922 22948 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.432327 22946 raft_consensus.cc:359] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.433025 22946 raft_consensus.cc:385] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.433331 22946 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.434190 22948 ts_tablet_manager.cc:1434] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:25.434106 22946 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.432698 22976 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.436518 22756 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.437070 22946 ts_tablet_manager.cc:1434] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:06:25.438618 22976 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.439688 22945 tablet_bootstrap.cc:654] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.439607 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.440248 22904 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:25.443126 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.443660 22946 tablet_bootstrap.cc:492] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
W20260709 13:06:25.444674 22797 leader_election.cc:343] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.444800 22972 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.451661 22946 tablet_bootstrap.cc:654] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.453004 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.454095 22976 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.454449 22976 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.454789 22976 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.464401 22976 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.466156 22948 tablet_bootstrap.cc:492] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.467779 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:06:25.469372 22797 leader_election.cc:343] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.473184 22972 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.475142 22948 tablet_bootstrap.cc:654] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.478296 22976 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.479384 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:25.479775 22904 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.487062 22904 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:25.489992 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.490931 22976 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.491395 22976 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:25.491652 22946 tablet_bootstrap.cc:492] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.492089 22946 ts_tablet_manager.cc:1403] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.020s
I20260709 13:06:25.492162 22976 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.494829 22946 raft_consensus.cc:359] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.495616 22945 tablet_bootstrap.cc:492] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:25.495615 22946 raft_consensus.cc:385] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.496315 22948 tablet_bootstrap.cc:492] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.496230 22946 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.496380 22945 ts_tablet_manager.cc:1403] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.079s	user 0.008s	sys 0.020s
I20260709 13:06:25.496938 22948 ts_tablet_manager.cc:1403] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 13:06:25.497685 22946 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.499310 22948 raft_consensus.cc:359] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.499966 22948 raft_consensus.cc:385] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.499586 22945 raft_consensus.cc:359] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.500231 22948 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.500398 22945 raft_consensus.cc:385] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.500787 22945 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.501713 22945 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.504021 22948 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.506065 22945 ts_tablet_manager.cc:1434] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:25.507431 22945 tablet_bootstrap.cc:492] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:25.507663 22946 ts_tablet_manager.cc:1434] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 13:06:25.508256 22948 ts_tablet_manager.cc:1434] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:06:25.511759 22946 tablet_bootstrap.cc:492] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.509248 22948 tablet_bootstrap.cc:492] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.517078 22945 tablet_bootstrap.cc:654] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.518787 22974 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.519254 22974 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.520524 22946 tablet_bootstrap.cc:654] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.521232 22974 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.523547 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.524169 22904 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:25.525286 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:25.526189 22974 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.526243 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:25.526726 22974 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.527127 22974 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:25.527598 22719 leader_election.cc:343] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.529162 22948 tablet_bootstrap.cc:654] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.535341 22974 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.538800 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:25.540195 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:25.540934 22904 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:25.545282 22719 leader_election.cc:343] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.546346 22651 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.556052 22974 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.555485 22904 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:25.558099 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:25.559646 22974 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.561360 22972 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.561976 22972 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.564611 22972 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:25.566737 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:25.567552 22831 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:25.569821 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.570605 22756 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:25.572570 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:25.574417 22972 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.574950 22972 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.575433 22972 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.583994 22946 tablet_bootstrap.cc:492] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.584146 22945 tablet_bootstrap.cc:492] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:25.584550 22946 ts_tablet_manager.cc:1403] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.073s	user 0.023s	sys 0.003s
I20260709 13:06:25.584628 22945 ts_tablet_manager.cc:1403] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.077s	user 0.035s	sys 0.031s
I20260709 13:06:25.584908 22948 tablet_bootstrap.cc:492] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.585368 22948 ts_tablet_manager.cc:1403] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.076s	user 0.032s	sys 0.027s
I20260709 13:06:25.586308 22972 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.587383 22946 raft_consensus.cc:359] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.589136 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:25.589648 22974 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:25.589874 22756 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.589963 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:25.590557 22831 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.590498 22974 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.587847 22948 raft_consensus.cc:359] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.593569 22948 raft_consensus.cc:385] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.593909 22948 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.594642 22948 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.596818 22948 ts_tablet_manager.cc:1434] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.000s
I20260709 13:06:25.589139 22946 raft_consensus.cc:385] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.597084 22756 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:25.597240 22946 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.587683 22945 raft_consensus.cc:359] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.598315 22945 raft_consensus.cc:385] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.598414 22871 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:25.598675 22945 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.599298 22951 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.599823 22951 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:25.599519 22945 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.599435 22948 tablet_bootstrap.cc:492] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.588313 22972 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:25.601783 22946 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.600660 22951 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.604262 22946 ts_tablet_manager.cc:1434] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:06:25.605368 22946 tablet_bootstrap.cc:492] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.606602 22831 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:25.608717 22974 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.609088 22948 tablet_bootstrap.cc:654] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.609200 22974 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.611249 22974 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.612375 22945 ts_tablet_manager.cc:1434] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:06:25.612818 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:25.613301 22945 tablet_bootstrap.cc:492] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:25.613708 22830 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:25.615444 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.616206 22904 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:25.618551 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:25.618201 22651 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.619699 22945 tablet_bootstrap.cc:654] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.619629 22974 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.620271 22974 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.620707 22974 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.624709 22946 tablet_bootstrap.cc:654] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.635646 22974 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.636168 23001 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.636871 23001 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.639381 23001 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.641502 22974 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.644505 22948 tablet_bootstrap.cc:492] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.644866 22948 ts_tablet_manager.cc:1403] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.019s
I20260709 13:06:25.646708 22948 raft_consensus.cc:359] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.647176 22948 raft_consensus.cc:385] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.647390 22948 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.647964 22948 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.649642 22948 ts_tablet_manager.cc:1434] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.003s
I20260709 13:06:25.650650 22948 tablet_bootstrap.cc:492] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.655328 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:25.655918 22905 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.643694 22654 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:25.655331 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:25.661962 22830 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:25.663139 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:25.663419 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:25.664232 22831 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.665268 22905 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:25.666679 22948 tablet_bootstrap.cc:654] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.655619 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.667366 22904 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:25.669538 22974 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.670104 22974 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.673599 22946 tablet_bootstrap.cc:492] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.673861 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:25.674700 22974 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.674969 23001 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.675678 22946 ts_tablet_manager.cc:1403] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.018s
I20260709 13:06:25.676528 22831 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:25.677958 22948 tablet_bootstrap.cc:492] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.678372 22948 ts_tablet_manager.cc:1403] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.009s
I20260709 13:06:25.678193 22946 raft_consensus.cc:359] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.678755 22946 raft_consensus.cc:385] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.678982 22946 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.679543 22946 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.681031 22948 raft_consensus.cc:359] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.682022 22948 raft_consensus.cc:385] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.682494 22948 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.684007 22948 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.686368 22945 tablet_bootstrap.cc:492] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:25.686847 22945 ts_tablet_manager.cc:1403] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.020s
I20260709 13:06:25.687462 22948 ts_tablet_manager.cc:1434] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:06:25.688330 23001 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:25.689374 22945 raft_consensus.cc:359] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.690037 22945 raft_consensus.cc:385] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.690308 22945 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.690810 22946 ts_tablet_manager.cc:1434] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:06:25.691085 22948 tablet_bootstrap.cc:492] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.691023 22945 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.691824 22946 tablet_bootstrap.cc:492] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.693130 22945 ts_tablet_manager.cc:1434] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:25.693980 22945 tablet_bootstrap.cc:492] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:25.697281 22948 tablet_bootstrap.cc:654] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.698509 22946 tablet_bootstrap.cc:654] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.699977 22945 tablet_bootstrap.cc:654] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.705206 23001 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.708244 22948 tablet_bootstrap.cc:492] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.708679 22948 ts_tablet_manager.cc:1403] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.018s	user 0.004s	sys 0.012s
I20260709 13:06:25.711170 22948 raft_consensus.cc:359] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.711786 22948 raft_consensus.cc:385] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.712039 22948 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.712635 22948 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.715809 22946 tablet_bootstrap.cc:492] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.716301 22946 ts_tablet_manager.cc:1403] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.011s
I20260709 13:06:25.718847 22945 tablet_bootstrap.cc:492] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:25.716974 22948 ts_tablet_manager.cc:1434] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:06:25.718952 22946 raft_consensus.cc:359] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.719974 22946 raft_consensus.cc:385] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.719882 22945 ts_tablet_manager.cc:1403] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.012s
I20260709 13:06:25.720552 22946 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.721405 22946 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.720060 22948 tablet_bootstrap.cc:492] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.723817 22946 ts_tablet_manager.cc:1434] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:25.724465 22945 raft_consensus.cc:359] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.725069 22946 tablet_bootstrap.cc:492] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.725306 22945 raft_consensus.cc:385] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.725625 22945 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.726591 22945 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.729353 22948 tablet_bootstrap.cc:654] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.731191 22946 tablet_bootstrap.cc:654] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.732092 22945 ts_tablet_manager.cc:1434] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:06:25.734455 22945 tablet_bootstrap.cc:492] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:25.739262 22974 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.743633 22946 tablet_bootstrap.cc:492] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.744138 22946 ts_tablet_manager.cc:1403] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 13:06:25.746804 22946 raft_consensus.cc:359] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.747480 22946 raft_consensus.cc:385] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.747812 22946 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.748562 22946 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.750715 22946 ts_tablet_manager.cc:1434] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:25.751008 22945 tablet_bootstrap.cc:654] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.751703 22946 tablet_bootstrap.cc:492] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.752251 22976 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.752836 22976 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.754144 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:25.754855 22831 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.756258 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.749461 22653 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:25.757653 22794 leader_election.cc:343] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.758975 22976 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.759171 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:25.760879 22797 leader_election.cc:343] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.761463 22797 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.762847 22976 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:25.767091 22946 tablet_bootstrap.cc:654] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.767709 22831 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:25.769147 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:25.770267 23001 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.770457 22945 tablet_bootstrap.cc:492] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:25.771178 22945 ts_tablet_manager.cc:1403] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.009s
I20260709 13:06:25.771108 23001 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:25.772291 23001 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.773898 22945 raft_consensus.cc:359] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.774535 22945 raft_consensus.cc:385] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.778236 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:25.778950 22904 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.774808 22945 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.783771 22945 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.788528 22904 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:25.790475 22974 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.791595 22948 tablet_bootstrap.cc:492] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.792011 22948 ts_tablet_manager.cc:1403] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.072s	user 0.024s	sys 0.033s
I20260709 13:06:25.794639 22948 raft_consensus.cc:359] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.795256 22948 raft_consensus.cc:385] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.795502 22948 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.796096 22948 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.799681 22945 ts_tablet_manager.cc:1434] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:06:25.800611 22948 ts_tablet_manager.cc:1434] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:06:25.799961 22653 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.801443 22948 tablet_bootstrap.cc:492] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.802788 22976 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.803227 22976 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.803658 22945 tablet_bootstrap.cc:492] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:25.818269 22945 tablet_bootstrap.cc:654] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.823076 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.823709 22948 tablet_bootstrap.cc:654] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.824170 22946 tablet_bootstrap.cc:492] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
W20260709 13:06:25.824574 22797 leader_election.cc:343] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.824787 22946 ts_tablet_manager.cc:1403] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.022s
I20260709 13:06:25.826856 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:25.828712 22794 leader_election.cc:343] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.828509 22946 raft_consensus.cc:359] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.829646 22946 raft_consensus.cc:385] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.829380 22794 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.830175 22946 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.830752 22960 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:25.830061 22976 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.831324 22946 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.833750 22946 ts_tablet_manager.cc:1434] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:25.834553 22946 tablet_bootstrap.cc:492] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.834762 22948 tablet_bootstrap.cc:492] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.835305 22948 ts_tablet_manager.cc:1403] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.008s
I20260709 13:06:25.837478 22948 raft_consensus.cc:359] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.838112 22948 raft_consensus.cc:385] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.838380 22948 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.838971 22948 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.839929 22946 tablet_bootstrap.cc:654] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.841867 22948 ts_tablet_manager.cc:1434] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:06:25.842145 22945 tablet_bootstrap.cc:492] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:25.842573 22945 ts_tablet_manager.cc:1403] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.005s
I20260709 13:06:25.843178 22948 tablet_bootstrap.cc:492] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.845389 22945 raft_consensus.cc:359] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.846058 22945 raft_consensus.cc:385] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.846328 22945 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.847019 22945 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.848922 22945 ts_tablet_manager.cc:1434] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:25.850713 22946 tablet_bootstrap.cc:492] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.851140 22946 ts_tablet_manager.cc:1403] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.003s
I20260709 13:06:25.852572 22948 tablet_bootstrap.cc:654] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.852842 22945 tablet_bootstrap.cc:492] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:25.853592 22946 raft_consensus.cc:359] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.854192 22946 raft_consensus.cc:385] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.854457 22946 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.855245 22946 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.857344 22951 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.857800 22951 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.860121 22946 ts_tablet_manager.cc:1434] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:06:25.860447 22951 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:25.860961 22946 tablet_bootstrap.cc:492] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:25.861876 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.862449 22756 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:25.863638 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:25.864576 22951 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.864956 22951 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.865424 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:25.866074 22831 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:25.866386 23001 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.867226 22946 tablet_bootstrap.cc:654] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.866968 23001 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.869266 22756 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.870864 23001 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.872213 22948 tablet_bootstrap.cc:492] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.872642 22948 ts_tablet_manager.cc:1403] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.006s
I20260709 13:06:25.872702 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.873652 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:06:25.874056 22719 leader_election.cc:343] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.874221 22831 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:25.875293 22948 raft_consensus.cc:359] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.875909 22948 raft_consensus.cc:385] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.876164 22948 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.876782 22948 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.878751 22948 ts_tablet_manager.cc:1434] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:06:25.879561 22948 tablet_bootstrap.cc:492] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.879225 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:25.880614 23001 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.881063 23001 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.881507 23001 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.865413 22951 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.885236 22948 tablet_bootstrap.cc:654] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.891427 23001 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.898504 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:25.899207 22831 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.907269 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:06:25.908947 22719 leader_election.cc:343] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.909670 23001 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.909875 22976 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:25.914652 22831 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:25.916040 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:25.917456 23001 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.918221 23001 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:25.919308 22945 tablet_bootstrap.cc:654] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.919239 23001 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.922279 22951 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.925822 22948 tablet_bootstrap.cc:492] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:25.925643 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:25.926456 22948 ts_tablet_manager.cc:1403] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.047s	user 0.033s	sys 0.011s
I20260709 13:06:25.926589 22756 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.925935 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:25.927493 22831 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.929335 22948 raft_consensus.cc:359] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.930229 22948 raft_consensus.cc:385] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.930183 22976 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.930857 22976 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.933683 22976 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.934208 22948 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.934705 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.934933 22948 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:25.936022 22794 leader_election.cc:343] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.940897 22948 ts_tablet_manager.cc:1434] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:06:25.941992 22948 tablet_bootstrap.cc:492] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:25.942252 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.943622 22976 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:25.944021 22960 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.944612 22960 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:25.945497 22797 leader_election.cc:343] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.945926 22797 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:25.932829 22951 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:25.949666 22960 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.950608 22976 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:25.952196 22960 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.952999 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.958772 22948 tablet_bootstrap.cc:654] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:25.958879 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.956045 22960 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.964025 22756 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:25.965528 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:25.965895 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.968250 22756 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:25.959484 22905 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:25.967738 22753 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.964372 22754 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:25.971043 22951 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:25.970104 22960 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:25.975637 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:25.977990 22831 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:25.978416 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:25.978729 22960 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.980273 22960 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.980314 22905 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:25.980729 22960 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.980993 22972 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.979440 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:25.981472 22972 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.983577 22972 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:25.974997 22951 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:25.984107 22976 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:25.984472 22976 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.984822 22976 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:25.984786 22951 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:25.987323 22946 tablet_bootstrap.cc:492] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:25.987753 22946 ts_tablet_manager.cc:1403] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.127s	user 0.025s	sys 0.048s
I20260709 13:06:25.991256 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:25.991784 22945 tablet_bootstrap.cc:492] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:25.992208 22945 ts_tablet_manager.cc:1403] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.140s	user 0.010s	sys 0.001s
W20260709 13:06:25.992627 22871 leader_election.cc:343] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:25.994124 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:25.995096 22945 raft_consensus.cc:359] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.996178 22945 raft_consensus.cc:385] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:25.995590 22831 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:25.996618 22945 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:25.998420 22960 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:25.998181 22945 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.007684 22945 ts_tablet_manager.cc:1434] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:06:26.008360 22945 tablet_bootstrap.cc:492] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:26.014523 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:26.017544 22969 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.018512 22969 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.018826 22969 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.023329 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:26.023926 22904 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.023329 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:26.025903 22755 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.022045 22960 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.034502 22904 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:26.020959 22976 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.036844 22976 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.038439 23030 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:06:26.039069 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:26.037698 22654 catalog_manager.cc:5697] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.042443 22946 raft_consensus.cc:359] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.043286 22946 raft_consensus.cc:385] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.043613 22946 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.051282 22946 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.048478 23030 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:26.052589 22755 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:26.056401 23030 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:26.051440 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:26.058138 22946 ts_tablet_manager.cc:1434] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.001s
I20260709 13:06:26.059791 22946 tablet_bootstrap.cc:492] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:26.046697 22969 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.061040 22945 tablet_bootstrap.cc:654] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.055662 22976 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.065289 22904 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.074944 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:26.075554 22755 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.090987 22755 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:26.093516 22948 tablet_bootstrap.cc:492] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:26.094095 22948 ts_tablet_manager.cc:1403] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.152s	user 0.019s	sys 0.023s
I20260709 13:06:26.097002 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:26.092343 22904 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:26.085891 22976 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.105647 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:26.106173 23028 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.106423 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.106652 23028 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:26.108057 22871 leader_election.cc:343] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.108850 23028 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.109395 22969 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:26.104898 22654 catalog_manager.cc:5697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.108199 22948 raft_consensus.cc:359] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.110118 22948 raft_consensus.cc:385] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.110258 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:26.110441 22948 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.110808 22831 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.111722 22948 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.098205 23031 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:06:26.115509 22960 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.099735 23035 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:26.116475 22960 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.094463 23030 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.118525 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:26.146991 22903 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 1: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:06:26.148538 22960 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.149187 22960 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.149710 22960 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.152271 22831 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:26.153563 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:26.132243 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:26.158627 22797 leader_election.cc:343] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.160497 22946 tablet_bootstrap.cc:654] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:26.160739 22702 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:26.164822 22948 ts_tablet_manager.cc:1434] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.070s	user 0.002s	sys 0.008s
I20260709 13:06:26.165766 22948 tablet_bootstrap.cc:492] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:26.119088 23035 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:26.172698 23035 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.175789 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:26.176160 22960 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.127043 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:26.117501 22976 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.129913 22830 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.174815 22969 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:26.178541 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.180311 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:26.182571 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.179785 23001 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.179616 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:26.182164 22951 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:06:26.185051 22974 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:06:26.197985 23001 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.194747 22969 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:26.201090 23008 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:26.198158 22794 leader_election.cc:343] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.203524 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:26.203753 22969 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:26.203532 22797 leader_election.cc:343] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:26.207221 22797 leader_election.cc:343] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.207720 23030 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:26.214048 23001 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:06:26.240062 22794 leader_election.cc:343] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.249634 22945 tablet_bootstrap.cc:492] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:26.250214 22945 ts_tablet_manager.cc:1403] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.242s	user 0.020s	sys 0.011s
I20260709 13:06:26.254208 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.254292 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:26.254973 22829 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:26.255208 22901 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:26.257166 23028 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:26.258386 22948 tablet_bootstrap.cc:654] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.247687 22794 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:26.264495 22654 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.272200 23030 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:26.266731 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:26.274250 22974 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.274627 22974 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.274948 22974 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.276630 22945 raft_consensus.cc:359] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.277403 22945 raft_consensus.cc:385] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.277724 22945 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
W20260709 13:06:26.281648 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 43546.8us
W20260709 13:06:26.282305 22794 leader_election.cc:343] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.282842 22946 tablet_bootstrap.cc:492] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:26.282685 22794 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:26.283681 23030 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:26.283900 22946 ts_tablet_manager.cc:1403] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.225s	user 0.015s	sys 0.009s
I20260709 13:06:26.284796 22651 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:26.286180 22946 raft_consensus.cc:359] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.288499 22946 raft_consensus.cc:385] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.288803 22946 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.288774 22973 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:26.289829 22946 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.290702 23008 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.292176 23001 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:26.293162 23008 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.296226 23008 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.303697 23030 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.304172 23030 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.303275 22974 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:26.309496 23039 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 13:06:26.309772 23040 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:06:26.311964 23030 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.322063 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.325116 22946 ts_tablet_manager.cc:1434] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.041s	user 0.011s	sys 0.000s
I20260709 13:06:26.327657 22974 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.327872 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.333185 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:26.333530 22797 leader_election.cc:343] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.334296 22945 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.336504 22948 tablet_bootstrap.cc:492] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:26.339732 22945 ts_tablet_manager.cc:1434] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.089s	user 0.013s	sys 0.001s
I20260709 13:06:26.339821 22948 ts_tablet_manager.cc:1403] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.174s	user 0.012s	sys 0.003s
I20260709 13:06:26.343191 22945 tablet_bootstrap.cc:492] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:26.344431 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:26.345006 22901 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.348268 22948 raft_consensus.cc:359] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.348917 22948 raft_consensus.cc:385] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.349167 22948 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.349620 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:26.349778 22948 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.350255 22828 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.351734 22948 ts_tablet_manager.cc:1434] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.001s
I20260709 13:06:26.355152 22948 tablet_bootstrap.cc:492] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:26.354568 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:26.357636 22719 leader_election.cc:343] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:26.360852 22794 leader_election.cc:343] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.361258 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:26.363421 23001 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:26.369618 23051 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:26.371268 22946 tablet_bootstrap.cc:492] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:26.372619 22960 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:26.373837 22719 leader_election.cc:343] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.382241 22945 tablet_bootstrap.cc:654] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.380973 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:26.379977 22901 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:26.388743 22974 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:26.391952 23001 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:26.393424 23048 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:26.396255 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:26.397218 23048 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.397897 23048 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.405658 23001 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:26.406267 23001 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:26.408073 22946 tablet_bootstrap.cc:654] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.410471 23001 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.394130 22828 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:26.422150 23048 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.426682 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.428241 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:26.432770 22719 leader_election.cc:343] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.433471 22654 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.439764 22653 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.441118 22948 tablet_bootstrap.cc:654] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.444272 22945 tablet_bootstrap.cc:492] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:26.444746 22945 ts_tablet_manager.cc:1403] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.102s	user 0.018s	sys 0.013s
I20260709 13:06:26.452551 22945 raft_consensus.cc:359] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.453979 22945 raft_consensus.cc:385] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.454324 22945 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.455497 22945 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.464118 22945 ts_tablet_manager.cc:1434] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.004s
I20260709 13:06:26.465265 22945 tablet_bootstrap.cc:492] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:26.480199 23001 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.481173 23001 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.463853 22951 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:06:26.483911 23001 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:06:26.486946 22719 leader_election.cc:343] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.490185 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:26.491392 23001 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:26.506691 22945 tablet_bootstrap.cc:654] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.508114 22946 tablet_bootstrap.cc:492] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:26.508594 22946 ts_tablet_manager.cc:1403] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.138s	user 0.031s	sys 0.008s
I20260709 13:06:26.512601 22946 raft_consensus.cc:359] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.513388 22946 raft_consensus.cc:385] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.513676 22946 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.514514 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.515574 22828 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:26.527639 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:26.525589 22946 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.532763 22948 tablet_bootstrap.cc:492] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:26.533270 22948 ts_tablet_manager.cc:1403] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.178s	user 0.012s	sys 0.014s
I20260709 13:06:26.535776 22948 raft_consensus.cc:359] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.536443 22948 raft_consensus.cc:385] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.536695 22948 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.537355 22948 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.539495 22946 ts_tablet_manager.cc:1434] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 13:06:26.542075 22945 tablet_bootstrap.cc:492] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:26.542552 22945 ts_tablet_manager.cc:1403] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.078s	user 0.024s	sys 0.005s
I20260709 13:06:26.542675 22973 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:26.543449 22960 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:26.545219 22945 raft_consensus.cc:359] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.545980 22945 raft_consensus.cc:385] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.546306 22945 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.547003 22945 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.547816 22946 tablet_bootstrap.cc:492] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:26.549229 22945 ts_tablet_manager.cc:1434] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:26.550405 22945 tablet_bootstrap.cc:492] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:26.556401 22945 tablet_bootstrap.cc:654] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.559207 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:26.562613 22948 ts_tablet_manager.cc:1434] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:06:26.563726 22948 tablet_bootstrap.cc:492] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:26.563822 23048 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.574815 23048 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:26.580440 22719 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:26.584694 22945 tablet_bootstrap.cc:492] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:26.585137 22945 ts_tablet_manager.cc:1403] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.019s
I20260709 13:06:26.587320 22945 raft_consensus.cc:359] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.587798 22945 raft_consensus.cc:385] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.588037 22945 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.588626 22945 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.589996 23035 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:26.593267 22946 tablet_bootstrap.cc:654] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.602196 22948 tablet_bootstrap.cc:654] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.605553 23028 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.606338 22945 ts_tablet_manager.cc:1434] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:06:26.607201 22945 tablet_bootstrap.cc:492] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:26.606014 23028 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.609642 23028 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.610476 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.611294 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:26.611835 22797 leader_election.cc:343] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.612008 22902 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:26.613265 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:26.613729 22945 tablet_bootstrap.cc:654] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.614413 23028 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.614876 23028 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.615499 23028 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.617889 22946 tablet_bootstrap.cc:492] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:26.618352 22946 ts_tablet_manager.cc:1403] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.001s
I20260709 13:06:26.622063 22946 raft_consensus.cc:359] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.622762 22946 raft_consensus.cc:385] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.623052 22946 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.623797 22946 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.626142 22946 ts_tablet_manager.cc:1434] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:06:26.628084 23028 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.629814 22945 tablet_bootstrap.cc:492] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:26.630226 22946 tablet_bootstrap.cc:492] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:26.630265 22945 ts_tablet_manager.cc:1403] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.001s
I20260709 13:06:26.633112 22945 raft_consensus.cc:359] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.633872 22945 raft_consensus.cc:385] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.634136 22945 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.634918 22945 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.635221 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:26.636327 22946 tablet_bootstrap.cc:654] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.637174 22945 ts_tablet_manager.cc:1434] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:26.638526 22945 tablet_bootstrap.cc:492] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
W20260709 13:06:26.646230 22797 leader_election.cc:343] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.648594 22945 tablet_bootstrap.cc:654] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.662582 23028 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.665638 22948 tablet_bootstrap.cc:492] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:26.666204 22948 ts_tablet_manager.cc:1403] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.103s	user 0.019s	sys 0.007s
I20260709 13:06:26.666394 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:26.666950 22904 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.669677 22948 raft_consensus.cc:359] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.670469 22948 raft_consensus.cc:385] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.670647 22948 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.671294 22948 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.672767 22945 tablet_bootstrap.cc:492] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:26.673203 22945 ts_tablet_manager.cc:1403] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.011s
I20260709 13:06:26.673389 22904 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:26.673600 22948 ts_tablet_manager.cc:1434] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:06:26.674484 22946 tablet_bootstrap.cc:492] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:26.674604 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:26.674908 22946 ts_tablet_manager.cc:1403] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.045s	user 0.027s	sys 0.003s
I20260709 13:06:26.675647 23028 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:26.676100 23028 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:26.676059 22945 raft_consensus.cc:359] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.676925 22945 raft_consensus.cc:385] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.677260 22945 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.677018 23028 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.677518 22946 raft_consensus.cc:359] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.678104 22945 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.678731 22948 tablet_bootstrap.cc:492] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:26.680286 22945 ts_tablet_manager.cc:1434] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:26.678122 22946 raft_consensus.cc:385] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.680938 22946 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.682376 22946 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.681830 22945 tablet_bootstrap.cc:492] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:26.686565 22948 tablet_bootstrap.cc:654] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.690358 22973 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:26.691474 22946 ts_tablet_manager.cc:1434] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.016s	user 0.001s	sys 0.005s
I20260709 13:06:26.696980 22946 tablet_bootstrap.cc:492] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:26.702291 22946 tablet_bootstrap.cc:654] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.702817 22651 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.694175 22945 tablet_bootstrap.cc:654] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.708652 22948 tablet_bootstrap.cc:492] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:26.708421 22973 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:26.709103 22948 ts_tablet_manager.cc:1403] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.006s
I20260709 13:06:26.711593 22948 raft_consensus.cc:359] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.712261 22948 raft_consensus.cc:385] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.712543 22948 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.713172 22948 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.714753 22945 tablet_bootstrap.cc:492] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:26.715047 22948 ts_tablet_manager.cc:1434] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:26.715210 22945 ts_tablet_manager.cc:1403] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.001s
I20260709 13:06:26.715833 22948 tablet_bootstrap.cc:492] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:26.717892 22945 raft_consensus.cc:359] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.719312 22945 raft_consensus.cc:385] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.719633 22945 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.720417 22945 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.723471 22948 tablet_bootstrap.cc:654] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.727051 22946 tablet_bootstrap.cc:492] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:26.727461 22946 ts_tablet_manager.cc:1403] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 13:06:26.728103 22945 ts_tablet_manager.cc:1434] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:06:26.728925 22945 tablet_bootstrap.cc:492] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:26.729812 22946 raft_consensus.cc:359] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.730427 22946 raft_consensus.cc:385] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.730667 22946 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.731353 22946 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.733649 22946 ts_tablet_manager.cc:1434] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:06:26.734467 22946 tablet_bootstrap.cc:492] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:26.735217 22945 tablet_bootstrap.cc:654] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.740522 22946 tablet_bootstrap.cc:654] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.751401 22948 tablet_bootstrap.cc:492] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:26.751977 22948 ts_tablet_manager.cc:1403] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.000s
I20260709 13:06:26.752722 23048 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.753368 23048 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.756258 23048 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.759003 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:26.759120 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.759647 22904 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:26.759935 22830 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:26.761487 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:26.763146 23048 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.763592 23048 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.764010 23048 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.768898 22948 raft_consensus.cc:359] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.769989 22948 raft_consensus.cc:385] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.770692 22948 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.772372 23048 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.772413 22948 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.773077 22946 tablet_bootstrap.cc:492] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:26.773185 22945 tablet_bootstrap.cc:492] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:26.773550 22946 ts_tablet_manager.cc:1403] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.000s
I20260709 13:06:26.773665 22945 ts_tablet_manager.cc:1403] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.045s	user 0.028s	sys 0.015s
I20260709 13:06:26.780807 23048 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.786294 22948 ts_tablet_manager.cc:1434] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.004s
I20260709 13:06:26.791608 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:26.792167 22830 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.792372 22945 raft_consensus.cc:359] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.792706 22946 raft_consensus.cc:359] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.793432 22946 raft_consensus.cc:385] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.793716 22946 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.794322 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:26.794905 22904 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.794322 22946 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.795681 23001 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.796095 23001 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.796612 22946 ts_tablet_manager.cc:1434] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:06:26.797997 23001 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.799029 22830 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:26.799773 22948 tablet_bootstrap.cc:492] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:26.800770 22904 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:26.800875 22946 tablet_bootstrap.cc:492] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:26.793092 22945 raft_consensus.cc:385] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.801544 22973 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.801802 22945 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.801992 22973 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.802551 22945 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.803305 22972 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.803695 22972 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.804039 22973 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:26.804529 22945 ts_tablet_manager.cc:1434] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.013s
I20260709 13:06:26.805730 22972 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:26.805809 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:26.806079 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.806722 22828 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:26.806919 23001 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:26.806722 22948 tablet_bootstrap.cc:654] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.807376 23001 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:26.808082 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.808468 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.808862 22830 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:26.809039 22828 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:26.810640 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:26.810894 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.811928 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.813177 22951 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.814006 22951 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.813825 22753 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:26.814458 22951 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.815244 22946 tablet_bootstrap.cc:654] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:26.815646 22871 leader_election.cc:343] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.817189 22972 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.811831 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:26.820119 22945 tablet_bootstrap.cc:492] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:26.820384 22969 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.820663 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:26.820758 22969 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.821362 22969 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.819214 23001 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.828372 22945 tablet_bootstrap.cc:654] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:26.819764 22972 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.830801 22972 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:26.834213 23028 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:26.836422 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.837283 22753 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:26.843603 23028 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:26.844779 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.848903 22828 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:26.851672 23028 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.858260 23028 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.859642 22871 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:26.861163 22972 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.861547 22972 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.861878 22972 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.861999 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.862607 22753 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:26.863842 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:26.864842 23030 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.865288 23030 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.865696 23030 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.868284 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:26.870110 22794 leader_election.cc:343] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.874069 22951 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.875360 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:26.878048 22974 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:26.882021 22974 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.882473 22974 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.888527 22973 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.872639 22969 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.893329 23008 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:26.877065 23028 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.889179 22973 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.897799 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:26.898006 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:26.901638 22830 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:26.901959 22871 leader_election.cc:343] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:26.901638 22904 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:26.908530 22651 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.899664 22951 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:26.913677 22948 tablet_bootstrap.cc:492] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:26.914253 22948 ts_tablet_manager.cc:1403] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.115s	user 0.016s	sys 0.019s
I20260709 13:06:26.922061 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:26.904335 23051 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:06:26.931406 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:26.932049 22828 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:26.931674 22972 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.932192 22969 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:26.934060 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:26.934672 22753 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:26.937139 22948 raft_consensus.cc:359] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.942965 22948 raft_consensus.cc:385] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.941246 22945 tablet_bootstrap.cc:492] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:26.946998 22945 ts_tablet_manager.cc:1403] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.127s	user 0.019s	sys 0.005s
I20260709 13:06:26.943516 22948 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.966969 23030 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.967948 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:26.929237 22973 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:26.943192 22969 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.973977 22969 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.968936 22828 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:26.971321 22948 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:26.969872 23084 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:06:26.974561 22945 raft_consensus.cc:359] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.977085 22945 raft_consensus.cc:385] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.977622 22945 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.978545 22945 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:26.976619 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 82677.5us
I20260709 13:06:26.981660 22946 tablet_bootstrap.cc:492] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:26.982089 22946 ts_tablet_manager.cc:1403] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.181s	user 0.017s	sys 0.041s
I20260709 13:06:26.985309 22946 raft_consensus.cc:359] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.985986 22946 raft_consensus.cc:385] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:26.970793 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:26.986274 22946 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:26.986871 22946 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.987130 22969 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:26.990091 22974 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:26.991947 22974 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:26.992774 22756 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 1: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:06:26.998788 22946 ts_tablet_manager.cc:1434] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:06:27.000092 22946 tablet_bootstrap.cc:492] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:27.013378 22973 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:27.014753 22973 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.014875 22972 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.015209 22973 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.015728 23030 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.017856 22945 ts_tablet_manager.cc:1434] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.000s
I20260709 13:06:27.015648 22830 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:27.018983 22945 tablet_bootstrap.cc:492] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
W20260709 13:06:27.045598 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 95757.7us
W20260709 13:06:27.042300 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 92823.7us
I20260709 13:06:27.051862 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:27.052561 22754 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.052502 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.062281 22960 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.067850 22948 ts_tablet_manager.cc:1434] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.153s	user 0.027s	sys 0.029s
I20260709 13:06:27.069051 22948 tablet_bootstrap.cc:492] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:27.066287 22753 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:27.078992 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:27.079730 22904 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:27.080593 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:27.081372 22969 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:27.080981 22973 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.081759 22969 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.082140 22969 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.083796 22946 tablet_bootstrap.cc:654] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.088667 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:27.089234 22755 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.090070 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:27.091142 23094 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:27.092972 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:27.093572 22751 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.093863 23094 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:27.099334 22903 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.069380 22960 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.096503 22828 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:27.085250 22973 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.104470 22972 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.103468 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.113253 22830 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:27.115866 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:27.116627 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:27.104591 22969 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.117158 22972 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.126725 22945 tablet_bootstrap.cc:654] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.131806 23094 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.138057 23008 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.160305 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.160101 23008 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.124445 22755 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:27.133415 22960 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.138609 22973 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:27.156143 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:27.194437 22969 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.198259 22972 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.215013 22945 tablet_bootstrap.cc:492] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.215476 22945 ts_tablet_manager.cc:1403] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.197s	user 0.018s	sys 0.000s
W20260709 13:06:27.195139 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 78621.5us
I20260709 13:06:27.213073 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:27.220425 22946 tablet_bootstrap.cc:492] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:27.222510 22946 ts_tablet_manager.cc:1403] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.223s	user 0.005s	sys 0.012s
W20260709 13:06:27.210453 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 47951.8us
I20260709 13:06:27.220867 22826 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:27.219455 22973 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:27.220539 23051 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.221582 22752 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.227064 22973 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.231506 23051 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.238132 22948 tablet_bootstrap.cc:654] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.241457 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:27.241941 23051 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.242053 22829 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:27.250711 22794 leader_election.cc:343] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:27.251672 22794 leader_election.cc:343] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.252858 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:27.253840 22828 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.254897 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.255937 22871 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:27.235065 23008 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.233574 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.257771 23102 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:27.252913 22960 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:27.263103 23102 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:27.264978 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:06:27.265746 22871 leader_election.cc:343] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.265890 23102 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.269534 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.259007 22751 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:27.268811 23008 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.266904 22754 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:27.274091 22960 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:27.284983 22702 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:27.326972 22871 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:06:27.328068 23100 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:06:27.329749 22946 raft_consensus.cc:359] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.330636 22946 raft_consensus.cc:385] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.330952 22946 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.331784 22946 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.333791 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.336091 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:27.307578 22960 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.339418 22960 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.341670 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.336858 22826 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.343776 22946 ts_tablet_manager.cc:1434] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.121s	user 0.008s	sys 0.000s
I20260709 13:06:27.347479 22969 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:27.348533 22794 leader_election.cc:343] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.352391 22752 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
W20260709 13:06:27.352944 22797 leader_election.cc:343] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.350033 23001 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.353379 22969 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:27.354081 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.347446 22829 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:27.354578 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.354678 22830 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:27.357184 22945 raft_consensus.cc:359] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.358141 22945 raft_consensus.cc:385] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.358462 22945 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
W20260709 13:06:27.350253 22719 leader_election.cc:343] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.385977 22946 tablet_bootstrap.cc:492] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:27.386114 22945 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.378811 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.353559 23001 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.354688 22969 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.359279 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:27.389158 22828 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
W20260709 13:06:27.399339 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 46428.1us
I20260709 13:06:27.400036 22945 ts_tablet_manager.cc:1434] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.184s	user 0.009s	sys 0.001s
I20260709 13:06:27.400203 22960 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.356397 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:27.400959 22945 tablet_bootstrap.cc:492] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.400070 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:27.401578 23102 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:27.401479 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45765.3us
I20260709 13:06:27.393232 23001 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.410065 22752 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.411713 22948 tablet_bootstrap.cc:492] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:27.412191 22948 ts_tablet_manager.cc:1403] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.343s	user 0.018s	sys 0.005s
I20260709 13:06:27.404124 23102 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:27.404214 22960 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:27.406816 22826 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:27.419377 22960 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:27.426743 22960 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.425112 22948 raft_consensus.cc:359] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.430827 22948 raft_consensus.cc:385] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.431206 22948 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.430084 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.435137 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:27.437949 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:27.441373 22719 leader_election.cc:343] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.441627 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:27.443116 22948 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.448387 23098 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:27.449400 23098 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.449774 23098 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.424971 23102 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.458694 22948 ts_tablet_manager.cc:1434] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.002s
I20260709 13:06:27.460690 22948 tablet_bootstrap.cc:492] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
W20260709 13:06:27.450767 22719 leader_election.cc:343] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.463712 22719 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:27.467032 23088 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.469815 22945 tablet_bootstrap.cc:654] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.473858 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.479001 22653 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:27.485476 23098 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.486955 23102 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.487393 23102 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.493264 23089 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.492357 23102 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.493743 23088 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:27.493256 22797 leader_election.cc:343] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.497442 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.498433 22752 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 1: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:06:27.497076 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:27.500728 22960 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.503734 22946 tablet_bootstrap.cc:654] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.510111 23089 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:27.511102 23098 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.512008 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:06:27.512579 22719 leader_election.cc:343] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:27.513484 22871 leader_election.cc:343] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.513705 22719 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:27.514487 23089 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.517659 22652 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.519500 22960 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:27.519517 22945 tablet_bootstrap.cc:492] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.522396 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.523294 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:27.530273 22828 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.531996 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:27.534039 23098 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.535427 22652 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.537587 22945 ts_tablet_manager.cc:1403] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.137s	user 0.016s	sys 0.004s
I20260709 13:06:27.541865 22945 raft_consensus.cc:359] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.542492 22945 raft_consensus.cc:385] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.544586 22828 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:27.542753 22945 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.546130 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:27.547396 23102 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:27.547444 22945 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.548099 23102 raft_consensus.cc:697] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:27.549548 22945 ts_tablet_manager.cc:1434] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:27.549117 23102 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.550390 22945 tablet_bootstrap.cc:492] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.528561 22948 tablet_bootstrap.cc:654] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.566910 22945 tablet_bootstrap.cc:654] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.568835 23102 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.569909 23102 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.570387 23098 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.572101 22948 tablet_bootstrap.cc:492] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:27.572101 22945 tablet_bootstrap.cc:492] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.572669 22948 ts_tablet_manager.cc:1403] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.112s	user 0.013s	sys 0.000s
I20260709 13:06:27.572714 22945 ts_tablet_manager.cc:1403] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.004s
I20260709 13:06:27.572984 23102 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.572903 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.575300 22948 raft_consensus.cc:359] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:27.575417 22871 leader_election.cc:343] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.575975 22948 raft_consensus.cc:385] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.576279 22948 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.576112 22945 raft_consensus.cc:359] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.577018 22945 raft_consensus.cc:385] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.577020 22948 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.578336 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.579006 22945 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
W20260709 13:06:27.580343 22868 leader_election.cc:343] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.578670 22946 tablet_bootstrap.cc:492] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:27.580873 22948 ts_tablet_manager.cc:1434] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:06:27.580984 22868 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:27.579873 22945 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.581246 22946 ts_tablet_manager.cc:1403] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.195s	user 0.019s	sys 0.004s
I20260709 13:06:27.581799 22948 tablet_bootstrap.cc:492] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:27.583680 22973 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.586247 22945 ts_tablet_manager.cc:1434] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:06:27.588868 22945 tablet_bootstrap.cc:492] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.589246 22948 tablet_bootstrap.cc:654] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.587517 22946 raft_consensus.cc:359] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.592811 22946 raft_consensus.cc:385] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.593252 22946 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.594173 22946 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.599012 22946 ts_tablet_manager.cc:1434] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:06:27.599647 22948 tablet_bootstrap.cc:492] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:27.600059 22948 ts_tablet_manager.cc:1403] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 13:06:27.600319 22946 tablet_bootstrap.cc:492] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:27.602375 22948 raft_consensus.cc:359] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.603008 22948 raft_consensus.cc:385] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.603266 22948 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.603909 22948 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.603926 22945 tablet_bootstrap.cc:654] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.613656 22948 ts_tablet_manager.cc:1434] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:06:27.614534 22948 tablet_bootstrap.cc:492] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:27.621143 22948 tablet_bootstrap.cc:654] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.607892 22946 tablet_bootstrap.cc:654] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.626389 22651 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.628553 22651 catalog_manager.cc:5697] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.630478 22651 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.632484 22651 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.637490 23098 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.638964 22945 tablet_bootstrap.cc:492] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.639591 22945 ts_tablet_manager.cc:1403] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.003s
I20260709 13:06:27.643322 22945 raft_consensus.cc:359] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.644295 22945 raft_consensus.cc:385] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.644739 22945 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.645527 22945 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.647074 22946 tablet_bootstrap.cc:492] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:27.647653 22946 ts_tablet_manager.cc:1403] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.048s	user 0.006s	sys 0.004s
I20260709 13:06:27.648545 22945 ts_tablet_manager.cc:1434] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:06:27.649718 22960 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.650215 22960 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.652199 22960 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.651949 22946 raft_consensus.cc:359] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.652642 22946 raft_consensus.cc:385] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.652889 22946 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.653529 22946 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.654036 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.654663 22903 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:27.649719 22945 tablet_bootstrap.cc:492] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.655918 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.656204 22948 tablet_bootstrap.cc:492] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:27.656667 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:27.657100 22948 ts_tablet_manager.cc:1403] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.024s
I20260709 13:06:27.659541 22948 raft_consensus.cc:359] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.660223 22948 raft_consensus.cc:385] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.660526 22948 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.661278 22948 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.662207 23092 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:27.662561 23092 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.662876 23092 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.663378 22948 ts_tablet_manager.cc:1434] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:06:27.664513 22945 tablet_bootstrap.cc:654] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.665267 22948 tablet_bootstrap.cc:492] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
W20260709 13:06:27.667739 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:27.671486 22946 ts_tablet_manager.cc:1434] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:06:27.671103 23092 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.672386 22946 tablet_bootstrap.cc:492] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:27.673288 22948 tablet_bootstrap.cc:654] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.674165 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:27.674746 22903 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.678375 22946 tablet_bootstrap.cc:654] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.682548 23092 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.684150 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:27.685632 22903 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:27.688416 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:27.689508 23092 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:27.690299 23092 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:27.691313 23092 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:27.694542 22797 leader_election.cc:343] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:27.696188 22797 leader_election.cc:343] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.711180 22948 tablet_bootstrap.cc:492] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:27.711591 22948 ts_tablet_manager.cc:1403] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.046s	user 0.024s	sys 0.016s
I20260709 13:06:27.711829 22945 tablet_bootstrap.cc:492] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.712232 22946 tablet_bootstrap.cc:492] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:27.712247 22945 ts_tablet_manager.cc:1403] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.063s	user 0.006s	sys 0.011s
I20260709 13:06:27.712743 22946 ts_tablet_manager.cc:1403] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.009s
I20260709 13:06:27.714378 22948 raft_consensus.cc:359] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.715066 22948 raft_consensus.cc:385] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.714464 22653 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:27.715332 22948 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.715396 22946 raft_consensus.cc:359] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.716033 22946 raft_consensus.cc:385] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.715834 22945 raft_consensus.cc:359] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.716440 22946 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.716554 22945 raft_consensus.cc:385] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.716874 22945 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.717152 22946 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.717506 22945 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.719480 22946 ts_tablet_manager.cc:1434] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:06:27.716053 22948 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.719846 22945 ts_tablet_manager.cc:1434] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:06:27.720629 22945 tablet_bootstrap.cc:492] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.721060 22948 ts_tablet_manager.cc:1434] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:06:27.721822 22948 tablet_bootstrap.cc:492] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:27.725992 22945 tablet_bootstrap.cc:654] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.726473 22946 tablet_bootstrap.cc:492] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:27.736248 22946 tablet_bootstrap.cc:654] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.738294 23092 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.738759 23092 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.740819 23092 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.741499 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.742379 22945 tablet_bootstrap.cc:492] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.742781 22945 ts_tablet_manager.cc:1403] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 13:06:27.743407 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:27.743418 22797 leader_election.cc:343] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:27.746371 22794 leader_election.cc:343] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.746702 22948 tablet_bootstrap.cc:654] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.746809 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:27.747677 23092 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.748566 22945 raft_consensus.cc:359] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.749281 22945 raft_consensus.cc:385] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.749559 22945 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.750173 22946 tablet_bootstrap.cc:492] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:27.750671 22946 ts_tablet_manager.cc:1403] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.013s
I20260709 13:06:27.750253 22945 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.751950 22948 tablet_bootstrap.cc:492] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:27.752324 22948 ts_tablet_manager.cc:1403] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.031s	user 0.005s	sys 0.003s
I20260709 13:06:27.752316 22945 ts_tablet_manager.cc:1434] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:06:27.753249 22946 raft_consensus.cc:359] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.753921 22945 tablet_bootstrap.cc:492] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.753880 22946 raft_consensus.cc:385] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.754853 22946 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.754688 22948 raft_consensus.cc:359] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.755493 22946 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.757977 22969 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.760910 22945 tablet_bootstrap.cc:654] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.755358 22948 raft_consensus.cc:385] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.765255 22948 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.765975 22948 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.768460 22948 ts_tablet_manager.cc:1434] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:06:27.769766 22948 tablet_bootstrap.cc:492] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:27.775065 22948 tablet_bootstrap.cc:654] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.777341 22946 ts_tablet_manager.cc:1434] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:06:27.778457 22946 tablet_bootstrap.cc:492] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:27.787001 22946 tablet_bootstrap.cc:654] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.792276 23088 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.793648 22951 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.796921 22951 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.797513 22951 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.803511 22969 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.804816 22945 tablet_bootstrap.cc:492] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.805243 22945 ts_tablet_manager.cc:1403] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.017s
I20260709 13:06:27.807619 22951 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.792752 23088 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.807956 22945 raft_consensus.cc:359] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.808604 22945 raft_consensus.cc:385] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.808877 22945 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.809000 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.809563 22945 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.809906 23088 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.811539 22945 ts_tablet_manager.cc:1434] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:27.811486 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.812582 22752 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:27.814040 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.814633 22946 tablet_bootstrap.cc:492] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:27.814427 22948 tablet_bootstrap.cc:492] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:27.815644 22946 ts_tablet_manager.cc:1403] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.003s
I20260709 13:06:27.815950 22948 ts_tablet_manager.cc:1403] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.005s
I20260709 13:06:27.817863 23092 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.818338 23092 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.815039 22945 tablet_bootstrap.cc:492] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.819069 22948 raft_consensus.cc:359] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.819957 22948 raft_consensus.cc:385] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.820222 22948 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.820391 23092 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.820859 22948 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.821195 22871 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:27.821645 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.822274 22948 ts_tablet_manager.cc:1434] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:27.822358 23097 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:27.822577 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.822826 23097 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.823099 22948 tablet_bootstrap.cc:492] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:27.823205 23097 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:27.824563 22794 leader_election.cc:343] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.825047 22945 tablet_bootstrap.cc:654] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.825461 23094 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.826735 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:27.827543 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.827904 23088 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:27.829044 22868 leader_election.cc:343] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.830257 22948 tablet_bootstrap.cc:654] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.830539 23094 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.831108 23094 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:27.833384 22797 leader_election.cc:343] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.833913 22797 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:27.835320 22960 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.835981 22945 tablet_bootstrap.cc:492] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.836404 22945 ts_tablet_manager.cc:1403] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.000s
I20260709 13:06:27.838833 22945 raft_consensus.cc:359] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.839433 22945 raft_consensus.cc:385] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.839694 22945 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.840528 22945 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.842657 22945 ts_tablet_manager.cc:1434] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:27.843473 22945 tablet_bootstrap.cc:492] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.844720 22969 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.845157 22969 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.846293 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.847189 22969 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.847577 23094 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.818342 22946 raft_consensus.cc:359] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.848296 22946 raft_consensus.cc:385] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.848583 22946 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.849187 22946 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.849773 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.851132 22946 ts_tablet_manager.cc:1434] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 13:06:27.852030 22946 tablet_bootstrap.cc:492] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:27.853549 23089 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.853881 22945 tablet_bootstrap.cc:654] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.854153 23089 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.859692 23089 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.847100 22752 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:27.862012 22945 tablet_bootstrap.cc:492] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.861835 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.862437 22825 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:27.862520 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.863278 22945 ts_tablet_manager.cc:1403] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.002s
I20260709 13:06:27.864938 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.865517 22823 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
W20260709 13:06:27.865595 22868 leader_election.cc:343] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.866477 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:27.867233 22945 raft_consensus.cc:359] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.867864 22945 raft_consensus.cc:385] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.868119 22945 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.868700 22945 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.869478 22951 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:27.870918 22945 ts_tablet_manager.cc:1434] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:27.871747 22945 tablet_bootstrap.cc:492] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.875522 23089 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:27.875713 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:27.877329 22945 tablet_bootstrap.cc:654] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.878273 23089 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.878561 23094 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:27.879660 23089 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.879681 23094 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.881069 23094 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:27.880517 22871 leader_election.cc:343] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.888240 22871 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:27.889369 22945 tablet_bootstrap.cc:492] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.889763 22945 ts_tablet_manager.cc:1403] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 13:06:27.889726 22969 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:27.892076 23094 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.877038 23097 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.893769 23098 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.894269 22945 raft_consensus.cc:359] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.894933 22945 raft_consensus.cc:385] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.894403 23098 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.895205 22945 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.895951 22945 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.898038 23088 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.898607 23088 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.899242 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.900449 22945 ts_tablet_manager.cc:1434] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:27.903146 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.903503 22945 tablet_bootstrap.cc:492] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.906232 23097 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.908903 23094 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.909556 22945 tablet_bootstrap.cc:654] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.896970 23098 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:06:27.902406 22719 leader_election.cc:343] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:27.901073 23088 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:27.919052 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:27.919780 22751 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.922267 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:27.923041 22752 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:27.938529 23092 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:27.941210 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:27.941953 22825 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.947315 22946 tablet_bootstrap.cc:654] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.949883 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.950951 22823 raft_consensus.cc:2393] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:27.952219 23094 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.952836 23094 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.954785 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:27.955518 22754 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.958954 23094 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.967422 22945 tablet_bootstrap.cc:492] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:27.967926 22945 ts_tablet_manager.cc:1403] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.065s	user 0.013s	sys 0.047s
I20260709 13:06:27.968295 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:27.967244 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:27.969039 22824 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:27.977617 22828 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:27.980522 22945 raft_consensus.cc:359] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.981426 22945 raft_consensus.cc:385] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.981781 22945 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.982514 22945 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.985095 22945 ts_tablet_manager.cc:1434] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.000s
I20260709 13:06:27.985164 23094 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.986932 23094 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.989202 23094 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:27.986348 22824 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:27.991688 22945 tablet_bootstrap.cc:492] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:27.991936 22946 tablet_bootstrap.cc:492] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:27.992308 22946 ts_tablet_manager.cc:1403] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.140s	user 0.009s	sys 0.002s
I20260709 13:06:27.993040 23088 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.993491 23088 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.994701 22946 raft_consensus.cc:359] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:27.995338 22946 raft_consensus.cc:385] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:27.995611 22946 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:27.996542 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:27.997076 22945 tablet_bootstrap.cc:654] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:27.996585 22946 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:27.998072 22719 leader_election.cc:343] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.003834 22946 ts_tablet_manager.cc:1434] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:06:28.005342 22946 tablet_bootstrap.cc:492] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:27.995357 23088 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:28.016594 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:28.016566 22945 tablet_bootstrap.cc:492] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:28.018167 22945 ts_tablet_manager.cc:1403] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.007s
I20260709 13:06:28.017704 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:28.021126 22945 raft_consensus.cc:359] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.021775 22945 raft_consensus.cc:385] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.022322 22946 tablet_bootstrap.cc:654] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:28.030033 22825 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:28.036172 22945 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.039430 22945 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.024655 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:28.023748 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:28.046754 23088 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.026309 22951 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 13:06:28.033800 22754 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:28.060611 22948 tablet_bootstrap.cc:492] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:28.026016 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:28.061080 22948 ts_tablet_manager.cc:1403] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.238s	user 0.126s	sys 0.060s
I20260709 13:06:28.040323 23152 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:06:28.021642 22751 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:28.061504 22951 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:28.026176 23094 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:06:28.068109 22951 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.068648 22951 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.072556 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:28.073045 22945 ts_tablet_manager.cc:1434] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.002s
I20260709 13:06:28.072309 22948 raft_consensus.cc:359] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.075107 22948 raft_consensus.cc:385] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.075536 22945 tablet_bootstrap.cc:492] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:28.038093 23089 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.075564 22948 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.077049 22948 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:28.072335 22871 leader_election.cc:343] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.085247 22948 ts_tablet_manager.cc:1434] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:06:28.074074 22825 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:28.086324 22948 tablet_bootstrap.cc:492] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:28.075244 23097 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:06:28.089520 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:28.090143 22826 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:28.096123 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:28.096748 22946 tablet_bootstrap.cc:492] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:28.097203 22946 ts_tablet_manager.cc:1403] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.092s	user 0.023s	sys 0.000s
I20260709 13:06:28.097357 22973 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:28.097549 22868 leader_election.cc:343] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.097476 23102 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:28.097904 22973 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:28.098135 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:28.098989 23102 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:28.098752 22973 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.099180 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:28.099905 23098 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:28.100359 23098 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:06:28.091358 22871 leader_election.cc:343] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.101081 23098 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.128358 23092 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.128844 23092 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.130604 23092 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:28.148667 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:28.149226 22905 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:28.151901 22951 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.171854 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:28.172663 22750 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.168272 23088 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:28.167737 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57089.1us
I20260709 13:06:28.170948 22951 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:28.179018 22945 tablet_bootstrap.cc:654] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:28.194823 22750 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:28.180171 23089 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:28.212075 22946 raft_consensus.cc:359] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.213395 22946 raft_consensus.cc:385] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.214653 22946 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.243732 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:28.244850 23102 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:28.245494 23102 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:28.245813 22973 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:28.246567 23102 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.260632 22948 tablet_bootstrap.cc:654] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:28.257835 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:28.260123 22946 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.278028 22946 ts_tablet_manager.cc:1434] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.181s	user 0.016s	sys 0.002s
I20260709 13:06:28.262903 22903 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.279166 22946 tablet_bootstrap.cc:492] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:28.281554 22752 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:28.269047 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 120826us
I20260709 13:06:28.285621 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:28.286417 23094 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:28.286760 23094 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.287091 23094 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.287444 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:28.288518 23161 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:28.288895 23161 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.289191 23161 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.315411 22945 tablet_bootstrap.cc:492] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:28.315999 22945 ts_tablet_manager.cc:1403] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.241s	user 0.011s	sys 0.008s
I20260709 13:06:28.319129 22948 tablet_bootstrap.cc:492] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:28.319682 22948 ts_tablet_manager.cc:1403] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.234s	user 0.009s	sys 0.008s
I20260709 13:06:28.322867 22948 raft_consensus.cc:359] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.323681 22948 raft_consensus.cc:385] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.324023 22948 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.280565 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:06:28.325776 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46901.4us
I20260709 13:06:28.324777 22948 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.301148 23161 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.335249 22653 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.338013 22653 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.311967 22903 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:28.353868 22945 raft_consensus.cc:359] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.367501 22945 raft_consensus.cc:385] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.367862 22945 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.333549 23092 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:28.376355 22946 tablet_bootstrap.cc:654] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:28.378814 23161 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:28.380131 22945 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.370323 23092 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.385829 22948 ts_tablet_manager.cc:1434] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.066s	user 0.030s	sys 0.021s
I20260709 13:06:28.379657 23102 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.388463 22948 tablet_bootstrap.cc:492] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:28.389063 22945 ts_tablet_manager.cc:1434] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.073s	user 0.013s	sys 0.000s
I20260709 13:06:28.390228 22945 tablet_bootstrap.cc:492] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:28.394929 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:28.388329 23092 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.410331 22755 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.391127 23102 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.419497 23102 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:28.422962 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:28.423373 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:28.423743 22823 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.423957 22825 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.413805 23094 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.440366 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:28.441102 22756 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.443173 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:28.469722 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:28.475111 23098 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.475577 23098 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.480803 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:28.481813 23088 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:28.482074 22719 leader_election.cc:343] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.482401 23088 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:28.483423 23088 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.486275 22951 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.486725 22951 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.487421 23163 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:28.529526 23164 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.534621 23164 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.568552 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:28.572958 22826 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:28.568742 22702 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:28.588487 22946 tablet_bootstrap.cc:492] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:28.588884 22946 ts_tablet_manager.cc:1403] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.310s	user 0.013s	sys 0.002s
I20260709 13:06:28.578634 23094 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:28.590924 22951 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:28.592664 22948 tablet_bootstrap.cc:654] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:28.587889 22871 leader_election.cc:343] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.592984 23098 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:28.606408 22946 raft_consensus.cc:359] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.607285 22946 raft_consensus.cc:385] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.607658 22946 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.608475 22946 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.613608 22946 ts_tablet_manager.cc:1434] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:06:28.616704 22946 tablet_bootstrap.cc:492] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:28.612644 23164 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:28.638051 23089 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.638593 23089 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.643257 22756 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:28.672689 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:28.669258 23088 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.673523 22900 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:28.673547 23088 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.652392 23092 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.675925 23092 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:28.675808 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:28.672931 22826 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:28.678947 23092 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:28.686021 23092 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.686373 23092 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.688289 23089 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:28.684880 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:28.694610 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:28.703019 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:28.706566 22946 tablet_bootstrap.cc:654] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:28.690213 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:28.709707 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:28.711588 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:28.719242 22900 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.720120 22945 tablet_bootstrap.cc:654] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:28.691083 22823 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:28.723819 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:28.725849 22831 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.730253 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:28.717087 22825 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:28.733039 22871 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:28.731616 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:28.735088 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:28.734697 23092 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.733100 22751 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.737289 22824 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 1: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:06:28.739316 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:28.739936 22903 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.740667 22900 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
W20260709 13:06:28.741767 22868 leader_election.cc:343] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.741978 22973 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:28.742419 22973 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:28.743517 23164 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:28.743714 22797 leader_election.cc:343] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.744045 23164 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.746120 23164 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:28.743216 22973 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.747309 22903 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
W20260709 13:06:28.747830 22871 leader_election.cc:343] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.743592 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:28.748605 23094 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:28.749035 23156 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:28.750413 23094 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:28.750720 23164 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:28.751026 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:28.750447 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:28.751371 23164 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:28.748503 23092 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:28.752485 23164 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.754082 23156 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:28.754657 23156 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:28.755647 23156 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.759232 22751 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
W20260709 13:06:28.759608 22797 leader_election.cc:343] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:28.762583 22871 leader_election.cc:343] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.762928 22948 tablet_bootstrap.cc:492] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:28.763448 22948 ts_tablet_manager.cc:1403] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.375s	user 0.014s	sys 0.029s
I20260709 13:06:28.761857 23088 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:06:28.768769 22794 leader_election.cc:343] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.771927 22948 raft_consensus.cc:359] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.763018 22871 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:28.772591 22948 raft_consensus.cc:385] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.772867 22948 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.773548 22948 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.775532 22948 ts_tablet_manager.cc:1434] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:06:28.776307 22948 tablet_bootstrap.cc:492] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:28.772495 22831 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:28.783949 22945 tablet_bootstrap.cc:492] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:28.784358 22945 ts_tablet_manager.cc:1403] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.394s	user 0.009s	sys 0.015s
I20260709 13:06:28.788401 23094 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:28.812513 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:28.813241 22900 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:06:28.803215 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:28.819006 22946 tablet_bootstrap.cc:492] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:28.819505 22946 ts_tablet_manager.cc:1403] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.203s	user 0.012s	sys 0.011s
I20260709 13:06:28.836786 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:28.845703 23089 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:28.846215 23089 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:06:28.846558 22868 leader_election.cc:343] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.846984 22868 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:28.847523 23097 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:28.847736 23098 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:28.848835 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:28.850299 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:28.850764 22945 raft_consensus.cc:359] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.864413 23097 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:06:28.872845 22945 raft_consensus.cc:385] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.874565 22945 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.875304 22945 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.878082 22945 ts_tablet_manager.cc:1434] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.003s
I20260709 13:06:28.894212 22945 tablet_bootstrap.cc:492] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:28.880095 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:28.872094 22946 raft_consensus.cc:359] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.897895 22946 raft_consensus.cc:385] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.898187 22946 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.891947 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:28.899829 22719 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:28.878223 22948 tablet_bootstrap.cc:654] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:28.850791 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:28.900827 22946 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.851418 23089 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.909302 22951 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:28.909076 22751 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:28.912364 23088 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:28.913084 22946 ts_tablet_manager.cc:1434] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.093s	user 0.002s	sys 0.003s
I20260709 13:06:28.914053 22946 tablet_bootstrap.cc:492] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:28.914528 22651 catalog_manager.cc:5697] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:28.909508 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58800.7us
I20260709 13:06:28.918159 22951 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:28.921411 22948 tablet_bootstrap.cc:492] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:28.921921 22948 ts_tablet_manager.cc:1403] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.146s	user 0.015s	sys 0.001s
I20260709 13:06:28.928891 22946 tablet_bootstrap.cc:654] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:28.928200 22945 tablet_bootstrap.cc:654] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:28.933420 22797 leader_election.cc:343] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:28.933925 22797 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:28.934628 23164 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:28.933663 22948 raft_consensus.cc:359] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.935614 22948 raft_consensus.cc:385] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:28.935913 22948 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:28.936518 22948 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:28.938755 22951 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:28.943933 22948 ts_tablet_manager.cc:1434] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.002s
I20260709 13:06:28.945148 22948 tablet_bootstrap.cc:492] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:28.942626 22751 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:28.950799 23089 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:28.958464 22948 tablet_bootstrap.cc:654] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.003830 23088 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:29.004627 22652 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.008198 22652 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.009485 22946 tablet_bootstrap.cc:492] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:29.010528 22946 ts_tablet_manager.cc:1403] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.097s	user 0.035s	sys 0.007s
I20260709 13:06:29.011412 22945 tablet_bootstrap.cc:492] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:29.011811 22945 ts_tablet_manager.cc:1403] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.118s	user 0.024s	sys 0.030s
I20260709 13:06:29.011632 23161 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:29.016599 22945 raft_consensus.cc:359] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.017231 22945 raft_consensus.cc:385] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.017097 22946 raft_consensus.cc:359] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.017540 22945 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.017777 22946 raft_consensus.cc:385] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.018040 22946 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.018158 22945 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.018628 22946 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.020697 22946 ts_tablet_manager.cc:1434] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.000s
I20260709 13:06:29.022547 22945 ts_tablet_manager.cc:1434] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:06:29.023365 22945 tablet_bootstrap.cc:492] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:29.029493 22945 tablet_bootstrap.cc:654] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.031791 22946 tablet_bootstrap.cc:492] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:29.047348 22948 tablet_bootstrap.cc:492] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:29.047804 22948 ts_tablet_manager.cc:1403] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.103s	user 0.011s	sys 0.001s
I20260709 13:06:29.050531 22948 raft_consensus.cc:359] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.051218 22948 raft_consensus.cc:385] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.051546 22948 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.052157 22948 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.052182 22653 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:29.054982 22946 tablet_bootstrap.cc:654] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.054761 22653 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:29.063174 22945 tablet_bootstrap.cc:492] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:29.063572 22945 ts_tablet_manager.cc:1403] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.017s
I20260709 13:06:29.064771 22948 ts_tablet_manager.cc:1434] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:06:29.066126 22948 tablet_bootstrap.cc:492] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:29.065982 22945 raft_consensus.cc:359] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.067691 22945 raft_consensus.cc:385] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.067972 22945 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.068679 22945 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.071171 22945 ts_tablet_manager.cc:1434] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:06:29.071856 22948 tablet_bootstrap.cc:654] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.072007 22945 tablet_bootstrap.cc:492] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:29.075750 22946 tablet_bootstrap.cc:492] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:29.076287 22946 ts_tablet_manager.cc:1403] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.000s
I20260709 13:06:29.079912 22945 tablet_bootstrap.cc:654] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.081473 23164 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.081521 22654 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:29.081995 23164 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.084306 23164 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.085949 22948 tablet_bootstrap.cc:492] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:29.086112 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.086335 22948 ts_tablet_manager.cc:1403] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 13:06:29.086112 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:29.088038 22794 leader_election.cc:343] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:29.088838 22797 leader_election.cc:343] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.089221 22797 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:29.090111 22946 raft_consensus.cc:359] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.090791 22946 raft_consensus.cc:385] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.091100 22946 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.091722 23164 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:29.091775 22946 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.095275 22946 ts_tablet_manager.cc:1434] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:06:29.095984 22945 tablet_bootstrap.cc:492] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:29.096146 22946 tablet_bootstrap.cc:492] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:29.096410 22945 ts_tablet_manager.cc:1403] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.002s
I20260709 13:06:29.098474 22969 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.098903 22945 raft_consensus.cc:359] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.098881 22969 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.099771 22945 raft_consensus.cc:385] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.100067 22945 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.100992 22945 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.101449 22969 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:29.100822 22948 raft_consensus.cc:359] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.101895 22948 raft_consensus.cc:385] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.102156 22948 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.102741 22948 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.103370 22945 ts_tablet_manager.cc:1434] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:06:29.104228 22945 tablet_bootstrap.cc:492] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:29.104609 22948 ts_tablet_manager.cc:1434] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:06:29.108851 22948 tablet_bootstrap.cc:492] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:29.109742 22945 tablet_bootstrap.cc:654] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.109789 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:29.114367 22871 leader_election.cc:343] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.117658 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:06:29.119149 22868 leader_election.cc:343] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.119560 22868 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:29.120849 22969 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:29.123965 22946 tablet_bootstrap.cc:654] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.124940 22948 tablet_bootstrap.cc:654] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.131227 22948 tablet_bootstrap.cc:492] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:29.131569 22948 ts_tablet_manager.cc:1403] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:06:29.133438 22948 raft_consensus.cc:359] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.134106 22948 raft_consensus.cc:385] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.134374 22948 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.134990 22945 tablet_bootstrap.cc:492] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:29.134959 22948 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.135496 22945 ts_tablet_manager.cc:1403] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.000s
I20260709 13:06:29.135514 22946 tablet_bootstrap.cc:492] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:29.136878 22946 ts_tablet_manager.cc:1403] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.002s
I20260709 13:06:29.137893 22969 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.138326 22969 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.138978 22945 raft_consensus.cc:359] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.139576 22945 raft_consensus.cc:385] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.139554 22946 raft_consensus.cc:359] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.140043 22945 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.140213 22969 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:29.140247 22946 raft_consensus.cc:385] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.140684 22946 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.141017 22945 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.141389 22946 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.142710 22948 ts_tablet_manager.cc:1434] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:06:29.143254 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.143565 22948 tablet_bootstrap.cc:492] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
W20260709 13:06:29.144619 22871 leader_election.cc:343] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.144562 23092 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:29.149948 22948 tablet_bootstrap.cc:654] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.149896 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:29.152962 22945 ts_tablet_manager.cc:1434] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:06:29.155249 22945 tablet_bootstrap.cc:492] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
W20260709 13:06:29.161435 22868 leader_election.cc:343] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.162101 22868 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:29.163249 23097 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:29.164650 22946 ts_tablet_manager.cc:1434] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:06:29.169484 22969 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.169965 22969 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.171355 22945 tablet_bootstrap.cc:654] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.172645 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:29.173991 22871 leader_election.cc:343] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.182154 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:29.182765 22831 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:29.184084 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:29.184840 23097 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:29.185199 23097 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.185549 23097 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.184983 23152 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:29.179901 22969 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:29.193583 23097 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.201602 22948 tablet_bootstrap.cc:492] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:29.202030 22948 ts_tablet_manager.cc:1403] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.059s	user 0.018s	sys 0.017s
I20260709 13:06:29.202366 23156 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.202948 23156 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.204946 22948 raft_consensus.cc:359] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.205766 22948 raft_consensus.cc:385] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.206163 22948 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.206874 22948 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.207897 23156 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.208425 22946 tablet_bootstrap.cc:492] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:29.210039 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.210816 22903 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:29.212287 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:29.212666 22948 ts_tablet_manager.cc:1434] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:06:29.213084 23152 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:29.213438 23152 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.213521 22948 tablet_bootstrap.cc:492] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:29.213757 23152 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.218011 22946 tablet_bootstrap.cc:654] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.222697 23164 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.223124 23164 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.226070 23164 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.226784 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.227371 22750 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:29.227806 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.228375 22903 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:29.228926 22948 tablet_bootstrap.cc:654] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.229466 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:29.230700 23092 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:29.231225 23092 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.231686 23092 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.235793 22946 tablet_bootstrap.cc:492] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:29.236300 22946 ts_tablet_manager.cc:1403] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.011s
I20260709 13:06:29.237241 22948 tablet_bootstrap.cc:492] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:29.237841 22948 ts_tablet_manager.cc:1403] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 13:06:29.239419 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.240180 22946 raft_consensus.cc:359] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.240911 22946 raft_consensus.cc:385] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.241168 22946 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.241549 22969 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.241917 22946 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.241999 22969 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.243911 22948 raft_consensus.cc:359] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.244679 22948 raft_consensus.cc:385] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.245172 22948 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.244719 22946 ts_tablet_manager.cc:1434] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:06:29.251812 23161 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.245800 22948 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.252555 23161 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.252942 22946 tablet_bootstrap.cc:492] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:29.254050 22948 ts_tablet_manager.cc:1434] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:06:29.254441 23161 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.246043 22973 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:06:29.246735 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.243966 22969 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:06:29.256940 22871 leader_election.cc:343] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.258812 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.259754 22946 tablet_bootstrap.cc:654] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:29.260324 22719 leader_election.cc:343] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.261861 22948 tablet_bootstrap.cc:492] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:29.249615 23156 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:06:29.266994 23152 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.268573 23152 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:06:29.245793 22797 leader_election.cc:343] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.269593 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:29.270098 22903 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.270128 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:06:29.273608 22868 leader_election.cc:343] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.274025 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:29.277253 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:29.277832 22754 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.278437 22948 tablet_bootstrap.cc:654] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.279624 23092 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.283289 22973 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:29.283922 22945 tablet_bootstrap.cc:492] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:29.284492 22945 ts_tablet_manager.cc:1403] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.129s	user 0.011s	sys 0.003s
I20260709 13:06:29.287294 23097 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:29.287853 23097 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:29.288224 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:29.289233 22945 raft_consensus.cc:359] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.289865 22945 raft_consensus.cc:385] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.290128 22945 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.290231 22948 tablet_bootstrap.cc:492] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:29.290751 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:29.291044 22948 ts_tablet_manager.cc:1403] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.003s
I20260709 13:06:29.291873 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:29.293521 22948 raft_consensus.cc:359] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:29.294064 22719 leader_election.cc:343] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.294010 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:29.294355 22948 raft_consensus.cc:385] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.294533 22719 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:29.294654 22826 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.294831 22948 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.290747 22945 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.296249 22948 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.297528 22945 ts_tablet_manager.cc:1434] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:06:29.298054 22948 ts_tablet_manager.cc:1434] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:29.298949 23161 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:29.299070 22948 tablet_bootstrap.cc:492] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
W20260709 13:06:29.302647 22871 leader_election.cc:343] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:29.304482 22797 leader_election.cc:343] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.304948 22948 tablet_bootstrap.cc:654] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.307574 23092 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.308939 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:29.309571 22902 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.316509 22945 tablet_bootstrap.cc:492] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:29.319252 23161 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.319844 23161 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.322443 23161 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.318552 22946 tablet_bootstrap.cc:492] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:29.323911 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:29.324513 22825 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:29.323932 22946 ts_tablet_manager.cc:1403] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.071s	user 0.023s	sys 0.013s
I20260709 13:06:29.325830 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:29.326696 23161 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:29.327060 23161 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.327461 23161 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.327821 22946 raft_consensus.cc:359] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.328609 22946 raft_consensus.cc:385] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.328878 22946 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.329255 23152 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.329521 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.329723 23152 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.329493 22946 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.332623 22946 ts_tablet_manager.cc:1434] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:29.335037 22946 tablet_bootstrap.cc:492] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:29.335951 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:29.337232 22797 leader_election.cc:343] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.337391 23161 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.340039 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.340559 22946 tablet_bootstrap.cc:654] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.341429 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:06:29.342805 22719 leader_election.cc:343] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.343669 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:29.344419 22825 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.348369 22902 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:29.343751 23161 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:06:29.353437 22719 leader_election.cc:343] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.366113 23164 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.366726 23164 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:29.370028 22794 leader_election.cc:343] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.370632 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:29.372129 23156 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:29.379077 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.381083 22945 tablet_bootstrap.cc:654] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.352221 23152 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:06:29.382407 22797 leader_election.cc:343] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.369310 23164 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.386082 22754 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:29.408499 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.411623 22902 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:29.413733 22797 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:29.405295 22903 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:29.416394 22948 tablet_bootstrap.cc:492] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:29.416798 22948 ts_tablet_manager.cc:1403] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.118s	user 0.040s	sys 0.037s
I20260709 13:06:29.391921 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:29.419318 23156 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:29.419364 22826 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:29.420583 23156 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:29.422204 23156 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.427642 22825 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:29.428803 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:29.430022 23088 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:29.430466 23088 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:29.431192 23088 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.431479 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:29.420281 23164 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:29.443073 23092 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:29.445747 22948 raft_consensus.cc:359] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.448042 23163 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:29.451148 22948 raft_consensus.cc:385] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.449591 23092 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.455091 22948 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.456375 22948 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.457859 23092 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.459988 22946 tablet_bootstrap.cc:492] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:29.460394 22946 ts_tablet_manager.cc:1403] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.126s	user 0.013s	sys 0.015s
W20260709 13:06:29.461509 22719 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:29.485673 22945 tablet_bootstrap.cc:492] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:29.480625 22951 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.486200 22945 ts_tablet_manager.cc:1403] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.170s	user 0.017s	sys 0.009s
I20260709 13:06:29.496004 22946 raft_consensus.cc:359] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.496630 22946 raft_consensus.cc:385] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.496881 22946 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.478955 23089 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:06:29.502686 22653 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:29.498669 22951 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.510154 22946 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.522192 23164 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:29.517253 23160 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.524329 23160 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.524430 23164 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.527745 23160 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.528883 22946 ts_tablet_manager.cc:1434] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.001s
I20260709 13:06:29.529523 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.530346 22903 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:29.531273 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:29.532182 22826 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:29.490979 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:29.532989 23098 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:29.533123 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47014.8us
I20260709 13:06:29.533540 23098 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:29.534670 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:29.534418 23098 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.538175 22951 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:29.540036 22654 catalog_manager.cc:5697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:29.569021 22946 tablet_bootstrap.cc:492] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:29.572885 22948 ts_tablet_manager.cc:1434] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.156s	user 0.008s	sys 0.003s
I20260709 13:06:29.573894 22948 tablet_bootstrap.cc:492] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:29.549075 23161 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.576757 22945 raft_consensus.cc:359] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.581436 22945 raft_consensus.cc:385] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.581812 22945 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.583369 22945 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.588517 23092 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.594322 22945 ts_tablet_manager.cc:1434] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.103s	user 0.006s	sys 0.001s
I20260709 13:06:29.595194 22945 tablet_bootstrap.cc:492] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:29.602819 23161 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:29.691190 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:29.692961 22824 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:29.722052 23092 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.722967 23225 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.689133 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.723528 23225 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.724575 22756 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:29.730509 23163 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.731333 23163 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.737331 22951 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.749673 22951 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.766059 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:29.779867 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:29.780488 23225 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.781441 23097 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:29.781874 23097 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.790872 23097 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:29.792645 22797 leader_election.cc:343] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.792784 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:29.793612 22905 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.796285 23098 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.796825 23098 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.804848 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.805630 22899 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:29.806998 22948 tablet_bootstrap.cc:654] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.820259 22951 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:29.826900 23097 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.829566 23097 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:29.830601 22946 tablet_bootstrap.cc:654] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.834528 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.837920 23088 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.836494 23163 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.839385 23098 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:29.846417 22751 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:29.846681 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:29.846974 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.848760 23088 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.851236 23088 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.852058 22945 tablet_bootstrap.cc:654] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:29.851909 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:29.852617 22826 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.853802 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:29.854455 22824 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:29.862479 22654 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:29.863776 22948 tablet_bootstrap.cc:492] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:29.864403 22948 ts_tablet_manager.cc:1403] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.291s	user 0.010s	sys 0.032s
I20260709 13:06:29.868263 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:29.870558 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:29.872104 22719 leader_election.cc:343] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.875078 22905 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:29.876433 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.878706 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:29.881301 23156 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:29.881731 23156 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.882185 23156 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.884227 22797 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:06:29.888765 22797 leader_election.cc:343] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:29.891877 22794 leader_election.cc:343] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.892496 23223 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:29.892803 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:29.893651 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.897801 23092 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.898267 23164 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:29.898387 23092 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.898981 23164 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:29.900166 23164 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.888164 23152 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.894217 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:29.911778 23097 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:29.911571 23152 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.892855 23223 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.912628 23223 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.912166 23097 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.894280 22826 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:29.914979 23097 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.926723 23226 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:29.939558 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:29.954564 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:29.958371 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:29.960494 23098 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:29.965451 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.966070 23098 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:29.966996 23098 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.967461 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:29.968081 22755 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:29.969439 22946 tablet_bootstrap.cc:492] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:29.969990 22946 ts_tablet_manager.cc:1403] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.401s	user 0.033s	sys 0.014s
I20260709 13:06:29.959710 22752 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:29.964191 22825 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
W20260709 13:06:29.972066 22871 leader_election.cc:343] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:29.975600 23088 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:29.977144 23152 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:29.977813 22946 raft_consensus.cc:359] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.978518 22946 raft_consensus.cc:385] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:29.978816 22946 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:29.980480 22946 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.981530 22871 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:29.986835 22973 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:29.991325 23156 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.992538 23097 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:29.994562 23097 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:30.000470 22948 raft_consensus.cc:359] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.001173 22948 raft_consensus.cc:385] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.001525 22948 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.000954 22946 ts_tablet_manager.cc:1434] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
W20260709 13:06:30.003679 22797 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:30.004331 22797 leader_election.cc:343] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.005034 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:30.001932 23156 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.011955 22948 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.011377 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:30.007426 23225 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.018829 22945 tablet_bootstrap.cc:492] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:30.022748 22948 ts_tablet_manager.cc:1434] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.158s	user 0.007s	sys 0.003s
I20260709 13:06:30.015632 23225 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.024330 23225 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.032850 23152 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.034551 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:30.035485 22824 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 1: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:06:30.034986 23152 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.038406 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:30.024302 23088 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.040308 23088 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.040663 23088 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.043872 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:30.044094 22797 leader_election.cc:343] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:30.045357 22797 leader_election.cc:343] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.049120 22945 ts_tablet_manager.cc:1403] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.454s	user 0.011s	sys 0.026s
I20260709 13:06:30.045013 23092 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.062899 22946 tablet_bootstrap.cc:492] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:30.067929 23152 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.067369 22654 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:30.072391 23092 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:30.079294 22948 tablet_bootstrap.cc:492] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.077641 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:30.090060 23247 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:06:30.033381 23223 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.090289 22905 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.096643 22752 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:30.097718 23225 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.099691 23225 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.099741 23088 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:30.103742 22871 leader_election.cc:343] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.105018 22871 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:30.104465 22945 raft_consensus.cc:359] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.110128 22951 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.106519 22654 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:30.115866 22719 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:30.107205 22945 raft_consensus.cc:385] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.118810 22945 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.123678 23223 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.127032 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:30.127523 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:30.127696 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:30.132871 22903 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:30.128328 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:30.128484 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:30.126042 23088 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:06:30.125972 22702 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:30.134076 22904 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:30.137275 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:30.138064 22752 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 1: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:06:30.138356 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:30.138952 22754 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.139951 22951 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:30.141531 22945 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.146071 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:30.158201 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:30.158936 22829 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:06:30.160163 22948 tablet_bootstrap.cc:654] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.165819 22754 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:30.168054 22797 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:30.168855 23223 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.175434 22946 tablet_bootstrap.cc:654] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:30.176908 22719 leader_election.cc:343] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.178081 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:30.179955 23234 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.181187 22945 ts_tablet_manager.cc:1434] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.132s	user 0.006s	sys 0.001s
I20260709 13:06:30.170619 23223 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:30.180016 22653 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.186154 22945 tablet_bootstrap.cc:492] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:30.186194 23223 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.197259 23092 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:30.213131 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:30.216706 22905 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.219830 22946 tablet_bootstrap.cc:492] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
W20260709 13:06:30.220208 22794 leader_election.cc:343] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:30.222438 22794 leader_election.cc:343] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.221652 22946 ts_tablet_manager.cc:1403] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.159s	user 0.002s	sys 0.012s
I20260709 13:06:30.222870 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:06:30.225203 22794 leader_election.cc:343] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.225601 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.226276 23092 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.226665 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.227404 23092 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.227952 23092 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.228129 23223 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.228463 23092 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.237246 23092 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.227995 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.239511 23164 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.224004 22652 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.246753 23092 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.248361 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:30.248855 22946 raft_consensus.cc:359] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.249667 22946 raft_consensus.cc:385] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.250319 22946 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.251160 22946 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:30.254298 22797 leader_election.cc:343] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.255718 22905 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:30.256840 23234 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.258211 22946 ts_tablet_manager.cc:1434] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 13:06:30.257434 23234 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.262054 22945 tablet_bootstrap.cc:654] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.262514 23234 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.262633 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:30.297741 22948 tablet_bootstrap.cc:492] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.297909 22828 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:30.298228 22948 ts_tablet_manager.cc:1403] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.219s	user 0.015s	sys 0.013s
I20260709 13:06:30.299297 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:30.300688 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:06:30.301491 22719 leader_election.cc:343] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.302693 23234 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.303131 23234 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.303472 23234 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.313493 22948 raft_consensus.cc:359] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.315356 22948 raft_consensus.cc:385] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.314924 22946 tablet_bootstrap.cc:492] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:30.316747 23234 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.320326 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:30.320824 22828 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.315692 22948 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.326200 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:30.327570 22905 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.328382 22948 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.331111 23234 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.331991 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:30.333181 22948 ts_tablet_manager.cc:1434] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
W20260709 13:06:30.334141 22719 leader_election.cc:343] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.337013 22946 tablet_bootstrap.cc:654] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.337757 22948 tablet_bootstrap.cc:492] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.349336 22905 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:30.354719 22828 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:30.356073 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:30.356792 23234 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.357647 23234 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:30.359294 23234 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:30.359920 22797 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:30.360975 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.363740 23242 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.363854 22945 tablet_bootstrap.cc:492] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:30.364216 23242 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:30.364401 22945 ts_tablet_manager.cc:1403] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.178s	user 0.010s	sys 0.012s
I20260709 13:06:30.366536 22948 tablet_bootstrap.cc:654] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.367141 22945 raft_consensus.cc:359] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.365020 23242 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.375510 22945 raft_consensus.cc:385] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.375975 22945 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.377388 22945 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.379396 22945 ts_tablet_manager.cc:1434] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:06:30.379752 22652 catalog_manager.cc:5697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.382540 22945 tablet_bootstrap.cc:492] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:30.382616 22948 tablet_bootstrap.cc:492] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.383555 22948 ts_tablet_manager.cc:1403] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.002s
I20260709 13:06:30.386420 22948 raft_consensus.cc:359] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.386956 22946 tablet_bootstrap.cc:492] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:30.387003 22948 raft_consensus.cc:385] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.387475 22948 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.387555 22946 ts_tablet_manager.cc:1403] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.073s	user 0.017s	sys 0.004s
I20260709 13:06:30.388127 22948 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.390162 22948 ts_tablet_manager.cc:1434] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:30.391105 22948 tablet_bootstrap.cc:492] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.395160 22945 tablet_bootstrap.cc:654] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.395243 22652 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.395604 23242 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:30.397511 22948 tablet_bootstrap.cc:654] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.398537 22946 raft_consensus.cc:359] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.399154 22946 raft_consensus.cc:385] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.399407 22946 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.400216 22946 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.407661 22946 ts_tablet_manager.cc:1434] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 13:06:30.422075 23246 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.422572 23246 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.426141 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:30.426960 23246 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:06:30.427492 22868 leader_election.cc:343] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.428166 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:30.430711 22871 leader_election.cc:343] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.431190 22871 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:30.434065 23246 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.436561 22946 tablet_bootstrap.cc:492] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:30.437611 22948 tablet_bootstrap.cc:492] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.438086 22948 ts_tablet_manager.cc:1403] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.016s
I20260709 13:06:30.442008 22948 raft_consensus.cc:359] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.442633 22948 raft_consensus.cc:385] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.442936 22948 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.443683 22948 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.444974 22946 tablet_bootstrap.cc:654] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.445891 22948 ts_tablet_manager.cc:1434] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:06:30.447199 22948 tablet_bootstrap.cc:492] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.462270 22945 tablet_bootstrap.cc:492] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:30.462697 22945 ts_tablet_manager.cc:1403] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.080s	user 0.008s	sys 0.014s
I20260709 13:06:30.465575 23092 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.466015 23092 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.467985 23092 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.469895 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:30.469920 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:30.470633 22905 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:30.470749 22754 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:30.471665 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:30.460711 22948 tablet_bootstrap.cc:654] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.473102 23092 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.473492 23092 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.473836 23092 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.477097 22948 tablet_bootstrap.cc:492] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.477535 22948 ts_tablet_manager.cc:1403] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.005s
I20260709 13:06:30.479890 22948 raft_consensus.cc:359] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.480288 22948 raft_consensus.cc:385] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.481477 22948 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.481832 22945 raft_consensus.cc:359] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.482087 22948 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.482671 22945 raft_consensus.cc:385] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.483172 22945 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.483551 23246 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:30.484220 22948 ts_tablet_manager.cc:1434] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:06:30.483816 22945 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.482697 23092 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.485549 22948 tablet_bootstrap.cc:492] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.486071 22945 ts_tablet_manager.cc:1434] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:06:30.494915 22945 tablet_bootstrap.cc:492] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:30.498890 22946 tablet_bootstrap.cc:492] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:30.498935 22948 tablet_bootstrap.cc:654] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.499436 22946 ts_tablet_manager.cc:1403] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.016s
I20260709 13:06:30.500237 22945 tablet_bootstrap.cc:654] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.510604 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:30.511152 22754 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.514015 22946 raft_consensus.cc:359] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.514818 22946 raft_consensus.cc:385] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.515139 22946 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.515939 22946 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.516803 22951 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:30.517970 23092 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.519943 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:30.520533 22905 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.523296 22951 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.523736 22951 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.525480 22951 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:30.526098 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:30.526427 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:30.520021 22754 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:30.526990 22823 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
W20260709 13:06:30.527449 22871 leader_election.cc:343] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.528012 22948 tablet_bootstrap.cc:492] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.528270 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:30.528455 22948 ts_tablet_manager.cc:1403] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.008s
I20260709 13:06:30.529165 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.530161 22951 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.530493 22951 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.530776 22951 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.531374 22948 raft_consensus.cc:359] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.532084 22948 raft_consensus.cc:385] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.532428 22948 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.533092 22948 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.533540 22905 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:30.534139 23152 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.534595 23152 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:30.535247 22948 ts_tablet_manager.cc:1434] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:30.535399 23152 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.536582 22951 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.538102 22946 ts_tablet_manager.cc:1434] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:06:30.539060 22946 tablet_bootstrap.cc:492] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:30.543941 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:30.544840 22946 tablet_bootstrap.cc:654] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.545614 23092 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.546110 23092 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:30.546581 22871 leader_election.cc:343] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.550329 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:30.556257 22794 leader_election.cc:343] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.558022 23092 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.558928 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:30.561678 22951 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:30.565227 23236 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.565672 23236 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.567574 23236 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:06:30.567802 22797 leader_election.cc:343] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.567866 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:30.568576 22823 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.568454 22797 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.569087 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:30.571135 23223 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.572770 22945 tablet_bootstrap.cc:492] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:30.573257 22945 ts_tablet_manager.cc:1403] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.079s	user 0.006s	sys 0.006s
I20260709 13:06:30.573536 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:30.575739 22945 raft_consensus.cc:359] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.576373 22945 raft_consensus.cc:385] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.576637 22945 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.577782 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:30.578510 22946 tablet_bootstrap.cc:492] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:30.577260 22945 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.579051 22946 ts_tablet_manager.cc:1403] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.017s
I20260709 13:06:30.581600 22948 tablet_bootstrap.cc:492] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.583968 22945 ts_tablet_manager.cc:1434] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:06:30.584131 22823 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:30.585147 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.585877 22945 tablet_bootstrap.cc:492] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:30.586117 23229 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:30.586560 23229 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:30.587303 23229 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.588086 22948 tablet_bootstrap.cc:654] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.590777 23160 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.591435 22946 raft_consensus.cc:359] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.592106 22946 raft_consensus.cc:385] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.592423 22946 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.593078 22946 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.594691 22945 tablet_bootstrap.cc:654] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.595652 22946 ts_tablet_manager.cc:1434] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 13:06:30.622596 22946 tablet_bootstrap.cc:492] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:30.630963 23152 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:30.638450 22948 tablet_bootstrap.cc:492] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.642791 22948 ts_tablet_manager.cc:1403] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.061s	user 0.017s	sys 0.000s
I20260709 13:06:30.642267 22945 tablet_bootstrap.cc:492] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:30.644239 22945 ts_tablet_manager.cc:1403] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.018s
I20260709 13:06:30.643399 22653 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.646752 22945 raft_consensus.cc:359] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.647382 22945 raft_consensus.cc:385] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.647661 22945 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.649880 22948 raft_consensus.cc:359] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.644369 22946 tablet_bootstrap.cc:654] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.652899 22948 raft_consensus.cc:385] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.653678 22948 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.654516 22948 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.658422 22948 ts_tablet_manager.cc:1434] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:06:30.659554 22948 tablet_bootstrap.cc:492] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.652556 22945 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.666266 22945 ts_tablet_manager.cc:1434] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:06:30.652329 22652 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:30.667394 22945 tablet_bootstrap.cc:492] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:30.686863 22945 tablet_bootstrap.cc:654] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.692041 22946 tablet_bootstrap.cc:492] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:30.692538 22946 ts_tablet_manager.cc:1403] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.021s
I20260709 13:06:30.693692 23092 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:30.696022 22946 raft_consensus.cc:359] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.696754 22946 raft_consensus.cc:385] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.697057 22946 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.697808 22946 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.699350 22948 tablet_bootstrap.cc:654] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.700206 22946 ts_tablet_manager.cc:1434] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.007s
I20260709 13:06:30.701709 22946 tablet_bootstrap.cc:492] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:30.704263 22945 tablet_bootstrap.cc:492] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:30.704747 22945 ts_tablet_manager.cc:1403] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.000s
I20260709 13:06:30.707454 22946 tablet_bootstrap.cc:654] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.708272 22945 raft_consensus.cc:359] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.709233 22945 raft_consensus.cc:385] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.709617 22945 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.710652 22945 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.713356 22945 ts_tablet_manager.cc:1434] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:30.714383 22945 tablet_bootstrap.cc:492] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:30.718756 23229 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.719648 22948 tablet_bootstrap.cc:492] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.719798 22945 tablet_bootstrap.cc:654] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.720081 22948 ts_tablet_manager.cc:1403] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.009s
I20260709 13:06:30.719359 23229 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.722803 22948 raft_consensus.cc:359] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.723830 22948 raft_consensus.cc:385] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.723716 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:30.724291 22948 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.724849 22752 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 1: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:06:30.725087 22948 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.727617 23092 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.727813 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:30.728576 22828 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 1: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:06:30.728191 23092 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.729642 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:30.730561 23246 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.731103 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:30.731351 22945 tablet_bootstrap.cc:492] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:30.732200 22948 ts_tablet_manager.cc:1434] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:06:30.732525 22945 ts_tablet_manager.cc:1403] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.009s
I20260709 13:06:30.731302 23092 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:06:30.732887 22797 leader_election.cc:343] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.733558 22948 tablet_bootstrap.cc:492] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.735160 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:30.722822 23229 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:30.740216 22948 tablet_bootstrap.cc:654] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:30.741636 22794 leader_election.cc:343] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.742218 22794 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.744374 23152 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.749001 22948 tablet_bootstrap.cc:492] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.749425 22948 ts_tablet_manager.cc:1403] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.005s
I20260709 13:06:30.751384 22948 raft_consensus.cc:359] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.751842 22948 raft_consensus.cc:385] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.752060 22948 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.752619 22948 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.755292 22948 ts_tablet_manager.cc:1434] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:30.757268 22948 tablet_bootstrap.cc:492] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.760218 22945 raft_consensus.cc:359] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.760919 22945 raft_consensus.cc:385] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.761160 22945 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.761767 22945 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.763190 22948 tablet_bootstrap.cc:654] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.763607 22945 ts_tablet_manager.cc:1434] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.000s
I20260709 13:06:30.764542 22945 tablet_bootstrap.cc:492] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:30.773598 22945 tablet_bootstrap.cc:654] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.780936 22948 tablet_bootstrap.cc:492] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.781587 22948 ts_tablet_manager.cc:1403] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.000s
I20260709 13:06:30.784664 22945 tablet_bootstrap.cc:492] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:30.784358 22948 raft_consensus.cc:359] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.785069 22945 ts_tablet_manager.cc:1403] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.007s
I20260709 13:06:30.785187 22948 raft_consensus.cc:385] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.785583 22948 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.786262 22948 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.787734 22945 raft_consensus.cc:359] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.788335 22945 raft_consensus.cc:385] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.788352 22948 ts_tablet_manager.cc:1434] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:30.788698 22945 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.789511 22948 tablet_bootstrap.cc:492] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.789302 22945 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.791971 22945 ts_tablet_manager.cc:1434] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:30.792953 22945 tablet_bootstrap.cc:492] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:30.799182 22945 tablet_bootstrap.cc:654] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.799614 22948 tablet_bootstrap.cc:654] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.804513 23092 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.805150 23092 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.808725 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:30.808745 23092 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.810120 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:30.811038 22797 leader_election.cc:343] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:30.812284 22794 leader_election.cc:343] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.812855 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.813983 22948 tablet_bootstrap.cc:492] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.814437 22948 ts_tablet_manager.cc:1403] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.002s
I20260709 13:06:30.814349 23152 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.814883 22945 tablet_bootstrap.cc:492] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:30.815331 22945 ts_tablet_manager.cc:1403] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.003s
I20260709 13:06:30.817276 22948 raft_consensus.cc:359] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.817988 22948 raft_consensus.cc:385] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.817679 22945 raft_consensus.cc:359] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.818374 22948 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.819061 22948 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.821009 22948 ts_tablet_manager.cc:1434] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.005s
I20260709 13:06:30.818403 22945 raft_consensus.cc:385] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.822275 22945 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.822903 22945 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.823516 23246 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.824448 22752 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.825340 22945 ts_tablet_manager.cc:1434] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:06:30.829696 23242 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.830150 23242 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.832227 23242 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.832757 22948 tablet_bootstrap.cc:492] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.834569 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:30.836551 22797 leader_election.cc:343] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.838311 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:30.839128 22904 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:30.839752 22948 tablet_bootstrap.cc:654] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.840525 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:30.841984 23242 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.842458 23242 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.842864 23242 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.844975 22948 tablet_bootstrap.cc:492] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:30.845357 22948 ts_tablet_manager.cc:1403] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:06:30.848694 22948 raft_consensus.cc:359] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.849386 22948 raft_consensus.cc:385] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.849654 22948 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.850318 22948 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.854017 22948 ts_tablet_manager.cc:1434] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:30.854146 22752 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.855141 22948 tablet_bootstrap.cc:492] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:30.863190 22945 tablet_bootstrap.cc:492] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:30.872785 23236 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:30.883320 22945 tablet_bootstrap.cc:654] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.893998 23098 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.894439 23098 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.881253 23242 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.901003 23098 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:30.903226 22948 tablet_bootstrap.cc:654] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:30.898641 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:30.906715 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:30.909215 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:30.910661 23246 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:30.917363 23092 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:30.919378 23092 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.919742 23092 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.921311 23092 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.922307 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:30.922039 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:30.925450 22904 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:30.925696 22753 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:30.926975 22794 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:30.931876 23223 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:30.929198 23092 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:30.933609 23092 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.934006 23092 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.941817 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:06:30.943811 22797 leader_election.cc:343] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:30.952481 22946 tablet_bootstrap.cc:492] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:30.953099 22946 ts_tablet_manager.cc:1403] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.252s	user 0.063s	sys 0.086s
I20260709 13:06:30.963828 23223 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.964614 23223 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.971930 23092 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.986907 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:30.987587 22753 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:30.987457 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:30.990367 23092 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:30.993690 22946 raft_consensus.cc:359] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.997849 22946 raft_consensus.cc:385] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:30.998113 22946 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:30.975647 23242 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.001825 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:31.002104 22905 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.002374 22900 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.004283 22946 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:30.997220 23223 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.019989 22945 tablet_bootstrap.cc:492] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:31.020465 22945 ts_tablet_manager.cc:1403] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.158s	user 0.011s	sys 0.013s
W20260709 13:06:31.021937 22797 leader_election.cc:343] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.024072 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:31.024626 22903 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.027324 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:31.028482 22754 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.024585 22945 raft_consensus.cc:359] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.029449 22945 raft_consensus.cc:385] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.029826 23236 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.029999 22945 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.030294 23236 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.030642 22945 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.031283 23229 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.031770 23229 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.032176 23236 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.032778 22945 ts_tablet_manager.cc:1434] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
W20260709 13:06:31.032866 22794 leader_election.cc:343] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.033339 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.033675 22945 tablet_bootstrap.cc:492] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:31.034238 23152 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:31.038468 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:31.038548 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:31.039566 22904 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:31.041601 22828 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:06:31.044234 22948 tablet_bootstrap.cc:492] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:31.041993 23229 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.041637 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:31.046672 22871 leader_election.cc:343] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.047359 22948 ts_tablet_manager.cc:1403] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.192s	user 0.023s	sys 0.024s
I20260709 13:06:31.047767 22946 ts_tablet_manager.cc:1434] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.088s	user 0.011s	sys 0.002s
I20260709 13:06:31.050248 22948 raft_consensus.cc:359] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.051038 22948 raft_consensus.cc:385] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.051528 22948 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.051651 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:31.050738 22946 tablet_bootstrap.cc:492] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:31.052282 22948 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.053544 23160 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.054731 22948 ts_tablet_manager.cc:1434] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:31.055759 22948 tablet_bootstrap.cc:492] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:31.056816 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:31.057600 22828 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:31.058760 22946 tablet_bootstrap.cc:654] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.063407 22945 tablet_bootstrap.cc:654] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.064180 22948 tablet_bootstrap.cc:654] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.081223 22753 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:31.084550 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.085863 23098 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.086442 23098 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:31.087570 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:31.088119 23229 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:31.088464 23229 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.088912 23229 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.093020 22797 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:31.094916 23152 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.095330 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.095423 23152 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:31.095715 23156 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:31.096613 22871 leader_election.cc:343] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.097667 23098 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.098009 23160 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:31.096577 23152 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.100600 22900 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:31.102463 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.104228 23297 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.104825 23297 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:31.111389 23297 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.119654 23092 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.120154 23092 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.123160 23234 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.123323 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:31.125561 23092 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.123793 23234 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.128000 22903 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
W20260709 13:06:31.128386 22794 leader_election.cc:343] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.138469 23223 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.138928 23223 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.140355 22903 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.146606 23223 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.148352 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:31.153898 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:31.155761 22794 leader_election.cc:343] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.151567 23234 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.157214 23098 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.157722 23098 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.159890 23098 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.161471 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:06:31.162899 22871 leader_election.cc:343] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.177992 23234 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.186214 22945 tablet_bootstrap.cc:492] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:31.187966 22945 ts_tablet_manager.cc:1403] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.155s	user 0.013s	sys 0.010s
I20260709 13:06:31.189677 23297 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.195159 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.196314 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.198021 23152 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.199877 22946 tablet_bootstrap.cc:492] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:31.200280 22946 ts_tablet_manager.cc:1403] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.150s	user 0.034s	sys 0.082s
I20260709 13:06:31.190313 23297 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.203596 22946 raft_consensus.cc:359] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.204279 22946 raft_consensus.cc:385] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.204540 22946 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.205149 22946 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.207104 22946 ts_tablet_manager.cc:1434] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:31.208288 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:31.208856 22905 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:31.209888 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:31.210121 22946 tablet_bootstrap.cc:492] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:31.209245 22653 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.210640 23223 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:31.210988 23223 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.211315 23223 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.211220 22653 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.213454 23297 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.214416 22945 raft_consensus.cc:359] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.215181 22945 raft_consensus.cc:385] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.215467 22945 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.215615 22946 tablet_bootstrap.cc:654] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.220687 22945 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.222594 22948 tablet_bootstrap.cc:492] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:31.223101 22948 ts_tablet_manager.cc:1403] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.168s	user 0.017s	sys 0.013s
I20260709 13:06:31.223423 22945 ts_tablet_manager.cc:1434] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:06:31.224433 22945 tablet_bootstrap.cc:492] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:31.229610 22948 raft_consensus.cc:359] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.230415 22948 raft_consensus.cc:385] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.231863 22948 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.232836 22948 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.241830 22948 ts_tablet_manager.cc:1434] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:06:31.243348 22948 tablet_bootstrap.cc:492] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
W20260709 13:06:31.252863 22702 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:31.239270 22945 tablet_bootstrap.cc:654] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.260587 22948 tablet_bootstrap.cc:654] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.269925 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:31.271404 22945 tablet_bootstrap.cc:492] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:31.271826 22945 ts_tablet_manager.cc:1403] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.001s
I20260709 13:06:31.206598 23229 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.274099 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:31.275350 23229 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.275755 22823 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:31.275935 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:06:31.277529 22871 leader_election.cc:343] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.280057 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.276070 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:31.281245 22831 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:31.282518 23223 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.285238 22945 raft_consensus.cc:359] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.289844 22945 raft_consensus.cc:385] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.288647 22948 tablet_bootstrap.cc:492] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:31.290413 22945 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.290623 22948 ts_tablet_manager.cc:1403] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.000s
I20260709 13:06:31.291464 23152 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.291270 22945 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.289389 22719 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:06:31.291828 22797 leader_election.cc:343] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.292460 22797 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.292793 23234 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:31.293356 22797 leader_election.cc:343] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.289844 23223 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.293752 22797 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.294342 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.294661 23098 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.286208 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:31.295267 23242 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:31.296773 22899 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.291904 23152 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.298888 23152 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.300696 23098 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:31.301064 23098 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.301682 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:31.302785 23098 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.302946 22901 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:31.304078 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:31.304795 23242 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:31.305150 23242 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.305500 23242 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.308947 22899 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:31.310128 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:31.310861 23152 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.311300 23152 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:31.312031 23098 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.313454 23152 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.296284 22948 raft_consensus.cc:359] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.316474 22948 raft_consensus.cc:385] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.316406 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:31.316820 22948 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.317646 22946 tablet_bootstrap.cc:492] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
W20260709 13:06:31.317750 22871 leader_election.cc:343] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.318161 22948 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.318534 23156 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:31.318894 22946 ts_tablet_manager.cc:1403] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.108s	user 0.013s	sys 0.049s
I20260709 13:06:31.320719 22948 ts_tablet_manager.cc:1434] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:06:31.321713 22948 tablet_bootstrap.cc:492] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:31.322638 23098 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.324198 23242 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.326347 23242 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.295773 22945 ts_tablet_manager.cc:1434] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:06:31.327775 22945 tablet_bootstrap.cc:492] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:31.329823 23247 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.330307 23247 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.321307 22946 raft_consensus.cc:359] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.331410 22946 raft_consensus.cc:385] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.331696 22946 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.332625 23247 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.333966 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.334580 22753 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:31.336009 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:31.336488 22945 tablet_bootstrap.cc:654] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.332286 22946 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.338768 23098 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:31.339946 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:31.340515 22899 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:31.321589 22797 leader_election.cc:343] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.344929 23098 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.346896 22946 ts_tablet_manager.cc:1434] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:06:31.346962 23098 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.347882 22946 tablet_bootstrap.cc:492] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:31.350750 22948 tablet_bootstrap.cc:654] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.350778 22899 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:31.352227 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:31.353683 23152 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.354733 23152 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:31.355443 23098 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.355547 23152 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.357452 23098 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.347451 22654 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.358700 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.361035 22945 tablet_bootstrap.cc:492] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:31.360978 23229 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:31.361452 23229 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.361611 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:31.361866 23229 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.361531 22945 ts_tablet_manager.cc:1403] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.004s
I20260709 13:06:31.363066 22750 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.365516 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:31.366824 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:31.369302 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.370141 22945 raft_consensus.cc:359] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.370849 22945 raft_consensus.cc:385] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.371086 22945 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.371730 22945 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.373296 22945 ts_tablet_manager.cc:1434] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.000s
I20260709 13:06:31.374286 22945 tablet_bootstrap.cc:492] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:31.379693 22946 tablet_bootstrap.cc:654] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:31.394721 22797 leader_election.cc:343] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:31.395566 22797 leader_election.cc:343] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:31.396541 22797 leader_election.cc:343] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.404477 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:31.405090 22829 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:31.401760 23229 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.410305 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:31.410965 22828 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.413934 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:31.414690 22830 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.418138 22945 tablet_bootstrap.cc:654] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.425917 22948 tablet_bootstrap.cc:492] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:31.425951 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:31.426484 22948 ts_tablet_manager.cc:1403] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.105s	user 0.017s	sys 0.026s
I20260709 13:06:31.419443 22750 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:31.421558 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:31.440459 23229 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.440927 22823 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.442945 22948 raft_consensus.cc:359] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.444900 22948 raft_consensus.cc:385] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.445186 22948 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.448236 22948 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.450296 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:31.450914 22827 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.452615 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:06:31.453259 22871 leader_election.cc:343] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.457603 23098 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.458348 23098 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:31.459326 23098 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.456621 22654 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:31.489238 22948 ts_tablet_manager.cc:1434] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.062s	user 0.008s	sys 0.000s
I20260709 13:06:31.492846 22946 tablet_bootstrap.cc:492] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:31.493482 22946 ts_tablet_manager.cc:1403] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.146s	user 0.026s	sys 0.000s
I20260709 13:06:31.494956 22948 tablet_bootstrap.cc:492] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Bootstrap starting.
I20260709 13:06:31.497529 22828 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:31.499063 22945 tablet_bootstrap.cc:492] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:31.499663 22830 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:31.503371 23234 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.503821 23234 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.499475 22945 ts_tablet_manager.cc:1403] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.125s	user 0.043s	sys 0.021s
I20260709 13:06:31.506501 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.508608 23234 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.514061 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:31.514909 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:31.515573 23247 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.516057 23247 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:31.516737 22946 raft_consensus.cc:359] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.516819 23247 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.517437 22946 raft_consensus.cc:385] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.517999 22946 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.519547 22823 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:31.521772 22946 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.515600 22899 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:06:31.525444 22750 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.526620 22946 ts_tablet_manager.cc:1434] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:06:31.529038 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:31.539987 22827 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:31.540653 22948 tablet_bootstrap.cc:654] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.551685 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.562940 22945 raft_consensus.cc:359] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.563651 22945 raft_consensus.cc:385] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.564064 22945 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.563474 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:31.571446 22946 tablet_bootstrap.cc:492] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:31.568969 22652 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:31.574347 22945 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.577433 23229 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.579459 22945 ts_tablet_manager.cc:1434] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.000s
I20260709 13:06:31.586759 23246 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:31.587221 23246 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:31.588398 22945 tablet_bootstrap.cc:492] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:31.588027 23246 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.591257 23092 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.590830 23234 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:31.648487 23298 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:31.672073 23297 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.672467 23092 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:31.682746 23297 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.683318 23297 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.625579 23229 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:31.715265 23246 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.715900 23246 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.731133 23229 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.732689 23246 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.701737 23152 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.743319 22948 tablet_bootstrap.cc:492] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: No bootstrap required, opened a new log
I20260709 13:06:31.743862 22948 ts_tablet_manager.cc:1403] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Time spent bootstrapping tablet: real 0.253s	user 0.018s	sys 0.002s
I20260709 13:06:31.738469 23246 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.765115 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:31.765435 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:31.765861 22823 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:31.767554 23246 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.769562 23246 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:31.762665 23152 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.774657 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:06:31.771302 22794 leader_election.cc:343] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:31.778899 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:31.779095 23325 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:31.779960 23325 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.780380 23325 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.779552 22830 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:31.735239 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.786038 22946 tablet_bootstrap.cc:700] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Time spent opening tablet: real 0.146s	user 0.015s	sys 0.018s
I20260709 13:06:31.786648 22945 tablet_bootstrap.cc:700] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent opening tablet: real 0.139s	user 0.008s	sys 0.024s
I20260709 13:06:31.746652 23297 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.874528 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:31.787695 22948 raft_consensus.cc:359] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.875223 22948 raft_consensus.cc:385] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:31.875537 22948 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Initialized, Role: FOLLOWER
I20260709 13:06:31.881793 22946 tablet_bootstrap.cc:654] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.867787 22945 tablet_bootstrap.cc:654] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:31.891351 22948 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.858852 23236 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.888038 23322 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:31.904589 23092 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.904939 23092 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:31.791901 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 140147us
I20260709 13:06:31.896062 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:31.898593 23322 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:31.922910 23329 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:06:31.924412 22948 ts_tablet_manager.cc:1434] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Time spent starting tablet: real 0.180s	user 0.010s	sys 0.013s
I20260709 13:06:31.931972 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.898337 23236 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.919884 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.934966 22752 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:31.921984 23322 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:31.924669 22899 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 0.
I20260709 13:06:31.936669 23236 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.975183 23092 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.976440 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:31.986415 23152 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:31.986663 23325 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:31.990268 23325 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:32.000638 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:32.002183 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:32.003253 22900 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:32.003399 23152 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:32.004384 23298 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:32.004774 23298 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.005097 23298 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.010185 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:32.014878 23337 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:32.035960 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:32.036753 22896 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:32.038156 23338 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:32.038537 23338 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.038887 23338 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:06:32.046936 22871 leader_election.cc:343] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:32.049052 22871 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:32.056032 22945 tablet_bootstrap.cc:492] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:32.053076 23298 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.056484 22945 ts_tablet_manager.cc:1403] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.468s	user 0.040s	sys 0.070s
I20260709 13:06:32.064952 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:32.065645 22755 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
W20260709 13:06:32.066722 22797 leader_election.cc:343] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:32.067135 22797 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:32.068032 23234 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.067927 23337 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:32.068969 22797 leader_election.cc:343] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:32.068596 23234 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:32.076592 22797 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:32.078495 22797 consensus_peers.cc:597] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:32.084539 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:32.084949 23234 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.085345 22823 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:32.091924 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:32.092428 22831 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:32.100226 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:32.100282 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:32.101305 23234 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:32.102769 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:32.101693 23234 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.105055 23234 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.048394 23322 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.109841 23322 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:32.112268 22946 tablet_bootstrap.cc:492] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.112453 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:32.112829 22946 ts_tablet_manager.cc:1403] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.542s	user 0.029s	sys 0.071s
I20260709 13:06:32.113374 23160 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:32.113829 23160 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.114140 23160 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:32.115767 22946 raft_consensus.cc:359] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.116528 22946 raft_consensus.cc:385] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.116801 22946 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.117528 22946 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.119783 22946 ts_tablet_manager.cc:1434] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:32.120754 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:32.121328 22831 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.121645 22946 tablet_bootstrap.cc:492] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.126309 22945 raft_consensus.cc:359] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.127563 22946 tablet_bootstrap.cc:654] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.115819 23234 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.157739 22751 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 1: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:06:32.158556 23234 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.163858 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:32.164426 22825 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.164980 22945 raft_consensus.cc:385] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.166553 22945 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.167574 22945 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.063665 23298 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.133620 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:06:32.134961 22871 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:32.135012 23229 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:32.182344 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 47474.1us
I20260709 13:06:32.182400 22903 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:32.138372 22831 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:32.139047 23338 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.129057 23160 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.148838 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:32.186352 23322 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:32.187608 22897 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.187922 23338 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.188479 23160 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.140553 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:32.189596 23229 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:32.191718 22945 ts_tablet_manager.cc:1434] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.135s	user 0.001s	sys 0.005s
I20260709 13:06:32.192607 22945 tablet_bootstrap.cc:492] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:32.193956 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:32.194437 22897 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:06:32.194834 22903 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:32.179569 22825 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:32.199355 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:32.201594 23297 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:32.201696 23229 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:32.202220 23297 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:32.203037 23297 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:32.204474 22871 leader_election.cc:343] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:32.207563 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:32.208211 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:32.208762 22903 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:32.209899 22945 tablet_bootstrap.cc:654] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.208178 22897 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:32.219179 23337 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.219802 23337 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.222898 22945 tablet_bootstrap.cc:492] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:32.223305 22945 ts_tablet_manager.cc:1403] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:06:32.227415 22946 tablet_bootstrap.cc:492] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.227933 22946 ts_tablet_manager.cc:1403] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.106s	user 0.009s	sys 0.008s
I20260709 13:06:32.227495 22897 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
W20260709 13:06:32.231187 22871 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:32.230904 22946 raft_consensus.cc:359] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.232059 22946 raft_consensus.cc:385] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.232460 22946 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.232084 22945 raft_consensus.cc:359] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.233184 22945 raft_consensus.cc:385] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.233124 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:32.234640 23234 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:32.235169 23234 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:06:32.235373 22797 leader_election.cc:343] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:32.236018 23234 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.233716 22945 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.238757 22945 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.233124 22946 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.242213 22945 ts_tablet_manager.cc:1434] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:06:32.235268 23337 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.243446 22945 tablet_bootstrap.cc:492] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:32.244110 22946 ts_tablet_manager.cc:1434] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260709 13:06:32.251530 22903 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:06:32.251578 22946 tablet_bootstrap.cc:492] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.259502 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:32.260330 22831 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 1: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:06:32.262662 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:32.263217 22831 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:32.267661 22946 tablet_bootstrap.cc:654] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:32.272243 22931 tablet.cc:2406] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:32.270682 22831 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:32.274605 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:32.275717 23098 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:32.276194 23098 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:32.270988 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:32.277086 23098 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.278861 23242 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:32.279453 23242 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:32.280355 23242 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.281818 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:32.282469 22751 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:32.288384 22652 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:32.290052 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:32.291170 23322 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:32.291718 23322 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:32.292570 23322 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.293355 22946 tablet_bootstrap.cc:492] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.293870 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:32.294864 22946 ts_tablet_manager.cc:1403] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.004s
I20260709 13:06:32.295869 22753 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.297600 22946 raft_consensus.cc:359] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.298242 22946 raft_consensus.cc:385] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.298506 22946 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.299110 22946 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.300156 22945 tablet_bootstrap.cc:654] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.292551 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:32.303901 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:32.304811 23160 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:06:32.306847 22794 leader_election.cc:343] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:32.307090 22946 ts_tablet_manager.cc:1434] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:06:32.308131 22946 tablet_bootstrap.cc:492] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
W20260709 13:06:32.326472 22797 leader_election.cc:343] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:32.332050 22797 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:32.346534 23242 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:32.356256 22754 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.362046 22946 tablet_bootstrap.cc:654] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:32.370275 22702 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:32.378144 23098 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:32.371606 22755 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.386214 22945 tablet_bootstrap.cc:492] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:32.386719 22945 ts_tablet_manager.cc:1403] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.144s	user 0.014s	sys 0.019s
I20260709 13:06:32.377003 22751 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:06:32.404577 22945 raft_consensus.cc:359] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.407074 22945 raft_consensus.cc:385] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.407366 22945 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.408608 22945 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.410516 22945 ts_tablet_manager.cc:1434] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:06:32.411377 22945 tablet_bootstrap.cc:492] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Bootstrap starting.
I20260709 13:06:32.424819 22945 tablet_bootstrap.cc:654] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.439671 22945 tablet_bootstrap.cc:492] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: No bootstrap required, opened a new log
I20260709 13:06:32.440188 22945 ts_tablet_manager.cc:1403] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.004s
I20260709 13:06:32.449159 23326 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:06:32.449747 23326 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.455052 23326 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.455893 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:32.459220 22654 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.460089 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:32.460714 22654 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.464224 22654 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.465852 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:32.467674 23326 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:32.467435 22653 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 2, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:32.470302 22946 tablet_bootstrap.cc:492] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.470815 22946 ts_tablet_manager.cc:1403] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.163s	user 0.014s	sys 0.013s
I20260709 13:06:32.475036 22946 raft_consensus.cc:359] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.475685 22946 raft_consensus.cc:385] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.475932 22946 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.482524 22946 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.486438 22946 ts_tablet_manager.cc:1434] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:06:32.487279 22946 tablet_bootstrap.cc:492] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.490422 22652 catalog_manager.cc:5697] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 0 to 2, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:32.494532 22946 tablet_bootstrap.cc:654] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.490302 22945 raft_consensus.cc:359] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.495085 22945 raft_consensus.cc:385] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.495362 22945 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.496085 22945 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.498544 22945 ts_tablet_manager.cc:1434] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.000s
I20260709 13:06:32.508601 22946 tablet_bootstrap.cc:492] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.508988 22946 ts_tablet_manager.cc:1403] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.000s
I20260709 13:06:32.510754 22946 raft_consensus.cc:359] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.511276 22946 raft_consensus.cc:385] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.511456 22946 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.511919 22946 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.513676 22946 ts_tablet_manager.cc:1434] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:06:32.514585 22946 tablet_bootstrap.cc:492] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.519807 22946 tablet_bootstrap.cc:654] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.521461 23246 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.521921 23246 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.523854 23246 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:32.525131 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:32.525497 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:32.525741 22824 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:32.526072 22753 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:32.527118 22871 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:32.527797 23246 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:32.528108 23246 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.528422 23246 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.530730 22654 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.533097 22654 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.534915 22946 tablet_bootstrap.cc:492] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.535283 22946 ts_tablet_manager.cc:1403] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.003s
I20260709 13:06:32.535459 23246 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.537279 22946 raft_consensus.cc:359] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.537928 22946 raft_consensus.cc:385] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.538100 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:32.538318 22946 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.538626 22753 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.539022 22946 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.539561 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:32.540096 22823 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.543851 22946 ts_tablet_manager.cc:1434] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 13:06:32.544121 23322 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.544765 22946 tablet_bootstrap.cc:492] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.544576 23322 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.545447 22753 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:32.546563 23322 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:32.547737 22871 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:32.548956 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:32.550679 22946 tablet_bootstrap.cc:654] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.551681 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:32.552542 22824 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:32.553610 23323 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:32.554075 23323 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:32.554847 23323 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:32.561635 22871 leader_election.cc:343] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:32.562366 22946 tablet_bootstrap.cc:492] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.562760 22946 ts_tablet_manager.cc:1403] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 13:06:32.564194 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:32.565358 22946 raft_consensus.cc:359] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.565766 23098 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:32.566006 22946 raft_consensus.cc:385] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.566097 23098 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.566291 22946 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.566406 23098 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.567013 22946 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.568018 22823 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:32.569109 22946 ts_tablet_manager.cc:1434] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:06:32.569974 22946 tablet_bootstrap.cc:492] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.570659 23246 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:32.575487 22946 tablet_bootstrap.cc:654] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.589041 23234 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.589687 23234 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.590586 22946 tablet_bootstrap.cc:492] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.590998 22946 ts_tablet_manager.cc:1403] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.003s
I20260709 13:06:32.592402 23234 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.593415 22946 raft_consensus.cc:359] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.595587 22946 raft_consensus.cc:385] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.595856 22946 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.596536 22946 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.598649 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:32.599741 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:32.600342 22946 ts_tablet_manager.cc:1434] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:32.601253 22946 tablet_bootstrap.cc:492] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.600092 23098 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.601899 23234 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.608702 23234 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.605238 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:32.611160 23326 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:32.612434 23234 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.613306 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:32.613987 23098 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:32.614228 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:32.614316 22754 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.607524 22946 tablet_bootstrap.cc:654] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.617082 22897 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:32.618880 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:32.619647 23326 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:32.620069 23326 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.620419 23326 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.628659 23326 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.633003 23326 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.634074 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:32.634277 22753 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.634696 22897 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.639840 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:32.640244 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:32.640770 22824 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.640583 22829 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 0.
I20260709 13:06:32.642331 22897 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:32.646210 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:32.646761 22831 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.648554 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:32.649297 23234 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:32.649763 23234 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:32.650746 23234 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.653362 22831 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:06:32.653750 23098 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.654402 22824 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:32.654181 23098 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.658530 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:06:32.645555 22754 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
W20260709 13:06:32.660315 22871 leader_election.cc:343] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:32.662475 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:32.663244 23323 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:32.663884 23323 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:32.664156 22897 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:32.664733 23323 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.665977 23160 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:32.659520 22653 catalog_manager.cc:5697] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.674989 22654 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:32.678608 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:32.679279 22831 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 0.
I20260709 13:06:32.680233 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:32.684792 23330 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:32.685227 23330 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.685634 23330 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.696097 22946 tablet_bootstrap.cc:492] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.696760 22946 ts_tablet_manager.cc:1403] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.001s
I20260709 13:06:32.696403 23330 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.699772 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:32.696424 23098 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:06:32.701077 22871 leader_election.cc:343] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:32.701428 23330 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:32.704008 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:32.704589 22831 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.711515 22831 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:06:32.711233 22946 raft_consensus.cc:359] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.712504 22946 raft_consensus.cc:385] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.712845 22946 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.713740 22654 catalog_manager.cc:5697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.713681 22946 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.716361 22946 ts_tablet_manager.cc:1434] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.019s	user 0.000s	sys 0.005s
I20260709 13:06:32.717216 22946 tablet_bootstrap.cc:492] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.723670 22946 tablet_bootstrap.cc:654] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.731245 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:32.732280 23330 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:32.733670 23330 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:32.734557 23330 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.738302 22831 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:32.744180 23234 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:32.757397 22946 tablet_bootstrap.cc:492] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.757939 22946 ts_tablet_manager.cc:1403] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.016s
I20260709 13:06:32.760630 22946 raft_consensus.cc:359] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.766088 22654 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:32.769130 22946 raft_consensus.cc:385] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.769498 22946 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.770151 22946 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.772748 22946 ts_tablet_manager.cc:1434] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:06:32.774245 22946 tablet_bootstrap.cc:492] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.780656 22946 tablet_bootstrap.cc:654] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.789229 22946 tablet_bootstrap.cc:492] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.789701 22946 ts_tablet_manager.cc:1403] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:06:32.792253 22946 raft_consensus.cc:359] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.792963 22946 raft_consensus.cc:385] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.793255 22946 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.794006 22946 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.796072 22946 ts_tablet_manager.cc:1434] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:32.797883 22946 tablet_bootstrap.cc:492] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Bootstrap starting.
I20260709 13:06:32.801710 22897 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:32.803042 23354 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:32.803809 22946 tablet_bootstrap.cc:654] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:32.816850 22753 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.821586 23354 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:32.822335 23098 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:32.824532 23354 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:32.831224 22946 tablet_bootstrap.cc:492] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: No bootstrap required, opened a new log
I20260709 13:06:32.839166 22946 ts_tablet_manager.cc:1403] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Time spent bootstrapping tablet: real 0.042s	user 0.024s	sys 0.009s
I20260709 13:06:32.841230 22754 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:32.842499 22946 raft_consensus.cc:359] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.843453 22946 raft_consensus.cc:385] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:32.843873 22946 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Initialized, Role: FOLLOWER
I20260709 13:06:32.843887 23337 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:32.844980 22946 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.847630 22946 ts_tablet_manager.cc:1434] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:06:32.873549 23229 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:32.888650 22749 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.892887 23229 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:32.909608 23246 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:32.915946 23236 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.916373 23236 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:32.921139 22748 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:32.935441 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:06:32.940764 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:06:32.941077 23236 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:32.942209 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:32.943742 23236 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:32.960398 22751 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.040058 23236 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.062549 23234 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.075032 23323 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.075311 23098 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.092765 23229 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:33.102172 23229 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.157544 23323 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.168582 22751 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:33.176553 23229 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:33.903934 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:33.927853 23407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:33.928872 23408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:33.930488 23410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:33.935555 21629 server_base.cc:1061] running on GCE node
I20260709 13:06:33.937703 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:33.938042 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:33.938328 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602393938307 us; error 0 us; skew 500 ppm
I20260709 13:06:33.939244 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:33.944890 21629 webserver.cc:533] Webserver started at http://127.21.31.68:46205/ using document root <none> and password file <none>
I20260709 13:06:33.945618 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:33.945847 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:33.946202 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:33.947973 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-3-root/instance:
uuid: "c45e1805f049467697a7358fb566de2f"
format_stamp: "Formatted at 2026-07-09 13:06:33 on dist-test-slave-9s6w"
I20260709 13:06:33.952967 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 13:06:33.957504 23415 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:33.958773 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:33.959225 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-3-root
uuid: "c45e1805f049467697a7358fb566de2f"
format_stamp: "Formatted at 2026-07-09 13:06:33 on dist-test-slave-9s6w"
I20260709 13:06:33.959599 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:33.984309 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:33.986035 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:33.988631 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:33.995766 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:33.996033 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:33.996323 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:33.996552 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:34.080276 23478 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.68:42017 every 8 connection(s)
I20260709 13:06:34.080981 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.68:42017
I20260709 13:06:34.101172 21629 auto_leader_rebalancer-test.cc:384] add a tserver: c45e1805f049467697a7358fb566de2f
I20260709 13:06:34.122291 23479 heartbeater.cc:344] Connected to a master server at 127.21.31.126:35001
I20260709 13:06:34.122648 23479 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:34.123257 23479 heartbeater.cc:507] Master 127.21.31.126:35001 requested a full tablet report, sending...
I20260709 13:06:34.125173 22651 ts_manager.cc:194] Registered new tserver with Master: c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:06:34.126516 22651 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33010
I20260709 13:06:34.498375 22902 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:34.506047 22751 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:34.507835 23246 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.508065 22824 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:34.516937 23229 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.518644 22751 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.520115 23246 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.564446 22651 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:34.567910 22897 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:34.572438 22824 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:06:34.575271 22868 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7701b9b4e69441f7ae9de9554134de69. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:34.594340 22751 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:34.594368 22824 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:34.597903 23229 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.615813 23246 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.621539 23246 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.624420 22751 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:06:34.642040 22868 consensus_peers.cc:597] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9cf0bfa7a80d4f0286947f1e6ff89772. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:34.644279 22824 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.661253 22651 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.664435 22751 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:34.673743 22824 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:34.674227 22897 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:34.675587 23234 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.676421 23326 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:34.684878 22719 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8f0563f7efc8452c86a8c67c0189fe81. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:34.686499 23234 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.689575 22897 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.690641 22824 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.704636 22653 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.726917 22824 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:34.736052 22897 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:34.738152 23460 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.737780 22748 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:34.740963 23242 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:34.750658 23242 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.751961 22897 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:06:34.758368 22794 consensus_peers.cc:597] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f53f146bb25444ce8d2072559efe5e59. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:34.766800 22654 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:34.767755 22748 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:34.922923 23513 ts_tablet_manager.cc:933] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:34.927071 23513 tablet_copy_client.cc:323] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:34.941745 22915 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 7701b9b4e69441f7ae9de9554134de69 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:34.942284 22915 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 7701b9b4e69441f7ae9de9554134de69 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-7701b9b4e69441f7ae9de9554134de69
I20260709 13:06:34.948482 22915 tablet_copy_source_session.cc:215] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:34.951799 23513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7701b9b4e69441f7ae9de9554134de69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:34.964643 23513 tablet_copy_client.cc:806] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:34.965282 23513 tablet_copy_client.cc:670] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:34.970081 23513 tablet_copy_client.cc:538] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:34.975639 23513 tablet_bootstrap.cc:492] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:34.999367 23513 tablet_bootstrap.cc:492] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:35.000511 23513 tablet_bootstrap.cc:492] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:35.001183 23513 ts_tablet_manager.cc:1403] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.005s
I20260709 13:06:35.003424 23513 raft_consensus.cc:359] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:35.003965 23513 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:35.004498 23513 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:35.007115 23479 heartbeater.cc:499] Master 127.21.31.126:35001 was elected leader, sending a full tablet report...
I20260709 13:06:35.007485 23513 ts_tablet_manager.cc:1434] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 13:06:35.009397 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-7701b9b4e69441f7ae9de9554134de69 received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:35.009848 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-7701b9b4e69441f7ae9de9554134de69 on tablet 7701b9b4e69441f7ae9de9554134de69 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:35.037935 23513 ts_tablet_manager.cc:933] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:35.040609 23513 tablet_copy_client.cc:323] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:35.042263 22915 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 9cf0bfa7a80d4f0286947f1e6ff89772 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:35.042801 22915 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 9cf0bfa7a80d4f0286947f1e6ff89772 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-9cf0bfa7a80d4f0286947f1e6ff89772
I20260709 13:06:35.048081 22915 tablet_copy_source_session.cc:215] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:35.050869 23513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cf0bfa7a80d4f0286947f1e6ff89772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:35.065492 23513 tablet_copy_client.cc:806] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:35.066026 23513 tablet_copy_client.cc:670] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:35.069262 23513 tablet_copy_client.cc:538] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:35.076313 23513 tablet_bootstrap.cc:492] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:35.090941 23513 tablet_bootstrap.cc:492] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:35.091766 23513 tablet_bootstrap.cc:492] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:35.092276 23513 ts_tablet_manager.cc:1403] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.008s
I20260709 13:06:35.094347 23513 raft_consensus.cc:359] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:35.095036 23513 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:35.095510 23513 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:35.097807 23513 ts_tablet_manager.cc:1434] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:06:35.099359 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-9cf0bfa7a80d4f0286947f1e6ff89772 received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:35.099735 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-9cf0bfa7a80d4f0286947f1e6ff89772 on tablet 9cf0bfa7a80d4f0286947f1e6ff89772 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:35.142566 23513 ts_tablet_manager.cc:933] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:35.144344 23513 tablet_copy_client.cc:323] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:35.157953 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet f53f146bb25444ce8d2072559efe5e59 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:35.158510 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet f53f146bb25444ce8d2072559efe5e59 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-f53f146bb25444ce8d2072559efe5e59
I20260709 13:06:35.166450 22841 tablet_copy_source_session.cc:215] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:35.169037 23513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53f146bb25444ce8d2072559efe5e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:35.179137 23513 tablet_copy_client.cc:806] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:35.179620 23513 tablet_copy_client.cc:670] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:35.186822 23513 tablet_copy_client.cc:538] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:35.191483 23513 tablet_bootstrap.cc:492] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:35.204881 23521 ts_tablet_manager.cc:933] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:35.218768 23521 tablet_copy_client.cc:323] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.65:37213
I20260709 13:06:35.224815 23513 tablet_bootstrap.cc:492] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:35.226207 23513 tablet_bootstrap.cc:492] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:35.227561 23513 ts_tablet_manager.cc:1403] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.036s	user 0.027s	sys 0.004s
I20260709 13:06:35.230213 23513 raft_consensus.cc:359] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:35.231359 23513 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:35.231951 23513 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:35.239892 23513 ts_tablet_manager.cc:1434] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:06:35.241716 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-f53f146bb25444ce8d2072559efe5e59 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:35.242152 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-f53f146bb25444ce8d2072559efe5e59 on tablet f53f146bb25444ce8d2072559efe5e59 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:35.248839 22766 tablet_copy_service.cc:140] P 6a5191aaff76469bbf6c4735dc9e6509: Received BeginTabletCopySession request for tablet 8f0563f7efc8452c86a8c67c0189fe81 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:56780)
I20260709 13:06:35.249398 22766 tablet_copy_service.cc:161] P 6a5191aaff76469bbf6c4735dc9e6509: Beginning new tablet copy session on tablet 8f0563f7efc8452c86a8c67c0189fe81 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:56780: session id = c45e1805f049467697a7358fb566de2f-8f0563f7efc8452c86a8c67c0189fe81
I20260709 13:06:35.253990 22766 tablet_copy_source_session.cc:215] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:35.256583 23521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0563f7efc8452c86a8c67c0189fe81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:35.267050 23521 tablet_copy_client.cc:806] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:35.267575 23521 tablet_copy_client.cc:670] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:35.270958 23521 tablet_copy_client.cc:538] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:35.277642 23521 tablet_bootstrap.cc:492] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:35.304970 23521 tablet_bootstrap.cc:492] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:35.305727 23521 tablet_bootstrap.cc:492] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:35.306262 23521 ts_tablet_manager.cc:1403] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.005s
I20260709 13:06:35.308470 23521 raft_consensus.cc:359] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:35.309055 23521 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:35.309564 23521 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:35.311393 23521 ts_tablet_manager.cc:1434] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:06:35.312952 22766 tablet_copy_service.cc:342] P 6a5191aaff76469bbf6c4735dc9e6509: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-8f0563f7efc8452c86a8c67c0189fe81 received from {username='slave'} at 127.0.0.1:56780
I20260709 13:06:35.313380 22766 tablet_copy_service.cc:434] P 6a5191aaff76469bbf6c4735dc9e6509: ending tablet copy session c45e1805f049467697a7358fb566de2f-8f0563f7efc8452c86a8c67c0189fe81 on tablet 8f0563f7efc8452c86a8c67c0189fe81 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:35.481958 23453 raft_consensus.cc:1217] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:35.548130 23453 raft_consensus.cc:1217] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:35.593681 23453 raft_consensus.cc:1217] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:35.618742 23453 raft_consensus.cc:1217] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:36.006536 23460 raft_consensus.cc:1064] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:36.009169 23460 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:36.017122 23453 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.018220 22748 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.017122 22897 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.020213 23495 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.021013 23460 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.022166 23242 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.037983 23460 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.039841 22748 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.043061 22897 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.054078 23452 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.064862 23526 raft_consensus.cc:1064] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:36.066846 23499 raft_consensus.cc:1064] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:36.067281 23526 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:36.072296 23499 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:36.072079 22651 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:36.082206 22823 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.082912 22748 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.084807 23246 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.085216 23452 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.087857 23491 raft_consensus.cc:1064] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:36.090754 23491 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:36.094044 23246 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.095283 23499 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:36.099092 23452 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.101096 22902 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.102988 23506 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:36.115621 22831 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.117959 23506 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.149407 23506 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:06:36.181190 23453 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.196302 23506 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.202901 23536 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.182180 23452 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.221309 22751 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.223132 23536 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:36.187125 23538 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.270426 22831 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:36.302018 22748 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:36.302223 23451 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:36.303731 23242 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:36.305418 23242 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:36.306504 23451 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.321923 23495 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:06:36.340698 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 131192us
I20260709 13:06:36.374320 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f53f146bb25444ce8d2072559efe5e59 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:36.370062 22751 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.376056 23450 raft_consensus.cc:2955] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.361665 22902 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.389286 22651 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.399204 22831 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:36.409925 23548 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.411484 22653 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:36.416265 22654 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:36.441241 22824 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.454147 22823 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.464609 23537 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:36.468634 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 76421.7us
I20260709 13:06:36.513285 22748 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:06:36.518399 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46415.8us
I20260709 13:06:36.529757 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet f53f146bb25444ce8d2072559efe5e59 with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
W20260709 13:06:36.540890 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60817.6us
I20260709 13:06:36.543322 23450 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.583353 22829 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.586542 23561 tablet_replica.cc:333] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:06:36.587635 23561 raft_consensus.cc:2243] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.588573 23561 raft_consensus.cc:2272] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.591219 22897 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:36.593914 22653 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.599047 23451 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:36.604867 23557 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:36.612159 23452 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:36.613960 23561 ts_tablet_manager.cc:1916] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:36.617501 23246 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.630986 23506 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:36.620131 23451 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:06:36.638829 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 166096us
I20260709 13:06:36.647717 22755 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.651472 22653 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.654718 22750 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.658957 23561 ts_tablet_manager.cc:1929] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:36.659893 23561 log.cc:1199] T f53f146bb25444ce8d2072559efe5e59 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/f53f146bb25444ce8d2072559efe5e59
I20260709 13:06:36.661971 22829 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:36.671478 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7701b9b4e69441f7ae9de9554134de69 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:36.672264 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet f53f146bb25444ce8d2072559efe5e59 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:36.678438 23563 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.681125 23452 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.690625 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet 7701b9b4e69441f7ae9de9554134de69 with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:36.691900 22747 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:36.699337 23562 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:36.700784 22653 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.702260 23565 tablet_replica.cc:333] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:36.703483 22641 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8f0563f7efc8452c86a8c67c0189fe81 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:36.710469 23526 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:36.712834 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f0563f7efc8452c86a8c67c0189fe81 with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:06:36.713739 23561 tablet_replica.cc:333] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:06:36.714664 23561 raft_consensus.cc:2243] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.715231 23561 raft_consensus.cc:2272] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.716660 23565 raft_consensus.cc:2243] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:36.718519 23561 ts_tablet_manager.cc:1916] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:36.719570 22755 raft_consensus.cc:2955] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.723874 23565 raft_consensus.cc:2272] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:36.731947 23565 ts_tablet_manager.cc:1916] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:36.733691 23561 ts_tablet_manager.cc:1929] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:36.734177 23561 log.cc:1199] T 8f0563f7efc8452c86a8c67c0189fe81 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/8f0563f7efc8452c86a8c67c0189fe81
I20260709 13:06:36.737615 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 8f0563f7efc8452c86a8c67c0189fe81 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:36.743927 22824 raft_consensus.cc:2955] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:36.757696 23565 ts_tablet_manager.cc:1929] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:36.758108 23565 log.cc:1199] T 7701b9b4e69441f7ae9de9554134de69 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/7701b9b4e69441f7ae9de9554134de69
I20260709 13:06:36.761530 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet 7701b9b4e69441f7ae9de9554134de69 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:37.037722 22897 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:37.042032 22824 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:37.042140 23452 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:37.043210 23560 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:37.044181 23554 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:37.050796 23562 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:37.052171 22824 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:37.057534 23452 raft_consensus.cc:2955] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:37.061456 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9cf0bfa7a80d4f0286947f1e6ff89772 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:37.066992 22653 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:37.087003 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 9cf0bfa7a80d4f0286947f1e6ff89772 with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:06:37.090878 23566 tablet_replica.cc:333] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:06:37.091544 23566 raft_consensus.cc:2243] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.092140 23566 raft_consensus.cc:2272] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.095350 23566 ts_tablet_manager.cc:1916] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:37.106966 23566 ts_tablet_manager.cc:1929] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:37.107250 23566 log.cc:1199] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/9cf0bfa7a80d4f0286947f1e6ff89772
I20260709 13:06:37.108654 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 9cf0bfa7a80d4f0286947f1e6ff89772 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:38.158852 22824 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:06:38.165107 22794 consensus_peers.cc:597] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11e8d51b5b18431cb7d6df2884a4e879. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:38.165395 22905 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:38.166574 23552 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.167474 22755 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:38.168519 23552 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.174811 23546 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.180975 22755 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.181823 22905 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.192696 22902 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.193972 22653 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.207119 22824 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:38.208608 22750 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:38.209599 22868 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6436de6ceb37488b9ee8b6d261356a40. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:38.210991 23560 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.214519 23567 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.227182 23562 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.234089 22755 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.246045 22829 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.255851 22824 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.266185 22651 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:06:38.273871 22794 consensus_peers.cc:597] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f765d75643f47dd939002936724a315. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:38.276510 22905 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:38.278220 22750 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:38.279410 23552 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.280479 23546 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.314958 23495 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.318362 22747 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.334952 22902 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.337703 22654 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.342494 22750 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:06:38.359454 22719 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f7d0e2e573344ec49ba2412e5de1d519. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:38.360244 22902 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:38.361169 22823 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:38.361465 23568 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.364332 23568 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:38.368853 23563 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.370890 22902 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.373955 22823 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.391515 22651 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:38.621558 23590 ts_tablet_manager.cc:933] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:38.624673 23590 tablet_copy_client.cc:323] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:38.626441 22915 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 6436de6ceb37488b9ee8b6d261356a40 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:38.626940 22915 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 6436de6ceb37488b9ee8b6d261356a40 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-6436de6ceb37488b9ee8b6d261356a40
I20260709 13:06:38.634047 22915 tablet_copy_source_session.cc:215] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:38.638586 23590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6436de6ceb37488b9ee8b6d261356a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:38.653817 23590 tablet_copy_client.cc:806] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:38.654351 23590 tablet_copy_client.cc:670] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:38.658870 23590 tablet_copy_client.cc:538] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:38.666963 23590 tablet_bootstrap.cc:492] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:38.671205 23592 ts_tablet_manager.cc:933] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:38.685170 23592 tablet_copy_client.cc:323] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:38.687134 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 11e8d51b5b18431cb7d6df2884a4e879 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:38.687644 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 11e8d51b5b18431cb7d6df2884a4e879 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-11e8d51b5b18431cb7d6df2884a4e879
I20260709 13:06:38.695048 23590 tablet_bootstrap.cc:492] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:38.696679 23590 tablet_bootstrap.cc:492] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:38.698247 22841 tablet_copy_source_session.cc:215] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:38.698365 23590 ts_tablet_manager.cc:1403] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.015s
I20260709 13:06:38.706730 23592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11e8d51b5b18431cb7d6df2884a4e879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:38.707886 23590 raft_consensus.cc:359] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.714043 23590 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:38.714788 23590 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.725733 23592 tablet_copy_client.cc:806] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:38.726476 23592 tablet_copy_client.cc:670] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:38.732971 23592 tablet_copy_client.cc:538] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:38.752174 23592 tablet_bootstrap.cc:492] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:38.755168 23595 ts_tablet_manager.cc:933] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:38.760999 23590 ts_tablet_manager.cc:1434] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.056s	user 0.014s	sys 0.029s
I20260709 13:06:38.762796 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-6436de6ceb37488b9ee8b6d261356a40 received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:38.763201 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-6436de6ceb37488b9ee8b6d261356a40 on tablet 6436de6ceb37488b9ee8b6d261356a40 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:38.772284 23595 tablet_copy_client.cc:323] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:38.774283 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 7f765d75643f47dd939002936724a315 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:38.774798 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 7f765d75643f47dd939002936724a315 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-7f765d75643f47dd939002936724a315
I20260709 13:06:38.781463 23592 tablet_bootstrap.cc:492] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:38.781836 22841 tablet_copy_source_session.cc:215] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:38.782320 23592 tablet_bootstrap.cc:492] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:38.783010 23592 ts_tablet_manager.cc:1403] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.003s
I20260709 13:06:38.785063 23595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f765d75643f47dd939002936724a315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:38.787006 23592 raft_consensus.cc:359] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.787863 23592 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:38.788525 23592 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.795701 23592 ts_tablet_manager.cc:1434] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:06:38.802373 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-11e8d51b5b18431cb7d6df2884a4e879 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:38.802847 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-11e8d51b5b18431cb7d6df2884a4e879 on tablet 11e8d51b5b18431cb7d6df2884a4e879 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:38.803406 23595 tablet_copy_client.cc:806] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:38.804337 23595 tablet_copy_client.cc:670] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:38.807308 23595 tablet_copy_client.cc:538] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:38.818229 23595 tablet_bootstrap.cc:492] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:38.843924 23595 tablet_bootstrap.cc:492] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:38.844795 23595 tablet_bootstrap.cc:492] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:38.845481 23595 ts_tablet_manager.cc:1403] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.011s
I20260709 13:06:38.850386 23595 raft_consensus.cc:359] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.852872 23595 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:38.853586 23595 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.856030 23595 ts_tablet_manager.cc:1434] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:06:38.859753 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-7f765d75643f47dd939002936724a315 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:38.860224 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-7f765d75643f47dd939002936724a315 on tablet 7f765d75643f47dd939002936724a315 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:38.896430 23595 ts_tablet_manager.cc:933] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:38.898144 23595 tablet_copy_client.cc:323] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.65:37213
I20260709 13:06:38.900161 22766 tablet_copy_service.cc:140] P 6a5191aaff76469bbf6c4735dc9e6509: Received BeginTabletCopySession request for tablet f7d0e2e573344ec49ba2412e5de1d519 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:56780)
I20260709 13:06:38.900666 22766 tablet_copy_service.cc:161] P 6a5191aaff76469bbf6c4735dc9e6509: Beginning new tablet copy session on tablet f7d0e2e573344ec49ba2412e5de1d519 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:56780: session id = c45e1805f049467697a7358fb566de2f-f7d0e2e573344ec49ba2412e5de1d519
I20260709 13:06:38.908401 22766 tablet_copy_source_session.cc:215] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:38.911409 23595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d0e2e573344ec49ba2412e5de1d519. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:38.940374 23595 tablet_copy_client.cc:806] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:38.940912 23595 tablet_copy_client.cc:670] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:38.950762 23595 tablet_copy_client.cc:538] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:38.962869 23595 tablet_bootstrap.cc:492] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:38.980914 23595 tablet_bootstrap.cc:492] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:38.981608 23595 tablet_bootstrap.cc:492] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:38.982146 23595 ts_tablet_manager.cc:1403] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.007s
I20260709 13:06:38.984673 23595 raft_consensus.cc:359] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.985374 23595 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:38.985919 23595 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:38.988197 23595 ts_tablet_manager.cc:1434] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:06:38.990595 22766 tablet_copy_service.cc:342] P 6a5191aaff76469bbf6c4735dc9e6509: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-f7d0e2e573344ec49ba2412e5de1d519 received from {username='slave'} at 127.0.0.1:56780
I20260709 13:06:38.991852 22766 tablet_copy_service.cc:434] P 6a5191aaff76469bbf6c4735dc9e6509: ending tablet copy session c45e1805f049467697a7358fb566de2f-f7d0e2e573344ec49ba2412e5de1d519 on tablet f7d0e2e573344ec49ba2412e5de1d519 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:39.071841 23452 raft_consensus.cc:1217] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:39.162940 23452 raft_consensus.cc:1217] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:39.215931 23451 raft_consensus.cc:1217] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:39.428304 23452 raft_consensus.cc:1217] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:39.622457 23552 raft_consensus.cc:1064] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:39.624598 23552 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:39.630366 22903 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.630671 23452 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.631291 22747 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.631924 23585 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:39.632752 23552 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:39.633529 23586 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:39.657585 22902 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.660885 23552 raft_consensus.cc:1064] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:39.663208 23552 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:39.655982 23585 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.675752 23452 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.688948 23567 raft_consensus.cc:1064] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:39.691303 23567 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:39.693717 22747 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.700035 22651 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.733934 22829 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.735502 23451 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.737167 23567 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:39.746462 23567 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:39.737651 22750 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.754132 22902 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.755009 23609 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:39.755641 23451 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.757790 23552 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:39.758488 22747 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.759634 23552 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:39.759790 23581 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:39.795837 23610 log.cc:927] Time spent T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:06:39.809636 23613 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.814869 22824 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.816031 23614 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.831508 22747 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.833065 23603 raft_consensus.cc:1064] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:39.834925 22905 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.838729 23452 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.838588 23451 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.842217 22755 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:39.835901 23603 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:39.908967 22824 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.912070 22905 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.918455 23451 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:39.917574 23602 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:39.962359 23563 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
W20260709 13:06:39.993947 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 56063us
I20260709 13:06:40.018520 22829 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:40.030337 22654 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.040874 23603 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:06:40.040391 22654 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.090247 23452 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:40.092782 23626 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
W20260709 13:06:40.110601 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63712.4us
I20260709 13:06:40.125806 23585 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.128543 23452 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.135823 22829 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:40.157020 23451 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:40.159448 23626 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:40.166520 22899 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:40.171568 22896 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:40.172940 23586 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.180562 23451 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.197940 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7f765d75643f47dd939002936724a315 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:40.200632 23614 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:40.210263 22653 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.213671 23552 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:06:40.223727 22652 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:40.233899 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 11e8d51b5b18431cb7d6df2884a4e879 with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:06:40.247682 23602 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.248992 22823 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.252944 22738 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f765d75643f47dd939002936724a315 with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:06:40.262439 22902 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.274283 23637 tablet_replica.cc:333] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:06:40.282750 23637 raft_consensus.cc:2243] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:40.283649 22897 raft_consensus.cc:2955] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.284570 22896 raft_consensus.cc:2955] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.294698 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 11e8d51b5b18431cb7d6df2884a4e879 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:40.280845 22653 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.280879 22902 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:40.300673 23451 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.302204 23637 raft_consensus.cc:2272] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:40.307626 22747 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:40.311586 23632 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:40.320945 23637 ts_tablet_manager.cc:1916] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:40.334158 23450 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:40.338656 23567 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.343413 22750 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:40.350468 22747 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.353569 23450 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:40.360121 23602 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:40.361570 22897 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:40.368221 23602 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:40.371847 23581 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:40.401674 23450 raft_consensus.cc:2955] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.406636 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6436de6ceb37488b9ee8b6d261356a40 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:40.406720 23451 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.405200 23602 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.419152 23637 ts_tablet_manager.cc:1929] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:40.417887 22654 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:40.423142 23637 log.cc:1199] T 11e8d51b5b18431cb7d6df2884a4e879 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/11e8d51b5b18431cb7d6df2884a4e879
I20260709 13:06:40.426093 22897 raft_consensus.cc:2955] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.427975 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 11e8d51b5b18431cb7d6df2884a4e879 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:40.430796 22641 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f7d0e2e573344ec49ba2412e5de1d519 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:40.428782 23637 tablet_replica.cc:333] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:06:40.436458 23637 raft_consensus.cc:2243] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:40.450103 23637 raft_consensus.cc:2272] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:40.482807 23637 ts_tablet_manager.cc:1916] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:40.607613 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet 6436de6ceb37488b9ee8b6d261356a40 with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:40.612968 23638 tablet_replica.cc:333] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:40.614279 23638 raft_consensus.cc:2243] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:40.615072 23638 raft_consensus.cc:2272] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:40.619325 23638 ts_tablet_manager.cc:1916] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:40.636142 23638 ts_tablet_manager.cc:1929] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:40.636600 23638 log.cc:1199] T 6436de6ceb37488b9ee8b6d261356a40 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/6436de6ceb37488b9ee8b6d261356a40
I20260709 13:06:40.662305 23637 ts_tablet_manager.cc:1929] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:40.662858 23637 log.cc:1199] T 7f765d75643f47dd939002936724a315 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/7f765d75643f47dd939002936724a315
I20260709 13:06:40.656003 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet 6436de6ceb37488b9ee8b6d261356a40 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:40.665777 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 7f765d75643f47dd939002936724a315 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:40.679497 22652 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:40.691553 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet f7d0e2e573344ec49ba2412e5de1d519 with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:40.694346 23638 tablet_replica.cc:333] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:40.695392 23638 raft_consensus.cc:2243] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:40.696156 23638 raft_consensus.cc:2272] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:40.700641 23638 ts_tablet_manager.cc:1916] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:40.713258 23638 ts_tablet_manager.cc:1929] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:40.713672 23638 log.cc:1199] T f7d0e2e573344ec49ba2412e5de1d519 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/f7d0e2e573344ec49ba2412e5de1d519
I20260709 13:06:40.715086 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet f7d0e2e573344ec49ba2412e5de1d519 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
W20260709 13:06:41.505995 22685 debug-util.cc:398] Leaking SignalData structure 0x7b08001376e0 after lost signal to thread 21632
W20260709 13:06:41.506765 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080008ba60 after lost signal to thread 22688
W20260709 13:06:41.507663 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800303440 after lost signal to thread 22780
W20260709 13:06:41.508590 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800312a20 after lost signal to thread 22855
W20260709 13:06:41.509528 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080032c420 after lost signal to thread 22929
W20260709 13:06:41.510456 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800325160 after lost signal to thread 23478
I20260709 13:06:41.780417 22755 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:41.786425 22896 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:41.786911 22719 consensus_peers.cc:597] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ad75d8751c54bf78bcf4fa8cc6578c1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:41.787231 22823 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:41.788407 23602 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:41.789353 23627 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:41.822111 23602 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:41.824417 22903 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:41.829154 22823 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:41.852813 22823 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:41.859665 22651 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:06:41.866554 22794 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4bbeaee00aa942ccb232c812c9e14827. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:41.867673 22903 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:41.868499 22755 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:41.870709 23641 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:41.889788 23641 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:41.931964 23641 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:41.945007 22905 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:41.968565 22755 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:42.014178 22905 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:06:42.025699 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 58604.7us
W20260709 13:06:42.033030 22868 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 42c5a324a80f42d4965b69d86915a65d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.034094 22755 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:42.036406 23652 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:42.038657 22823 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:42.041601 23629 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:42.050460 23652 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:42.054306 22747 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:42.060962 22651 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:42.110723 22651 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:42.122316 22823 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:42.133395 22823 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:42.164570 22905 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:42.168023 22750 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:42.168778 22794 consensus_peers.cc:597] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 18051671d25f4aa7b30d4a299909f91a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:42.170325 23649 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:42.171614 23655 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:42.214928 23655 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:42.217868 22896 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:42.218390 22750 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:42.242760 22652 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:42.300844 23665 ts_tablet_manager.cc:933] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:42.314592 23665 tablet_copy_client.cc:323] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.65:37213
I20260709 13:06:42.317088 22766 tablet_copy_service.cc:140] P 6a5191aaff76469bbf6c4735dc9e6509: Received BeginTabletCopySession request for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:56780)
I20260709 13:06:42.317741 22766 tablet_copy_service.cc:161] P 6a5191aaff76469bbf6c4735dc9e6509: Beginning new tablet copy session on tablet 4ad75d8751c54bf78bcf4fa8cc6578c1 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:56780: session id = c45e1805f049467697a7358fb566de2f-4ad75d8751c54bf78bcf4fa8cc6578c1
I20260709 13:06:42.327260 22766 tablet_copy_source_session.cc:215] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:42.330994 23665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.347101 23665 tablet_copy_client.cc:806] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:42.347735 23665 tablet_copy_client.cc:670] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:42.351936 23665 tablet_copy_client.cc:538] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:42.361892 23665 tablet_bootstrap.cc:492] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:42.391163 23665 tablet_bootstrap.cc:492] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:42.392169 23665 tablet_bootstrap.cc:492] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:42.392874 23665 ts_tablet_manager.cc:1403] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.014s
I20260709 13:06:42.396412 23665 raft_consensus.cc:359] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:42.397535 23665 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:42.400651 23665 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:42.429991 23669 ts_tablet_manager.cc:933] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:42.442158 23669 tablet_copy_client.cc:323] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:42.444404 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 4bbeaee00aa942ccb232c812c9e14827 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:42.444940 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 4bbeaee00aa942ccb232c812c9e14827 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-4bbeaee00aa942ccb232c812c9e14827
I20260709 13:06:42.447283 23665 ts_tablet_manager.cc:1434] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.054s	user 0.016s	sys 0.007s
I20260709 13:06:42.452359 22766 tablet_copy_service.cc:342] P 6a5191aaff76469bbf6c4735dc9e6509: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-4ad75d8751c54bf78bcf4fa8cc6578c1 received from {username='slave'} at 127.0.0.1:56780
I20260709 13:06:42.452833 22766 tablet_copy_service.cc:434] P 6a5191aaff76469bbf6c4735dc9e6509: ending tablet copy session c45e1805f049467697a7358fb566de2f-4ad75d8751c54bf78bcf4fa8cc6578c1 on tablet 4ad75d8751c54bf78bcf4fa8cc6578c1 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:42.464545 22841 tablet_copy_source_session.cc:215] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:42.469344 23669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bbeaee00aa942ccb232c812c9e14827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.487005 23669 tablet_copy_client.cc:806] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:42.487607 23669 tablet_copy_client.cc:670] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:42.492708 23669 tablet_copy_client.cc:538] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:42.514771 23669 tablet_bootstrap.cc:492] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:42.533252 23669 tablet_bootstrap.cc:492] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:42.534206 23669 tablet_bootstrap.cc:492] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:42.534850 23669 ts_tablet_manager.cc:1403] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.005s
I20260709 13:06:42.537747 23669 raft_consensus.cc:359] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:42.538597 23669 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:42.539287 23669 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:42.541690 23669 ts_tablet_manager.cc:1434] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:06:42.543345 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-4bbeaee00aa942ccb232c812c9e14827 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:42.543743 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-4bbeaee00aa942ccb232c812c9e14827 on tablet 4bbeaee00aa942ccb232c812c9e14827 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:42.575006 23669 ts_tablet_manager.cc:933] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:42.579473 23669 tablet_copy_client.cc:323] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:42.581177 22915 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 42c5a324a80f42d4965b69d86915a65d from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:42.581741 22915 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 42c5a324a80f42d4965b69d86915a65d from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-42c5a324a80f42d4965b69d86915a65d
I20260709 13:06:42.589926 22915 tablet_copy_source_session.cc:215] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:42.593204 23669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42c5a324a80f42d4965b69d86915a65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.605091 23669 tablet_copy_client.cc:806] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:42.605624 23669 tablet_copy_client.cc:670] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:42.610389 23669 tablet_copy_client.cc:538] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:42.618114 23669 tablet_bootstrap.cc:492] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:42.646827 23669 tablet_bootstrap.cc:492] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:42.647756 23669 tablet_bootstrap.cc:492] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:42.648406 23669 ts_tablet_manager.cc:1403] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.009s
I20260709 13:06:42.652813 23669 raft_consensus.cc:359] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:42.653996 23669 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:42.654663 23669 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:42.664228 23669 ts_tablet_manager.cc:1434] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:06:42.671293 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-42c5a324a80f42d4965b69d86915a65d received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:42.671774 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-42c5a324a80f42d4965b69d86915a65d on tablet 42c5a324a80f42d4965b69d86915a65d with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:42.762038 23450 raft_consensus.cc:1217] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:42.768904 23669 ts_tablet_manager.cc:933] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:42.775342 23669 tablet_copy_client.cc:323] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:42.776961 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 18051671d25f4aa7b30d4a299909f91a from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:42.777592 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 18051671d25f4aa7b30d4a299909f91a from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-18051671d25f4aa7b30d4a299909f91a
I20260709 13:06:42.787947 22841 tablet_copy_source_session.cc:215] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:42.791131 23669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18051671d25f4aa7b30d4a299909f91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:42.809111 23450 raft_consensus.cc:1217] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:42.820731 23669 tablet_copy_client.cc:806] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:42.821238 23669 tablet_copy_client.cc:670] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:42.838075 23669 tablet_copy_client.cc:538] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:42.850837 23669 tablet_bootstrap.cc:492] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:42.870334 23669 tablet_bootstrap.cc:492] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:42.871465 23669 tablet_bootstrap.cc:492] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:42.872136 23669 ts_tablet_manager.cc:1403] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 13:06:42.874622 23669 raft_consensus.cc:359] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:42.875689 23669 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:42.876297 23669 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:42.880699 23669 ts_tablet_manager.cc:1434] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:06:42.882541 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-18051671d25f4aa7b30d4a299909f91a received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:42.882920 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-18051671d25f4aa7b30d4a299909f91a on tablet 18051671d25f4aa7b30d4a299909f91a with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:42.970773 23450 raft_consensus.cc:1217] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:43.141876 23643 raft_consensus.cc:1064] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:43.144308 23643 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:43.150759 23450 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.152936 23674 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:43.155823 22896 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.157740 23627 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:43.183902 22829 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.186936 23674 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:06:43.199517 23678 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.205396 23450 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.215837 23655 raft_consensus.cc:1064] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:43.218232 23655 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:43.227363 22896 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.231199 22823 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.234511 22651 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:43.267095 23450 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.268458 23654 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:43.302868 22896 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.305960 22750 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.308233 23654 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:43.313010 23636 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:43.333456 23654 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.339432 23451 raft_consensus.cc:1217] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:43.344244 22750 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.361723 23451 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.368772 22653 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.375510 22896 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.418722 23629 raft_consensus.cc:1064] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:43.421205 23629 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:43.424469 22829 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:43.430521 23451 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:43.432020 23691 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:43.440029 23686 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.446264 22750 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:43.449223 22756 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:43.448233 23450 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.466995 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4bbeaee00aa942ccb232c812c9e14827 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:43.471704 22653 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.475437 23690 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:43.486908 22755 raft_consensus.cc:2955] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.489260 23450 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.491664 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 4bbeaee00aa942ccb232c812c9e14827 with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:06:43.490670 23629 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:06:43.506944 22829 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.510234 23451 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:43.513393 22750 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.515897 23694 tablet_replica.cc:333] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:06:43.516988 23694 raft_consensus.cc:2243] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.519706 23694 raft_consensus.cc:2272] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.521842 22823 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:43.522114 23652 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:43.523662 23693 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:43.526188 23694 ts_tablet_manager.cc:1916] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:43.527185 23674 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:43.534122 23674 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:43.542658 23694 ts_tablet_manager.cc:1929] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:43.543109 23694 log.cc:1199] T 4bbeaee00aa942ccb232c812c9e14827 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/4bbeaee00aa942ccb232c812c9e14827
I20260709 13:06:43.544931 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 4bbeaee00aa942ccb232c812c9e14827 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:43.551227 23629 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.557284 23627 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.566450 22829 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.575824 22747 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.587172 23450 raft_consensus.cc:2955] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.603251 23451 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.625603 22653 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.627633 22824 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.663537 22641 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:43.675801 22651 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:43.677402 22905 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:43.686595 22747 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:43.687940 23629 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:43.695130 23629 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.698594 22750 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:06:43.705078 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50909.6us
I20260709 13:06:43.706998 23450 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:43.714100 23699 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:43.718637 23450 raft_consensus.cc:2955] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.722359 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ad75d8751c54bf78bcf4fa8cc6578c1 with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:06:43.723279 23694 tablet_replica.cc:333] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:06:43.724202 23694 raft_consensus.cc:2243] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.724911 23694 raft_consensus.cc:2272] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.730118 23694 ts_tablet_manager.cc:1916] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:43.737563 23690 raft_consensus.cc:1064] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:43.741559 23690 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:43.754242 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 42c5a324a80f42d4965b69d86915a65d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:43.760886 22653 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:43.769030 22903 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.770721 22750 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.771637 23450 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:43.770992 23691 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:43.774535 23654 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:43.776307 23654 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:43.786936 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet 42c5a324a80f42d4965b69d86915a65d with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:43.802830 23706 tablet_replica.cc:333] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:43.809221 23694 ts_tablet_manager.cc:1929] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:43.810309 23694 log.cc:1199] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/4ad75d8751c54bf78bcf4fa8cc6578c1
I20260709 13:06:43.811942 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 4ad75d8751c54bf78bcf4fa8cc6578c1 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:43.814083 23706 raft_consensus.cc:2243] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.815488 23706 raft_consensus.cc:2272] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.815860 23690 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.826514 23706 ts_tablet_manager.cc:1916] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:43.837764 23450 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.845757 23706 ts_tablet_manager.cc:1929] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:43.846242 23706 log.cc:1199] T 42c5a324a80f42d4965b69d86915a65d P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/42c5a324a80f42d4965b69d86915a65d
I20260709 13:06:43.847188 22903 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.848817 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet 42c5a324a80f42d4965b69d86915a65d (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:43.862537 22750 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.865747 22653 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.891513 22824 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:43.899475 23450 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:43.903861 23690 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:43.913153 23655 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.914203 22903 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:43.915781 23691 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:43.918484 23450 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.919546 22903 raft_consensus.cc:2955] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.941674 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 18051671d25f4aa7b30d4a299909f91a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:43.940344 22654 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:43.960119 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 18051671d25f4aa7b30d4a299909f91a with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:06:43.975374 23707 tablet_replica.cc:333] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:06:43.976517 23707 raft_consensus.cc:2243] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.977349 23707 raft_consensus.cc:2272] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.982786 23707 ts_tablet_manager.cc:1916] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:43.996639 23707 ts_tablet_manager.cc:1929] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:43.997036 23707 log.cc:1199] T 18051671d25f4aa7b30d4a299909f91a P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/18051671d25f4aa7b30d4a299909f91a
I20260709 13:06:43.998941 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 18051671d25f4aa7b30d4a299909f91a (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:45.059617 22905 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:45.066195 22755 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:45.066288 22829 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:45.067831 23700 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.068662 23709 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:45.080195 22868 consensus_peers.cc:597] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e8ff32d0405c426786004110e1c1f955. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:45.099642 23698 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.100847 22755 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.104272 22829 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.117015 22651 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.118464 22824 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:45.187885 22905 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:45.197199 22794 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0d74135bf874a0bb4b857b686effcfd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:45.204753 23714 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.236837 23690 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.257637 22903 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:45.276360 22905 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:06:45.295358 22868 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a42e3464af3e4e5eb580790b400a6eff. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:45.295912 22830 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:45.298513 23709 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.306828 22755 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:45.316243 23700 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.344656 22755 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:45.347469 23690 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:45.356417 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45952.8us
I20260709 13:06:45.417446 22651 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.421521 22755 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.428582 23698 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.470498 22747 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.481873 22830 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.508206 22654 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.509774 23737 ts_tablet_manager.cc:933] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:45.518949 22830 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:45.549189 23737 tablet_copy_client.cc:323] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:45.551121 22915 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet e8ff32d0405c426786004110e1c1f955 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:45.551695 22915 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet e8ff32d0405c426786004110e1c1f955 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-e8ff32d0405c426786004110e1c1f955
I20260709 13:06:45.555734 22905 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:45.556180 22794 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2147efadd4ea46a09bda395402a10299. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:45.557214 23729 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.558241 22755 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:45.559880 23736 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:45.564785 22915 tablet_copy_source_session.cc:215] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:45.568607 23737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ff32d0405c426786004110e1c1f955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.576507 23729 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.584967 22755 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.584808 22905 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.589622 23737 tablet_copy_client.cc:806] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:45.590190 23737 tablet_copy_client.cc:670] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:45.595666 23737 tablet_copy_client.cc:538] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:45.611490 22650 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:45.612408 23737 tablet_bootstrap.cc:492] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:45.655261 23737 tablet_bootstrap.cc:492] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:45.656082 23737 tablet_bootstrap.cc:492] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:45.656790 23737 ts_tablet_manager.cc:1403] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.045s	user 0.026s	sys 0.002s
I20260709 13:06:45.660723 23737 raft_consensus.cc:359] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:45.661453 23737 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:45.662388 23737 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:45.665285 23737 ts_tablet_manager.cc:1434] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:45.670993 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-e8ff32d0405c426786004110e1c1f955 received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:45.671382 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-e8ff32d0405c426786004110e1c1f955 on tablet e8ff32d0405c426786004110e1c1f955 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:45.749563 23737 ts_tablet_manager.cc:933] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:45.751209 23737 tablet_copy_client.cc:323] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:45.752694 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet d0d74135bf874a0bb4b857b686effcfd from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:45.753185 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet d0d74135bf874a0bb4b857b686effcfd from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-d0d74135bf874a0bb4b857b686effcfd
I20260709 13:06:45.768922 22841 tablet_copy_source_session.cc:215] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:45.773079 23737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d74135bf874a0bb4b857b686effcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.796730 23737 tablet_copy_client.cc:806] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:45.797664 23737 tablet_copy_client.cc:670] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:45.801610 23737 tablet_copy_client.cc:538] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:45.811250 23737 tablet_bootstrap.cc:492] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:45.841953 23737 tablet_bootstrap.cc:492] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:45.842808 23737 tablet_bootstrap.cc:492] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:45.843439 23737 ts_tablet_manager.cc:1403] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.007s
I20260709 13:06:45.846119 23737 raft_consensus.cc:359] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:45.846949 23737 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:45.847592 23737 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:45.849849 23737 ts_tablet_manager.cc:1434] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:06:45.851902 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-d0d74135bf874a0bb4b857b686effcfd received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:45.852283 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-d0d74135bf874a0bb4b857b686effcfd on tablet d0d74135bf874a0bb4b857b686effcfd with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:45.893507 23737 ts_tablet_manager.cc:933] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:45.895387 23737 tablet_copy_client.cc:323] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:45.897143 22915 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet a42e3464af3e4e5eb580790b400a6eff from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:45.897660 22915 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet a42e3464af3e4e5eb580790b400a6eff from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-a42e3464af3e4e5eb580790b400a6eff
I20260709 13:06:45.903345 22915 tablet_copy_source_session.cc:215] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:45.906597 23737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a42e3464af3e4e5eb580790b400a6eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:45.927667 23737 tablet_copy_client.cc:806] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:45.929183 23737 tablet_copy_client.cc:670] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:45.946853 23737 tablet_copy_client.cc:538] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:45.960443 23737 tablet_bootstrap.cc:492] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:45.993271 23737 tablet_bootstrap.cc:492] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:45.994164 23737 tablet_bootstrap.cc:492] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:45.994800 23737 ts_tablet_manager.cc:1403] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.013s
I20260709 13:06:45.997411 23737 raft_consensus.cc:359] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:45.998291 23737 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:45.998935 23737 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:46.001461 23737 ts_tablet_manager.cc:1434] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:06:46.002931 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-a42e3464af3e4e5eb580790b400a6eff received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:46.003312 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-a42e3464af3e4e5eb580790b400a6eff on tablet a42e3464af3e4e5eb580790b400a6eff with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:46.086957 23451 raft_consensus.cc:1217] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:46.128752 23737 ts_tablet_manager.cc:933] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:46.135247 23737 tablet_copy_client.cc:323] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:46.136713 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 2147efadd4ea46a09bda395402a10299 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:46.137207 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 2147efadd4ea46a09bda395402a10299 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-2147efadd4ea46a09bda395402a10299
I20260709 13:06:46.143450 22841 tablet_copy_source_session.cc:215] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:46.150974 23737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147efadd4ea46a09bda395402a10299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.167914 23737 tablet_copy_client.cc:806] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:46.168478 23737 tablet_copy_client.cc:670] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:46.179548 23737 tablet_copy_client.cc:538] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:46.187981 23737 tablet_bootstrap.cc:492] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:46.222571 23737 tablet_bootstrap.cc:492] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:46.223217 23737 tablet_bootstrap.cc:492] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:46.223709 23737 ts_tablet_manager.cc:1403] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.005s
I20260709 13:06:46.225684 23737 raft_consensus.cc:359] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:46.226480 23737 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:46.227070 23737 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:46.229382 23737 ts_tablet_manager.cc:1434] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:06:46.230916 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-2147efadd4ea46a09bda395402a10299 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:46.231302 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-2147efadd4ea46a09bda395402a10299 on tablet 2147efadd4ea46a09bda395402a10299 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:46.253677 23451 raft_consensus.cc:1217] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:46.523072 23451 raft_consensus.cc:1217] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:46.611932 23735 raft_consensus.cc:1064] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:46.615036 23735 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:46.622994 23451 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:46.625159 22830 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:46.625838 23698 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:46.628831 22755 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:46.631014 23700 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:46.633414 23700 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:46.659083 23735 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.671878 23450 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.683635 23714 raft_consensus.cc:1064] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:46.684248 22755 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.709758 22654 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.731659 23714 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:46.750923 22905 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:46.754887 23451 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:46.762719 23714 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:46.771703 22755 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:46.772244 23655 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:46.785360 23655 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:46.791563 23452 raft_consensus.cc:1217] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:46.824566 23690 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.841531 23450 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.860370 22830 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.872402 22651 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.894416 22903 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.897984 22755 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.906322 22905 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:46.912164 23450 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:46.913431 23762 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:46.918191 22830 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:46.919574 23698 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:46.921346 23727 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.924513 23450 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.935400 22824 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:46.943171 22830 raft_consensus.cc:2955] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.944315 22755 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:46.946086 23451 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:46.947494 23755 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:46.948541 23714 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:46.958773 23756 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.963745 22750 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.968086 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d0d74135bf874a0bb4b857b686effcfd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:46.968051 22650 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.970996 23451 raft_consensus.cc:2955] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.977908 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e8ff32d0405c426786004110e1c1f955 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:46.981492 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet e8ff32d0405c426786004110e1c1f955 with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:06:46.990423 23766 tablet_replica.cc:333] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:06:46.990707 22651 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:46.991444 23766 raft_consensus.cc:2243] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:46.992264 23766 raft_consensus.cc:2272] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:46.995726 23766 ts_tablet_manager.cc:1916] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:47.011305 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet d0d74135bf874a0bb4b857b686effcfd with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:06:47.019439 23767 tablet_replica.cc:333] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:06:47.020701 23767 raft_consensus.cc:2243] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:47.021445 23767 raft_consensus.cc:2272] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:47.023886 23766 ts_tablet_manager.cc:1929] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:47.024360 23766 log.cc:1199] T e8ff32d0405c426786004110e1c1f955 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/e8ff32d0405c426786004110e1c1f955
I20260709 13:06:47.025986 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet e8ff32d0405c426786004110e1c1f955 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:47.026516 23767 ts_tablet_manager.cc:1916] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:47.037263 23767 ts_tablet_manager.cc:1929] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:47.037636 23767 log.cc:1199] T d0d74135bf874a0bb4b857b686effcfd P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/d0d74135bf874a0bb4b857b686effcfd
I20260709 13:06:47.038975 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet d0d74135bf874a0bb4b857b686effcfd (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:47.119617 23727 raft_consensus.cc:1064] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:47.122555 23727 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:47.130076 23450 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:47.130368 22830 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:47.130474 22750 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:47.131572 23735 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:47.132825 23698 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:47.133608 23762 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:47.183077 23735 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.200492 22823 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.208851 23450 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.218019 22651 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.219784 22749 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.238274 22905 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:47.244843 23450 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:47.245961 23735 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:47.246491 22749 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:47.251089 23762 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:47.253978 23727 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.255851 23450 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.260787 22749 raft_consensus.cc:2955] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.263274 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a42e3464af3e4e5eb580790b400a6eff with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:47.274592 22650 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:47.289263 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet a42e3464af3e4e5eb580790b400a6eff with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:47.293771 23772 tablet_replica.cc:333] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:47.294816 23772 raft_consensus.cc:2243] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:47.295523 23772 raft_consensus.cc:2272] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:47.298723 23772 ts_tablet_manager.cc:1916] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:47.310653 23772 ts_tablet_manager.cc:1929] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:47.311255 23772 log.cc:1199] T a42e3464af3e4e5eb580790b400a6eff P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/a42e3464af3e4e5eb580790b400a6eff
I20260709 13:06:47.313287 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet a42e3464af3e4e5eb580790b400a6eff (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:47.333527 23756 raft_consensus.cc:1064] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:47.336411 23756 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:47.344116 22905 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:47.344259 22749 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:47.344467 23450 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:47.347046 23690 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:47.347952 23755 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:47.351301 23655 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:47.387053 23755 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.403229 22905 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.406033 23450 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.412515 22650 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:47.421241 22749 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.444041 22823 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:47.448931 23450 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:47.449290 22753 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:47.450249 23655 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:47.451001 23756 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:47.459885 22753 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.457276 23655 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.463882 23451 raft_consensus.cc:2955] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.485162 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2147efadd4ea46a09bda395402a10299 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:47.487783 22651 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:47.500619 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 2147efadd4ea46a09bda395402a10299 with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:06:47.501544 23767 tablet_replica.cc:333] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:06:47.502466 23767 raft_consensus.cc:2243] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:47.503052 23767 raft_consensus.cc:2272] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:47.506136 23767 ts_tablet_manager.cc:1916] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:47.519526 23767 ts_tablet_manager.cc:1929] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:47.519946 23767 log.cc:1199] T 2147efadd4ea46a09bda395402a10299 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/2147efadd4ea46a09bda395402a10299
I20260709 13:06:47.521554 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 2147efadd4ea46a09bda395402a10299 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:48.608661 22903 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:48.614400 22823 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:48.615108 22753 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:48.615345 22868 consensus_peers.cc:597] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 317f6a16e0af41e2984a9b1626ab0edc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:48.616228 23735 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:48.616974 23762 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:48.626138 23727 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.628440 22823 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.632607 22749 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.646023 22654 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.657346 22905 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:48.668278 22823 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:48.668654 22753 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:48.668855 22868 consensus_peers.cc:597] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b9578eb52c342078c7555d5100bdcad. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:48.670070 23735 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:48.671271 23762 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:48.681509 23782 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.688297 22749 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.692160 22823 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.696982 22905 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:48.704106 22650 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:48.734479 22868 consensus_peers.cc:597] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90274c6677ab4d37b51190e940c9608e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:48.737088 22823 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:48.738268 23735 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:48.738332 22749 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:48.739653 23762 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:48.745786 23735 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.747014 22823 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.749724 22749 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.764165 22654 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.764349 22905 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:06:48.772312 22868 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53ca1af745cb4fd48a40af06ae1c43ca. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:48.772818 22749 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:48.774202 23762 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:48.774662 22823 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:48.776016 23782 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:48.787477 23762 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.789665 22749 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.793669 22823 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:48.809543 22654 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.070669 23795 ts_tablet_manager.cc:933] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:49.073108 23795 tablet_copy_client.cc:323] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:49.074625 22915 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 317f6a16e0af41e2984a9b1626ab0edc from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:49.075042 22915 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 317f6a16e0af41e2984a9b1626ab0edc from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-317f6a16e0af41e2984a9b1626ab0edc
I20260709 13:06:49.081620 22915 tablet_copy_source_session.cc:215] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:49.084880 23795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317f6a16e0af41e2984a9b1626ab0edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.100834 23795 tablet_copy_client.cc:806] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:49.101572 23795 tablet_copy_client.cc:670] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:49.105203 23795 tablet_copy_client.cc:538] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:49.115773 23795 tablet_bootstrap.cc:492] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:49.139142 23795 tablet_bootstrap.cc:492] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:49.140933 23795 tablet_bootstrap.cc:492] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:49.141707 23795 ts_tablet_manager.cc:1403] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.011s
I20260709 13:06:49.144553 23795 raft_consensus.cc:359] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:49.145411 23795 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:49.146071 23795 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:49.153211 23795 ts_tablet_manager.cc:1434] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:06:49.155186 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-317f6a16e0af41e2984a9b1626ab0edc received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:49.155571 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-317f6a16e0af41e2984a9b1626ab0edc on tablet 317f6a16e0af41e2984a9b1626ab0edc with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:49.175757 23795 ts_tablet_manager.cc:933] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:49.177903 23795 tablet_copy_client.cc:323] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:49.179563 22915 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 90274c6677ab4d37b51190e940c9608e from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:49.180052 22915 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 90274c6677ab4d37b51190e940c9608e from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-90274c6677ab4d37b51190e940c9608e
I20260709 13:06:49.187346 22915 tablet_copy_source_session.cc:215] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:49.191042 23795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90274c6677ab4d37b51190e940c9608e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.207340 23795 tablet_copy_client.cc:806] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:49.207981 23795 tablet_copy_client.cc:670] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:49.211848 23795 tablet_copy_client.cc:538] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:49.223665 23795 tablet_bootstrap.cc:492] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:49.244356 23795 tablet_bootstrap.cc:492] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:49.245034 23795 tablet_bootstrap.cc:492] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:49.245738 23795 ts_tablet_manager.cc:1403] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.022s	user 0.021s	sys 0.000s
I20260709 13:06:49.248133 23795 raft_consensus.cc:359] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:49.248977 23795 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:49.249658 23795 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:49.254328 23795 ts_tablet_manager.cc:1434] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:06:49.256500 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-90274c6677ab4d37b51190e940c9608e received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:49.256837 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-90274c6677ab4d37b51190e940c9608e on tablet 90274c6677ab4d37b51190e940c9608e with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:49.262342 23795 ts_tablet_manager.cc:933] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:49.264470 23795 tablet_copy_client.cc:323] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:49.266198 22915 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 4b9578eb52c342078c7555d5100bdcad from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:49.266698 22915 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 4b9578eb52c342078c7555d5100bdcad from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-4b9578eb52c342078c7555d5100bdcad
I20260709 13:06:49.273147 22915 tablet_copy_source_session.cc:215] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:49.275532 23795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9578eb52c342078c7555d5100bdcad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.287318 23795 tablet_copy_client.cc:806] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:49.287840 23795 tablet_copy_client.cc:670] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:49.292352 23795 tablet_copy_client.cc:538] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:49.303227 23795 tablet_bootstrap.cc:492] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:49.323340 23795 tablet_bootstrap.cc:492] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:49.323930 23795 tablet_bootstrap.cc:492] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:49.324451 23795 ts_tablet_manager.cc:1403] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 13:06:49.326898 23803 ts_tablet_manager.cc:933] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:49.327174 23795 raft_consensus.cc:359] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:49.327989 23795 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:49.328598 23795 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:49.333035 23795 ts_tablet_manager.cc:1434] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:06:49.335791 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-4b9578eb52c342078c7555d5100bdcad received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:49.337090 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-4b9578eb52c342078c7555d5100bdcad on tablet 4b9578eb52c342078c7555d5100bdcad with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:49.345722 23803 tablet_copy_client.cc:323] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:49.347484 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 53ca1af745cb4fd48a40af06ae1c43ca from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:49.348189 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 53ca1af745cb4fd48a40af06ae1c43ca from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-53ca1af745cb4fd48a40af06ae1c43ca
I20260709 13:06:49.354234 22914 tablet_copy_source_session.cc:215] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:49.357185 23803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53ca1af745cb4fd48a40af06ae1c43ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.371956 23803 tablet_copy_client.cc:806] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:49.372535 23803 tablet_copy_client.cc:670] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:49.375772 23803 tablet_copy_client.cc:538] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:49.383677 23803 tablet_bootstrap.cc:492] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:49.405809 23803 tablet_bootstrap.cc:492] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:49.406629 23803 tablet_bootstrap.cc:492] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:49.407230 23803 ts_tablet_manager.cc:1403] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.013s
I20260709 13:06:49.410406 23803 raft_consensus.cc:359] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:49.411201 23803 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:49.411841 23803 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:49.414399 23803 ts_tablet_manager.cc:1434] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:06:49.422358 22914 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-53ca1af745cb4fd48a40af06ae1c43ca received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:49.422748 22914 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-53ca1af745cb4fd48a40af06ae1c43ca on tablet 53ca1af745cb4fd48a40af06ae1c43ca with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:49.626096 23450 raft_consensus.cc:1217] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:49.719569 23450 raft_consensus.cc:1217] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:49.761059 23450 raft_consensus.cc:1217] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:49.894181 23450 raft_consensus.cc:1217] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:50.203528 23762 raft_consensus.cc:1064] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:50.207072 23762 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:50.213631 23450 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.214457 22749 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.215188 22824 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.215217 23762 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:50.216681 23727 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:50.217429 23807 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:50.229477 23727 raft_consensus.cc:1064] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:50.232450 23727 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:50.246417 23735 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.263904 22753 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.328482 22824 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.339931 23450 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.356853 23807 raft_consensus.cc:1064] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:50.373338 23451 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.379868 22747 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.384469 23735 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:50.395298 23727 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:50.360678 23807 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:50.416536 22825 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:06:50.439760 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 71657.9us
I20260709 13:06:50.443619 22650 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.479472 22756 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.483852 23762 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:50.489893 23727 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:50.491344 22823 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.495973 23453 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.500304 23821 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:06:50.519735 23735 raft_consensus.cc:1064] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:50.522789 23735 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:50.531955 23727 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:50.548810 22749 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.550932 23762 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:50.557801 23829 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.577533 23762 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.580366 22824 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.589303 23451 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.591305 23450 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:50.594046 22752 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.595124 22896 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:50.603832 23450 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.597384 22824 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.607421 22653 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.598183 22747 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.614784 23735 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:50.599426 22829 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.621290 23735 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:50.639650 22650 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.642493 23450 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:50.644928 23829 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.649192 23450 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.653363 23735 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:50.662263 22747 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:50.663757 23821 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:50.663642 23818 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.666805 23451 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.669113 22747 raft_consensus.cc:2955] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.673255 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4b9578eb52c342078c7555d5100bdcad with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:50.672921 22749 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.686370 22896 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:50.689790 22905 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:50.696836 23451 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:50.698063 23818 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:50.703474 22654 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.705521 22654 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.710682 23453 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:50.713568 22829 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:50.713936 22825 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:50.714831 23818 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:50.713918 22824 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.723300 23762 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.725869 23727 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:50.726424 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b9578eb52c342078c7555d5100bdcad with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:50.727504 23727 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:50.733119 22824 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.739041 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 317f6a16e0af41e2984a9b1626ab0edc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:50.739151 22896 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:50.756022 22747 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:50.763731 23818 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:50.764483 22651 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.767722 23453 raft_consensus.cc:2955] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.769183 23451 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:50.773242 23822 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:50.774184 23762 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.775688 23841 tablet_replica.cc:333] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:50.775563 23451 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.777097 23841 raft_consensus.cc:2243] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.777863 23841 raft_consensus.cc:2272] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.786185 23735 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.787590 23841 ts_tablet_manager.cc:1916] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:50.795571 22749 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.796883 23450 raft_consensus.cc:2955] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.802997 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 90274c6677ab4d37b51190e940c9608e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:50.803752 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53ca1af745cb4fd48a40af06ae1c43ca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:50.804940 22825 raft_consensus.cc:2955] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.815689 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 317f6a16e0af41e2984a9b1626ab0edc with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:06:50.816181 23841 ts_tablet_manager.cc:1929] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:50.816684 23841 log.cc:1199] T 4b9578eb52c342078c7555d5100bdcad P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/4b9578eb52c342078c7555d5100bdcad
I20260709 13:06:50.819523 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet 4b9578eb52c342078c7555d5100bdcad (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:50.828498 23842 tablet_replica.cc:333] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:06:50.829610 23842 raft_consensus.cc:2243] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.830374 23842 raft_consensus.cc:2272] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.833855 23842 ts_tablet_manager.cc:1916] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:50.835825 22650 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:50.847889 22654 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.850952 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 90274c6677ab4d37b51190e940c9608e with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:06:50.858794 23842 ts_tablet_manager.cc:1929] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:50.859510 23842 log.cc:1199] T 317f6a16e0af41e2984a9b1626ab0edc P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/317f6a16e0af41e2984a9b1626ab0edc
I20260709 13:06:50.862267 23842 tablet_replica.cc:333] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:06:50.862470 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 317f6a16e0af41e2984a9b1626ab0edc (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:50.863541 23842 raft_consensus.cc:2243] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.864091 23842 raft_consensus.cc:2272] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.865974 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet 53ca1af745cb4fd48a40af06ae1c43ca with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:50.866863 23841 tablet_replica.cc:333] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:50.870066 23841 raft_consensus.cc:2243] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:50.870798 23842 ts_tablet_manager.cc:1916] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:50.871030 23841 raft_consensus.cc:2272] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:50.877991 23841 ts_tablet_manager.cc:1916] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:50.887507 23842 ts_tablet_manager.cc:1929] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:50.887943 23842 log.cc:1199] T 90274c6677ab4d37b51190e940c9608e P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/90274c6677ab4d37b51190e940c9608e
I20260709 13:06:50.889242 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 90274c6677ab4d37b51190e940c9608e (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:50.891590 23841 ts_tablet_manager.cc:1929] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:50.892028 23841 log.cc:1199] T 53ca1af745cb4fd48a40af06ae1c43ca P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/53ca1af745cb4fd48a40af06ae1c43ca
I20260709 13:06:50.893959 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet 53ca1af745cb4fd48a40af06ae1c43ca (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:51.945061 22756 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:51.952199 22825 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:51.953200 22719 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c7d4d50470a044f698ff4852a4b634ca. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:51.952199 22896 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:51.955293 23846 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:51.956247 23843 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:51.967089 23843 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:51.969161 22905 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:51.978057 22825 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:51.986588 22651 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.990473 22905 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:51.998023 22825 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:51.998319 22868 consensus_peers.cc:597] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 09a9ba6091094973a526986ef573e2a9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:52.000432 22756 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:51.999784 23762 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:06:52.005481 23735 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:52.022094 23853 raft_consensus.cc:2955] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:52.022972 22824 raft_consensus.cc:2955] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:52.038043 22825 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.043462 22653 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.049175 22756 raft_consensus.cc:2955] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:06:52.056340 22794 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 00324d171dc0468fa259f13c1d4d74ea. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:52.111893 22905 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:52.121574 23814 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:52.140890 22756 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:52.142949 23814 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:52.150240 23860 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:52.158699 22905 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:52.168382 22903 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.174983 22651 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.196384 22747 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:52.203785 22756 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:52.206046 23762 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:52.208025 22824 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:52.221958 23853 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:52.221983 22868 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a98a0b933184c5590c80b44a6e3a544. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:52.234663 23762 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:52.236321 22756 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:52.237782 22825 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:52.249416 22653 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.474207 23867 ts_tablet_manager.cc:933] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:52.487496 23867 tablet_copy_client.cc:323] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:52.504304 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 09a9ba6091094973a526986ef573e2a9 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:52.504844 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 09a9ba6091094973a526986ef573e2a9 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-09a9ba6091094973a526986ef573e2a9
I20260709 13:06:52.505667 23869 ts_tablet_manager.cc:933] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:52.509229 23869 tablet_copy_client.cc:323] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.65:37213
I20260709 13:06:52.510849 22766 tablet_copy_service.cc:140] P 6a5191aaff76469bbf6c4735dc9e6509: Received BeginTabletCopySession request for tablet c7d4d50470a044f698ff4852a4b634ca from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:56780)
I20260709 13:06:52.511399 22766 tablet_copy_service.cc:161] P 6a5191aaff76469bbf6c4735dc9e6509: Beginning new tablet copy session on tablet c7d4d50470a044f698ff4852a4b634ca from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:56780: session id = c45e1805f049467697a7358fb566de2f-c7d4d50470a044f698ff4852a4b634ca
I20260709 13:06:52.513620 22914 tablet_copy_source_session.cc:215] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:52.517062 23867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a9ba6091094973a526986ef573e2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.517928 22766 tablet_copy_source_session.cc:215] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:52.522053 23869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d4d50470a044f698ff4852a4b634ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.538049 23869 tablet_copy_client.cc:806] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:52.539649 23869 tablet_copy_client.cc:670] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:52.544471 23869 tablet_copy_client.cc:538] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:52.547927 23867 tablet_copy_client.cc:806] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:52.548485 23867 tablet_copy_client.cc:670] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:52.552300 23867 tablet_copy_client.cc:538] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:52.553058 23869 tablet_bootstrap.cc:492] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:52.560292 23867 tablet_bootstrap.cc:492] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:52.574605 23869 tablet_bootstrap.cc:492] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:52.575783 23869 tablet_bootstrap.cc:492] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:52.576563 23869 ts_tablet_manager.cc:1403] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.024s	user 0.021s	sys 0.001s
I20260709 13:06:52.580660 23869 raft_consensus.cc:359] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.581657 23869 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:52.582262 23869 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.585893 23869 ts_tablet_manager.cc:1434] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:06:52.587846 22766 tablet_copy_service.cc:342] P 6a5191aaff76469bbf6c4735dc9e6509: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-c7d4d50470a044f698ff4852a4b634ca received from {username='slave'} at 127.0.0.1:56780
I20260709 13:06:52.588471 22766 tablet_copy_service.cc:434] P 6a5191aaff76469bbf6c4735dc9e6509: ending tablet copy session c45e1805f049467697a7358fb566de2f-c7d4d50470a044f698ff4852a4b634ca on tablet c7d4d50470a044f698ff4852a4b634ca with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:52.589437 23867 tablet_bootstrap.cc:492] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:52.590538 23867 tablet_bootstrap.cc:492] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:52.591591 23867 ts_tablet_manager.cc:1403] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.009s
I20260709 13:06:52.594556 23867 raft_consensus.cc:359] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.595320 23867 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:52.595903 23867 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.598701 23867 ts_tablet_manager.cc:1434] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:06:52.601246 22914 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-09a9ba6091094973a526986ef573e2a9 received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:52.601706 22914 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-09a9ba6091094973a526986ef573e2a9 on tablet 09a9ba6091094973a526986ef573e2a9 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:52.605094 23869 ts_tablet_manager.cc:933] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:52.610595 23869 tablet_copy_client.cc:323] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:52.612404 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 00324d171dc0468fa259f13c1d4d74ea from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:52.612907 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 00324d171dc0468fa259f13c1d4d74ea from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-00324d171dc0468fa259f13c1d4d74ea
I20260709 13:06:52.619771 22841 tablet_copy_source_session.cc:215] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:52.622581 23869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00324d171dc0468fa259f13c1d4d74ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.634835 23869 tablet_copy_client.cc:806] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:52.635365 23869 tablet_copy_client.cc:670] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:52.638702 23869 tablet_copy_client.cc:538] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:52.644512 23869 tablet_bootstrap.cc:492] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:52.662523 23869 tablet_bootstrap.cc:492] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:52.663349 23869 tablet_bootstrap.cc:492] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:52.663925 23869 ts_tablet_manager.cc:1403] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 13:06:52.666110 23869 raft_consensus.cc:359] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.667076 23869 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:52.667671 23869 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.669898 23869 ts_tablet_manager.cc:1434] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:06:52.674505 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-00324d171dc0468fa259f13c1d4d74ea received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:52.674901 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-00324d171dc0468fa259f13c1d4d74ea on tablet 00324d171dc0468fa259f13c1d4d74ea with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:52.739936 23869 ts_tablet_manager.cc:933] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:52.775292 23869 tablet_copy_client.cc:323] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:52.776890 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 3a98a0b933184c5590c80b44a6e3a544 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:52.777412 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 3a98a0b933184c5590c80b44a6e3a544 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-3a98a0b933184c5590c80b44a6e3a544
I20260709 13:06:52.786803 22914 tablet_copy_source_session.cc:215] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:52.796682 23869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a98a0b933184c5590c80b44a6e3a544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:52.807507 23869 tablet_copy_client.cc:806] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:52.807979 23869 tablet_copy_client.cc:670] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:52.811507 23869 tablet_copy_client.cc:538] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:52.819155 23869 tablet_bootstrap.cc:492] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:52.835489 23869 tablet_bootstrap.cc:492] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:52.836232 23869 tablet_bootstrap.cc:492] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:52.837013 23869 ts_tablet_manager.cc:1403] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:06:52.839903 23869 raft_consensus.cc:359] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.840718 23869 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:52.841374 23869 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:52.847729 23869 ts_tablet_manager.cc:1434] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:52.849427 22914 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-3a98a0b933184c5590c80b44a6e3a544 received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:52.849884 22914 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-3a98a0b933184c5590c80b44a6e3a544 on tablet 3a98a0b933184c5590c80b44a6e3a544 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:52.890429 23450 raft_consensus.cc:1217] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:53.074103 23450 raft_consensus.cc:1217] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:53.195482 23450 raft_consensus.cc:1217] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:53.306641 23451 raft_consensus.cc:1217] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:53.354030 23881 raft_consensus.cc:1064] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:53.357038 23881 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:53.371019 23451 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.372314 22903 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.372430 22824 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.373852 23846 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.374568 23881 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.379309 23846 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.392141 23846 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.404464 22824 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.409461 22903 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.415100 22651 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.422878 23451 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.452561 22749 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:53.458037 23451 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:53.458655 22824 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:53.459748 23886 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:53.460868 23844 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:53.467049 23886 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.468633 23451 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.473217 22824 raft_consensus.cc:2955] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.485276 22654 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.488394 22641 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c7d4d50470a044f698ff4852a4b634ca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:53.505455 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet c7d4d50470a044f698ff4852a4b634ca with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:06:53.510138 23891 tablet_replica.cc:333] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:06:53.511407 23891 raft_consensus.cc:2243] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:53.517069 23891 raft_consensus.cc:2272] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:53.521210 23891 ts_tablet_manager.cc:1916] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:53.536477 23891 ts_tablet_manager.cc:1929] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:53.536831 23891 log.cc:1199] T c7d4d50470a044f698ff4852a4b634ca P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/c7d4d50470a044f698ff4852a4b634ca
I20260709 13:06:53.538494 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet c7d4d50470a044f698ff4852a4b634ca (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:53.573194 23862 raft_consensus.cc:1064] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:53.577049 23862 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:53.594671 23450 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.595875 22903 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.596326 23880 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.598104 23880 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.600817 22749 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.603983 23861 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.616430 23862 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.618379 22903 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.619115 22749 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.632705 23450 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.634521 22653 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.656961 22824 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:53.663717 22749 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:53.666452 23879 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:53.668885 23450 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:53.670109 23879 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:53.673477 23880 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.675359 22749 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.685307 23853 raft_consensus.cc:1064] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:53.686226 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 00324d171dc0468fa259f13c1d4d74ea with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:53.687780 23853 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:53.693248 23451 raft_consensus.cc:2955] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.705173 23450 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.706508 23735 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.709048 22824 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.710476 23735 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.710409 22653 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.717147 22749 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.718467 23735 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.731146 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 00324d171dc0468fa259f13c1d4d74ea with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:06:53.732499 23891 tablet_replica.cc:333] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:06:53.733866 23891 raft_consensus.cc:2243] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:53.734531 23891 raft_consensus.cc:2272] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:53.738514 23891 ts_tablet_manager.cc:1916] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:53.751570 23877 raft_consensus.cc:2955] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.753361 23891 ts_tablet_manager.cc:1929] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:53.753909 23891 log.cc:1199] T 00324d171dc0468fa259f13c1d4d74ea P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/00324d171dc0468fa259f13c1d4d74ea
I20260709 13:06:53.755318 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 00324d171dc0468fa259f13c1d4d74ea (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:53.758672 22824 raft_consensus.cc:2955] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.772519 23453 raft_consensus.cc:2955] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.776819 22653 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.783087 23735 raft_consensus.cc:1064] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:53.785075 23735 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:06:53.789402 23899 consensus_queue.cc:1175] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aad11e70ca94fce8f672db05f1c28c3" responder_term: 1 status { last_received { term: 1 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 1 index: 3 } }
I20260709 13:06:53.807124 23453 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.809454 23901 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:53.809911 22903 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:53.816339 22829 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.817550 23901 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:53.829202 22753 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:53.834440 23853 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:06:53.837908 23451 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:53.839321 23902 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:53.846885 23877 raft_consensus.cc:2955] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.848133 22824 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:53.850821 23853 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:53.854774 23451 raft_consensus.cc:2955] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.860980 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 09a9ba6091094973a526986ef573e2a9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:53.868496 22650 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.869068 22824 raft_consensus.cc:2955] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.878105 23899 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.882136 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 09a9ba6091094973a526986ef573e2a9 with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:06:53.888360 22829 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.896991 23453 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.911377 23907 tablet_replica.cc:333] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:06:53.912999 23907 raft_consensus.cc:2243] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:53.913609 23907 pending_rounds.cc:70] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: Trying to abort 1 pending ops.
I20260709 13:06:53.914018 23907 pending_rounds.cc:77] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305635486460592128 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "09a9ba6091094973a526986ef573e2a9" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } } }
I20260709 13:06:53.918186 23907 raft_consensus.cc:2917] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:06:53.918640 23907 raft_consensus.cc:2889] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:06:53.919351 23907 raft_consensus.cc:2272] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:53.920709 22654 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.923921 23907 ts_tablet_manager.cc:1916] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:53.945034 23907 ts_tablet_manager.cc:1929] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:53.945515 23907 log.cc:1199] T 09a9ba6091094973a526986ef573e2a9 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/09a9ba6091094973a526986ef573e2a9
I20260709 13:06:53.948167 22756 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.953866 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 09a9ba6091094973a526986ef573e2a9 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:53.969611 22903 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:53.977798 23453 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:53.978804 22756 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:53.978993 23901 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:53.980122 23902 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:53.986677 23902 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.988298 23453 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:53.994163 22756 raft_consensus.cc:2955] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:54.001070 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3a98a0b933184c5590c80b44a6e3a544 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:54.005599 22654 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:54.019373 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a98a0b933184c5590c80b44a6e3a544 with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:54.024564 23908 tablet_replica.cc:333] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:54.025265 23908 raft_consensus.cc:2243] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:54.025805 23908 raft_consensus.cc:2272] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:54.029137 23908 ts_tablet_manager.cc:1916] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:54.040426 23908 ts_tablet_manager.cc:1929] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:54.040766 23908 log.cc:1199] T 3a98a0b933184c5590c80b44a6e3a544 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/3a98a0b933184c5590c80b44a6e3a544
I20260709 13:06:54.042171 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet 3a98a0b933184c5590c80b44a6e3a544 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:54.114711 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:54.166613 22824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:06:54.167299 22824 raft_consensus.cc:606] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:54.183597 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:54.184041 22903 raft_consensus.cc:606] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:54.186542 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:54.187074 22903 raft_consensus.cc:606] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:54.189610 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:54.190034 22903 raft_consensus.cc:606] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:54.193153 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:54.193583 22903 raft_consensus.cc:606] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:54.196973 22905 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:54.197419 22905 raft_consensus.cc:606] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:54.198393 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:06:54.200088 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:54.231060 23853 raft_consensus.cc:993] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:54.231667 23902 raft_consensus.cc:1081] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:54.233373 22756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:06:54.234298 22756 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:54.234673 22756 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.239691 22756 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:54.241341 22756 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:54.241732 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:54.242347 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:54.242449 22903 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:54.242900 22824 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.243040 22903 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:54.243846 22903 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:54.245174 22903 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.249008 22824 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:54.250042 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:54.250451 22903 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:54.250924 23881 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:54.252475 23881 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:54.253396 23881 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:06:54.264215 22650 catalog_manager.cc:5697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.408242 23853 raft_consensus.cc:993] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:54.408747 23901 raft_consensus.cc:1081] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:54.410211 22749 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:06:54.410575 22749 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:54.410784 22749 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.415537 22749 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:54.417053 22749 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:54.417795 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:54.418514 22829 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.419435 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:54.420064 22903 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:54.420375 22903 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:54.421078 22903 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:54.423103 22903 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.426198 22829 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:54.427824 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:54.428658 23881 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:54.429126 23881 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:54.429275 23902 raft_consensus.cc:993] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:54.429776 23902 raft_consensus.cc:1081] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:54.430023 23881 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:54.431260 22903 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:54.431982 23453 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:06:54.432492 23453 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:54.432825 23453 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.441378 23453 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:54.443869 23453 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:54.445012 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:54.444682 22650 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.445597 22829 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.445923 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:54.446744 22905 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:54.447041 22905 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:54.447727 22905 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:54.448802 22905 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.451272 22829 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:54.454855 22905 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:54.455909 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:54.456600 23873 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:54.457683 23873 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:54.458482 23873 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:54.467579 22650 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.586095 23880 raft_consensus.cc:993] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:54.586573 23861 raft_consensus.cc:1081] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:54.587994 22749 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38488
I20260709 13:06:54.588475 22749 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:54.588812 22749 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.594936 22749 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:54.596199 22749 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:54.596897 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:54.597086 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:54.597506 22824 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:54.597652 22905 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.597836 22824 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:54.598606 22824 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:54.599418 22824 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.602366 22905 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:54.603259 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:54.604001 23886 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:54.604768 22824 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:54.604436 23886 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:54.605733 23886 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:54.618983 22650 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.691790 23853 raft_consensus.cc:993] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:54.692307 23902 raft_consensus.cc:1081] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:54.693897 22749 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:06:54.694388 22749 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:54.694718 22749 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.699285 22749 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:54.700496 22749 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:54.701182 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:54.701730 22824 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.701475 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:54.703837 22905 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:54.706189 22905 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:54.707087 22905 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:54.708382 22905 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.708403 22824 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:54.709595 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:54.710319 23886 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:54.710887 23886 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:54.711805 23886 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:54.716727 22905 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:54.726508 23901 raft_consensus.cc:993] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:54.727334 23853 raft_consensus.cc:1081] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:54.727770 22650 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.728693 23451 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:06:54.729108 23451 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:54.729423 23451 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.734931 22905 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.736377 23912 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:54.736423 23451 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:54.740388 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:54.741289 22824 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.742851 23451 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:54.751034 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:54.752866 22903 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:54.753158 22903 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:54.753927 22903 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:54.755097 22903 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:54.757562 22824 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:54.758867 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:54.759721 23873 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:54.760236 23873 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:54.761050 22903 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:54.761029 23873 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:54.767449 22829 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.768797 23912 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:54.775875 22653 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.826746 22905 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.827827 23912 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:54.837368 22824 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:54.838475 23881 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:54.852291 22824 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:54.853591 23873 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:54.868144 22905 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:54.878559 23873 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:55.050724 22905 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:55.051941 23886 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.072372 22824 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:55.073691 23886 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.102005 22824 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:55.109134 22753 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:55.109539 22794 consensus_peers.cc:597] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60a957de2a744260812f8da6f1bd9b87. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:55.109541 22905 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:55.110379 23880 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.111162 23931 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.126689 23880 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.129506 22753 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.133044 22903 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:55.134384 23886 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.147029 22905 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.151075 22749 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:55.156272 22654 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.171259 22824 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:55.180627 23886 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:55.198318 22719 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43c3bd6e29d8478b8aa6c100c8fd84c2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:55.200742 22905 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:55.201040 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:55.205367 23886 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.204627 22824 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:55.209411 22824 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:55.210278 23881 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.210584 23873 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:55.221356 23912 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.225211 22905 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.231227 22825 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.237371 22749 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:55.250209 22654 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.254873 22824 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:55.256011 23912 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:55.263676 22719 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 267f1f9ab3f742418b8505ba4859ab18. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:55.268461 22903 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:06:55.274482 22905 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:55.272564 23912 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:55.276662 23881 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.279292 23915 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:55.280259 22825 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.285882 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:55.290515 22905 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.305778 22896 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:55.309700 22653 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.343780 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:55.344527 22903 raft_consensus.cc:606] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:55.347040 22825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:06:55.347466 22825 raft_consensus.cc:606] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:55.349820 22905 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:55.350299 22905 raft_consensus.cc:606] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:55.352891 22905 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:55.353302 22905 raft_consensus.cc:606] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:55.356305 22824 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:55.356501 22868 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ea49b0ee95d4eafbb647436a98fcaa1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:55.357394 22896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:55.357884 22896 raft_consensus.cc:606] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:55.357697 23919 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:55.358369 22749 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:55.359912 23853 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.363724 22896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:55.364231 22896 raft_consensus.cc:606] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:55.366976 23853 raft_consensus.cc:993] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:55.368091 23902 raft_consensus.cc:1081] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:55.369793 22747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:06:55.370252 22747 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:55.370587 22747 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.413033 22747 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.414805 22747 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:55.433094 23931 raft_consensus.cc:993] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:55.433537 23931 raft_consensus.cc:1081] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:55.435089 22896 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43998
I20260709 13:06:55.435582 22896 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:55.435891 22896 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.442181 22896 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.444461 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:55.444967 22753 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.452272 22753 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:06:55.453145 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:55.453853 22825 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.454071 22871 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:55.455304 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:55.455807 22905 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:55.456079 22905 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:55.456764 22905 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.458796 22905 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.460572 22825 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:55.464887 22905 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:55.465636 23919 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:55.466284 23919 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:55.467113 23919 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.470371 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:55.471203 23881 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:55.471664 23881 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:55.472585 23881 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.476504 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:55.477017 22829 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:55.477281 22829 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:55.477960 22829 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.483362 22829 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.487250 22747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:06:55.487784 22747 raft_consensus.cc:606] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:55.490177 22829 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:06:55.476107 22896 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:55.495901 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:06:55.498054 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:55.505940 23886 raft_consensus.cc:993] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:55.506352 23886 raft_consensus.cc:1081] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:55.495517 23853 raft_consensus.cc:993] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:55.507664 23853 raft_consensus.cc:1081] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:55.509065 22753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:06:55.509554 22753 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:55.509886 22753 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.515640 23451 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38074
I20260709 13:06:55.516238 23451 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:55.516624 23451 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.522851 22753 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.524761 22753 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:55.524566 23451 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:55.526580 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:55.527065 22829 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.527139 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:55.527649 22903 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:55.527927 22903 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:55.528666 22903 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.530735 22903 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.532763 22829 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:55.526301 22650 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.540321 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:55.541020 22753 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:55.541394 22753 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:55.544593 22903 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:55.545408 22753 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:55.546526 22753 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.547961 23853 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.540238 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:55.551672 23886 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:55.552047 22753 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:55.552192 23886 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:55.551596 22824 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.553201 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:55.553068 23886 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.554050 23873 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:55.554525 23873 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:55.555302 23873 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:55.568650 23919 raft_consensus.cc:993] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:55.569998 23919 raft_consensus.cc:1081] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:55.572288 23450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:06:55.572923 23450 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:55.573264 23450 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.578945 23451 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:55.584745 23951 ts_tablet_manager.cc:933] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:55.584815 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:55.585983 22824 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.592078 23450 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:55.595432 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:55.596207 22749 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.600049 22824 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:55.600466 22650 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.606022 22749 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:55.607230 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:55.607969 23915 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:55.615434 23450 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:55.620394 23915 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:55.621249 23915 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:55.639403 23960 ts_tablet_manager.cc:933] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:55.644717 23951 tablet_copy_client.cc:323] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:55.646212 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 60a957de2a744260812f8da6f1bd9b87 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:55.646703 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 60a957de2a744260812f8da6f1bd9b87 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-60a957de2a744260812f8da6f1bd9b87
I20260709 13:06:55.648861 22654 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.658977 22653 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.661849 22653 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:55.675763 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 114330us
I20260709 13:06:55.679740 22749 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:55.681262 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:55.681785 22896 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:55.682096 22896 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:55.684573 22841 tablet_copy_source_session.cc:215] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:55.683009 22896 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:55.689234 22896 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.689837 23951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a957de2a744260812f8da6f1bd9b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:55.721784 22896 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:55.739145 23853 raft_consensus.cc:993] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:55.742532 23952 raft_consensus.cc:1081] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:55.745702 23453 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:06:55.746178 23453 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:55.746493 23453 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.747327 23961 ts_tablet_manager.cc:933] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:06:55.748374 22749 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:55.756524 23961 tablet_copy_client.cc:323] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.65:37213
W20260709 13:06:55.766247 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48560.6us
I20260709 13:06:55.767783 23453 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:55.778636 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:55.779187 22829 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.780969 23951 tablet_copy_client.cc:806] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:55.788157 22829 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:55.790392 23453 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:55.793984 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:55.795257 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:55.795774 22896 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:55.796101 22896 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:55.795568 22653 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.796751 22896 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:55.798019 22896 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.801679 23957 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:55.802047 23960 tablet_copy_client.cc:323] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.65:37213
I20260709 13:06:55.802289 23957 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:55.808821 23957 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:55.810580 22896 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:55.812197 22766 tablet_copy_service.cc:140] P 6a5191aaff76469bbf6c4735dc9e6509: Received BeginTabletCopySession request for tablet 267f1f9ab3f742418b8505ba4859ab18 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:56780)
I20260709 13:06:55.812913 22766 tablet_copy_service.cc:161] P 6a5191aaff76469bbf6c4735dc9e6509: Beginning new tablet copy session on tablet 267f1f9ab3f742418b8505ba4859ab18 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:56780: session id = c45e1805f049467697a7358fb566de2f-267f1f9ab3f742418b8505ba4859ab18
I20260709 13:06:55.828119 23919 raft_consensus.cc:993] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:55.832712 22766 tablet_copy_source_session.cc:215] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:55.833575 23919 raft_consensus.cc:1081] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:55.833859 22765 tablet_copy_service.cc:140] P 6a5191aaff76469bbf6c4735dc9e6509: Received BeginTabletCopySession request for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:56780)
I20260709 13:06:55.852931 22765 tablet_copy_service.cc:161] P 6a5191aaff76469bbf6c4735dc9e6509: Beginning new tablet copy session on tablet 43c3bd6e29d8478b8aa6c100c8fd84c2 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:56780: session id = c45e1805f049467697a7358fb566de2f-43c3bd6e29d8478b8aa6c100c8fd84c2
I20260709 13:06:55.857733 22747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:06:55.858217 22747 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:55.858530 22747 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.859182 23961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267f1f9ab3f742418b8505ba4859ab18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:55.838526 23951 tablet_copy_client.cc:670] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:55.841020 22653 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:55.865763 22765 tablet_copy_source_session.cc:215] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:55.852453 23967 ts_tablet_manager.cc:933] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:55.869107 23960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:55.873508 23951 tablet_copy_client.cc:538] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:55.873718 22747 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.877898 22747 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:55.878691 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:55.881039 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:55.883761 22824 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.883813 22896 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:55.884238 22896 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:55.884941 22896 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.885924 22896 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:55.883239 23967 tablet_copy_client.cc:323] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:06:55.888805 23951 tablet_bootstrap.cc:492] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:55.889536 22824 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:55.889954 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 8ea49b0ee95d4eafbb647436a98fcaa1 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:06:55.890476 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 8ea49b0ee95d4eafbb647436a98fcaa1 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-8ea49b0ee95d4eafbb647436a98fcaa1
I20260709 13:06:55.891227 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:55.893431 23961 tablet_copy_client.cc:806] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:55.894155 23881 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:55.894213 23961 tablet_copy_client.cc:670] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:55.894712 23881 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:55.898864 23881 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:55.906065 23961 tablet_copy_client.cc:538] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:55.910864 22914 tablet_copy_source_session.cc:215] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:55.913229 22896 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:55.919863 23960 tablet_copy_client.cc:806] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:55.921576 23967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea49b0ee95d4eafbb647436a98fcaa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:55.931965 23960 tablet_copy_client.cc:670] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:55.936473 23960 tablet_copy_client.cc:538] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:55.940379 22747 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:55.948585 23919 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.958038 23961 tablet_bootstrap.cc:492] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:55.948565 22653 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.959827 23967 tablet_copy_client.cc:806] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:55.960479 23967 tablet_copy_client.cc:670] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:55.964932 22824 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:55.966708 23965 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:55.968108 22824 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:55.970587 23881 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.977743 23967 tablet_copy_client.cc:538] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:55.981940 23960 tablet_bootstrap.cc:492] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:55.987491 23967 tablet_bootstrap.cc:492] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:06:55.981972 23951 tablet_bootstrap.cc:492] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:55.990271 23951 tablet_bootstrap.cc:492] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:55.992880 22825 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:55.994275 23919 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:55.996986 23951 ts_tablet_manager.cc:1403] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.108s	user 0.024s	sys 0.012s
I20260709 13:06:56.021572 22829 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:56.031903 23957 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:56.033985 22903 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:56.035913 23881 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:56.061602 23951 raft_consensus.cc:359] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:56.062971 23951 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:56.063791 23951 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:56.099038 22903 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:56.118359 22905 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:56.140473 23965 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:06:56.148959 23915 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:56.154582 22747 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:56.148115 23951 ts_tablet_manager.cc:1434] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.151s	user 0.006s	sys 0.001s
I20260709 13:06:56.169916 23915 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:06:56.186319 22747 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:06:56.240144 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 107153us
I20260709 13:06:56.240175 23915 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:06:56.261147 23967 tablet_bootstrap.cc:492] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:56.276655 22829 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:56.281669 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-60a957de2a744260812f8da6f1bd9b87 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:56.299928 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-60a957de2a744260812f8da6f1bd9b87 on tablet 60a957de2a744260812f8da6f1bd9b87 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:56.330832 23967 tablet_bootstrap.cc:492] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:56.348366 23967 ts_tablet_manager.cc:1403] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.361s	user 0.032s	sys 0.036s
I20260709 13:06:56.395370 22903 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:56.404312 23881 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:56.410471 22903 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:56.459512 23915 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:06:56.552690 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:56.583779 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 269890us
W20260709 13:06:56.618877 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48173.1us
I20260709 13:06:56.602676 23967 raft_consensus.cc:359] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:56.630276 23967 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:56.630915 23967 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:56.632726 23967 ts_tablet_manager.cc:1434] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.246s	user 0.006s	sys 0.001s
I20260709 13:06:56.631691 23960 tablet_bootstrap.cc:492] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:56.634927 23960 tablet_bootstrap.cc:492] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:56.635782 23960 ts_tablet_manager.cc:1403] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.654s	user 0.037s	sys 0.009s
I20260709 13:06:56.619902 23957 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
W20260709 13:06:56.648836 24005 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 13:06:56.650863 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 356531us
I20260709 13:06:56.670856 23961 tablet_bootstrap.cc:492] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:56.673770 23961 tablet_bootstrap.cc:492] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:56.674434 23961 ts_tablet_manager.cc:1403] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.717s	user 0.029s	sys 0.018s
W20260709 13:06:56.682802 23996 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.104s	user 0.003s	sys 0.000s
I20260709 13:06:56.689030 23960 raft_consensus.cc:359] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:56.691540 23960 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:56.692191 23960 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:56.701557 22914 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-8ea49b0ee95d4eafbb647436a98fcaa1 received from {username='slave'} at 127.0.0.1:42490
I20260709 13:06:56.701967 22914 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-8ea49b0ee95d4eafbb647436a98fcaa1 on tablet 8ea49b0ee95d4eafbb647436a98fcaa1 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:56.703948 23960 ts_tablet_manager.cc:1434] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.000s
I20260709 13:06:56.708572 22765 tablet_copy_service.cc:342] P 6a5191aaff76469bbf6c4735dc9e6509: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-43c3bd6e29d8478b8aa6c100c8fd84c2 received from {username='slave'} at 127.0.0.1:56780
I20260709 13:06:56.730908 22823 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:56.731052 22765 tablet_copy_service.cc:434] P 6a5191aaff76469bbf6c4735dc9e6509: ending tablet copy session c45e1805f049467697a7358fb566de2f-43c3bd6e29d8478b8aa6c100c8fd84c2 on tablet 43c3bd6e29d8478b8aa6c100c8fd84c2 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:56.755762 23961 raft_consensus.cc:359] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:56.756687 23961 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:56.757872 23961 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:56.759048 23912 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:56.769732 23452 raft_consensus.cc:1217] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:56.789283 23453 raft_consensus.cc:1217] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:56.800019 23961 ts_tablet_manager.cc:1434] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.125s	user 0.003s	sys 0.005s
I20260709 13:06:56.801608 22765 tablet_copy_service.cc:342] P 6a5191aaff76469bbf6c4735dc9e6509: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-267f1f9ab3f742418b8505ba4859ab18 received from {username='slave'} at 127.0.0.1:56780
I20260709 13:06:56.802021 22765 tablet_copy_service.cc:434] P 6a5191aaff76469bbf6c4735dc9e6509: ending tablet copy session c45e1805f049467697a7358fb566de2f-267f1f9ab3f742418b8505ba4859ab18 on tablet 267f1f9ab3f742418b8505ba4859ab18 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:56.832692 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:56.901504 23453 raft_consensus.cc:1217] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:06:56.941650 23453 raft_consensus.cc:1217] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:06:56.985534 22749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:06:56.986189 22749 raft_consensus.cc:606] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:57.014155 22749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:06:57.014703 22749 raft_consensus.cc:606] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:57.020053 22905 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:57.020592 22905 raft_consensus.cc:606] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:57.023067 22905 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:57.023553 22905 raft_consensus.cc:606] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:57.026031 22829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:06:57.026515 22829 raft_consensus.cc:606] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:57.028194 23912 raft_consensus.cc:993] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:57.028638 23912 raft_consensus.cc:1081] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:57.030216 23881 raft_consensus.cc:993] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:57.030727 23991 raft_consensus.cc:1081] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:57.039286 22905 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:06:57.039611 22903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:06:57.039768 22905 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:57.040119 22903 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:57.040190 22905 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:57.040536 22903 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:57.043749 22749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:06:57.044279 22749 raft_consensus.cc:606] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:57.045842 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:06:57.047323 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:57.047539 22903 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:57.049404 22903 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:57.049357 22905 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:57.050420 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:57.050984 22825 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:57.051234 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:57.051733 22749 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:57.051995 22749 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:57.052671 22749 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:57.053902 22749 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:57.057145 22825 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:06:57.059120 22905 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:57.059108 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:57.060184 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:57.060694 22753 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:57.060845 24010 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:57.061028 22753 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:57.061403 24010 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:57.061836 22753 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:57.062233 24010 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:57.062841 22753 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:57.071089 22753 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:06:57.071457 22749 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:06:57.073558 22871 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:57.074389 23952 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:57.071730 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:57.074846 23952 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:57.075214 22825 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:57.075641 23952 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:06:57.098243 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49447.2us
I20260709 13:06:57.101078 22653 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 3, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.112188 22825 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:06:57.114636 23881 raft_consensus.cc:993] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:57.115118 23881 raft_consensus.cc:1081] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:57.117789 23452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38074
I20260709 13:06:57.118379 23452 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:57.118746 23452 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.128724 23931 raft_consensus.cc:1064] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:57.131718 23931 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.139907 23452 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.147475 23452 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:57.147842 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:57.148379 22905 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.150496 23452 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:57.151561 22903 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:57.153493 23989 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:57.155292 23989 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:57.148058 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:57.156193 22753 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:57.156502 22753 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:57.157183 22753 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.160714 22753 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.169960 22749 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:57.170490 23986 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.172703 23989 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:57.174820 22905 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:57.175935 22753 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:57.176141 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:57.177928 24014 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:57.178436 24014 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:57.179266 24014 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.181389 24010 raft_consensus.cc:993] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:57.181860 24010 raft_consensus.cc:1081] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:57.184080 23452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:06:57.184615 23452 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:57.184962 23452 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.190879 22903 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.214176 23453 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.216616 23452 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.218520 23452 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:57.219831 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:57.220429 22753 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.220260 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:57.221091 22905 raft_consensus.cc:3055] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:57.221434 22905 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:57.222178 22905 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.223309 22905 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.229278 22905 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:57.230222 23416 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:57.231814 23962 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:57.232278 23962 raft_consensus.cc:697] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:57.232566 22753 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:57.233064 23962 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.248889 22653 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 3, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.259599 22749 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.251797 22651 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.273833 22654 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.284174 24008 raft_consensus.cc:993] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:57.284612 24008 raft_consensus.cc:1081] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:57.289602 23452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:06:57.290072 23452 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:57.290405 23452 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.294258 23912 raft_consensus.cc:1064] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:57.296221 23452 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.297835 23452 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:57.298713 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:57.300747 22753 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.303277 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:57.303799 22905 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:57.298260 23912 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.304100 22905 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:57.304833 22905 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.305897 22905 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.306437 22753 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:57.307507 23419 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:57.308961 24014 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:57.309414 24014 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:57.310271 24014 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.313658 22905 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:57.321853 22650 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.333472 22825 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.337847 22905 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:57.339443 22829 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:57.340662 23453 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:57.341046 23881 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:57.343600 23965 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:57.344372 23991 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:57.350353 23453 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:57.354334 22896 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:57.356282 23861 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:57.358829 23986 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:57.371706 24001 raft_consensus.cc:993] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:57.372167 24001 raft_consensus.cc:1081] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:57.372077 22896 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.372254 23861 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.374075 23991 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.380010 22650 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.380843 22753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38488
I20260709 13:06:57.381383 22753 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:57.381894 22753 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.381490 22825 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.388847 23452 raft_consensus.cc:2955] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.390072 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 60a957de2a744260812f8da6f1bd9b87 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:57.395419 22753 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.397743 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:57.398226 22829 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:57.398486 22829 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:57.399122 22829 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.400106 22829 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.403561 22651 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.415832 22896 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.419546 22829 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:57.405177 22651 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.422857 23450 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.424598 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:57.429339 23991 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:57.429863 23991 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:57.436277 23991 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.438228 22753 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:06:57.439932 23952 raft_consensus.cc:1064] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:57.440588 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:06:57.441159 23450 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.442279 23952 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.450030 22749 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:57.450714 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 60a957de2a744260812f8da6f1bd9b87 with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:06:57.451318 24008 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:57.452663 23450 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:57.486600 24036 tablet_replica.cc:333] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:06:57.505470 22651 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.509706 23450 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Advancing to term 3
I20260709 13:06:57.511420 22829 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:57.514150 24008 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:57.519565 23450 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:06:57.519768 24036 raft_consensus.cc:2243] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.520612 24036 raft_consensus.cc:2272] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.523442 24036 ts_tablet_manager.cc:1916] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:57.533748 22747 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:57.535091 24004 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:57.539280 22829 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:06:57.541033 24008 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:57.521384 24003 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:06:57.571911 22753 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.613564 22825 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:57.619796 23452 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:57.625025 23952 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:57.630336 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 94987.3us
I20260709 13:06:57.638417 24003 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:57.641856 22896 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:57.643323 23962 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:57.647924 22749 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:57.649763 22905 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:57.650913 22753 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:06:57.653096 23962 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:57.654886 22823 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:57.657080 23450 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:57.659478 23965 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:57.659983 24004 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:57.663951 23962 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:57.669399 23965 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:57.696311 22747 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:57.697764 24014 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:57.694561 24047 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.712001 23450 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.718473 22823 raft_consensus.cc:2955] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.722653 22747 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:57.724025 24014 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:57.735918 24036 ts_tablet_manager.cc:1929] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:57.738193 22903 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:57.747980 22652 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.757078 23962 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:57.753221 24003 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.765348 24036 log.cc:1199] T 60a957de2a744260812f8da6f1bd9b87 P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/60a957de2a744260812f8da6f1bd9b87
I20260709 13:06:57.768971 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2 with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:06:57.794742 23453 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:06:57.797063 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 54861.2us
I20260709 13:06:57.851648 24003 raft_consensus.cc:1064] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:06:57.878283 24062 tablet_replica.cc:333] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:06:57.887283 22753 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.815261 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 60a957de2a744260812f8da6f1bd9b87 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
W20260709 13:06:57.920815 22641 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 105560us
I20260709 13:06:57.923472 22641 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 43c3bd6e29d8478b8aa6c100c8fd84c2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:57.926831 24062 raft_consensus.cc:2243] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:57.930116 23453 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:57.932427 24048 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:57.943589 22823 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:57.945201 24062 raft_consensus.cc:2272] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:57.930114 24003 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:57.950235 24062 ts_tablet_manager.cc:1916] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:57.962078 22651 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.968475 22753 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:57.971230 23452 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:57.987775 24004 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:57.971932 22829 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:06:57.976498 22830 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:06:57.993917 23881 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:58.007823 24004 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:58.009277 24004 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:58.018549 24045 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.022851 23453 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.025099 22825 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.037621 24062 ts_tablet_manager.cc:1929] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:58.038089 24062 log.cc:1199] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/43c3bd6e29d8478b8aa6c100c8fd84c2
I20260709 13:06:58.042150 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 43c3bd6e29d8478b8aa6c100c8fd84c2 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:58.042371 22753 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.043111 22652 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 3 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.056670 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:58.072731 22905 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:58.080415 23453 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:58.080520 22753 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:06:58.081724 24045 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:58.083102 24004 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:58.093194 22905 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:58.094964 24042 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.104032 22756 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:58.104514 23452 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:06:58.106022 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8ea49b0ee95d4eafbb647436a98fcaa1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:58.106139 24008 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:58.107144 24003 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:58.108148 23453 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.114759 22654 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.129889 24042 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.135392 23452 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.138402 22756 raft_consensus.cc:2955] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.130091 22753 raft_consensus.cc:2955] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:06:58.131723 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ea49b0ee95d4eafbb647436a98fcaa1 with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:58.155412 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:58.157989 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 267f1f9ab3f742418b8505ba4859ab18 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:58.167956 22651 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 5 to 6, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.174748 24071 tablet_replica.cc:333] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:58.175917 24071 raft_consensus.cc:2243] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:58.176586 24071 raft_consensus.cc:2272] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:58.182806 24071 ts_tablet_manager.cc:1916] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:58.189435 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet 267f1f9ab3f742418b8505ba4859ab18 with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43622
I20260709 13:06:58.204939 24071 ts_tablet_manager.cc:1929] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:06:58.205470 24071 log.cc:1199] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/8ea49b0ee95d4eafbb647436a98fcaa1
I20260709 13:06:58.207085 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet 8ea49b0ee95d4eafbb647436a98fcaa1 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:58.207556 24071 tablet_replica.cc:333] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:06:58.208559 24071 raft_consensus.cc:2243] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:58.209363 24071 raft_consensus.cc:2272] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:58.212865 24071 ts_tablet_manager.cc:1916] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:58.228236 24071 ts_tablet_manager.cc:1929] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:06:58.228654 24071 log.cc:1199] T 267f1f9ab3f742418b8505ba4859ab18 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/267f1f9ab3f742418b8505ba4859ab18
I20260709 13:06:58.230089 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet 267f1f9ab3f742418b8505ba4859ab18 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:06:58.235211 22896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:58.235870 22896 raft_consensus.cc:606] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:58.238726 22756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f881fed251c4393ac053086bd572549"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:06:58.239305 22756 raft_consensus.cc:606] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:58.241745 22825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:06:58.242208 22825 raft_consensus.cc:606] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:58.254563 22896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:58.255079 22896 raft_consensus.cc:606] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:58.257447 22825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:06:58.257822 22825 raft_consensus.cc:606] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:58.258596 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:58.259639 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:58.543687 23972 raft_consensus.cc:993] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:58.544734 23972 raft_consensus.cc:1081] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:58.926638 24090 raft_consensus.cc:993] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:58.927682 24087 raft_consensus.cc:1081] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:58.929430 23450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:06:58.930068 23450 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.930505 23450 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:58.939622 23450 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:58.948359 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:58.949038 22748 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:58.952834 22896 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43998
I20260709 13:06:58.953370 22896 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.953929 22896 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:58.960301 24076 raft_consensus.cc:993] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:58.960738 24076 raft_consensus.cc:1081] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:06:58.965740 23450 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:58.968935 22748 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:06:58.970023 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:58.970824 24014 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:58.971292 24014 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:58.972030 24014 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:58.975208 24077 raft_consensus.cc:993] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:58.975639 24077 raft_consensus.cc:1081] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:58.977217 22896 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:58.981904 22896 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:58.984783 24079 raft_consensus.cc:993] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:58.985227 24079 raft_consensus.cc:1081] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:06:58.986105 22905 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f881fed251c4393ac053086bd572549"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:06:58.986598 22905 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:58.986907 22905 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:58.989431 22653 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 3 to 4, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.995153 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:58.995662 22756 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:59.000597 22905 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.002733 22905 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:59.006579 22756 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:06:59.010593 22896 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:06:59.011246 22756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:06:59.011731 22756 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:59.012046 22756 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.013489 24081 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:59.014590 24081 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:06:59.015738 24081 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:06:59.016657 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 155099us
I20260709 13:06:59.016999 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:59.017601 22748 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:59.016855 24081 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.017902 22748 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:59.018543 24081 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:06:59.018621 22748 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.019678 22748 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.021750 22871 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:59.022444 24065 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:06:59.022922 24065 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:59.023725 24065 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.025691 22748 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:06:59.026983 22871 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:59.027594 24084 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.028064 24084 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:59.028642 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:59.029214 22896 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:59.028849 24084 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.029564 22896 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:59.030535 22896 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:59.032018 22756 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.032922 22753 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:59.033931 22896 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:59.041961 24089 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:06:59.047019 24086 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:59.047415 24086 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:59.048249 24086 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.049584 24086 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.053367 22756 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:59.056959 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:59.057682 22825 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.062958 22896 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:06:59.064044 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:59.072022 22825 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:59.075398 22901 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:59.079006 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:59.079809 24049 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.080237 24049 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:59.081018 24049 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.086333 22829 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:59.089938 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:59.097082 22756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38488
I20260709 13:06:59.097654 22756 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:59.097985 22756 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.098706 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:59.099364 22830 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.107563 22756 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.109385 22756 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:59.110685 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:59.097000 22652 catalog_manager.cc:5697] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.111330 22901 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.120901 22901 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:59.121902 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:06:59.124075 24049 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.124688 24049 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:59.125517 24049 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.128646 22654 catalog_manager.cc:5697] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 3, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.136199 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:06:59.137068 22825 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:59.137461 22825 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:59.138175 22825 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:06:59.139506 22825 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.149550 22829 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:06:59.132208 22654 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 2, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.153223 22830 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:06:59.166612 22825 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:06:59.176774 22652 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.260780 22829 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:59.267060 22896 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:59.267752 22794 consensus_peers.cc:597] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 27b79b7fd4bf4e90a1a7f357a253688d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:59.268546 24081 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.269721 22756 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:59.270913 24081 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.276171 23931 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.281574 22896 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.283252 22756 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.292796 22829 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:59.296187 22652 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.312120 22753 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:59.316174 22901 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:59.317849 23931 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.320011 24067 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:59.322762 22794 consensus_peers.cc:597] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3833f41661774b918ddf80499fd85f2f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:59.335098 22753 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.335776 22901 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.335798 24081 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.351321 22652 catalog_manager.cc:5697] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.352239 22756 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:06:59.353605 23957 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:59.368858 22829 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:59.386914 22896 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:06:59.389441 23957 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:59.394629 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:59.405808 22753 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:06:59.407441 22794 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82d0d73de20d49d3a64fef8b052288e5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:59.408146 22901 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:06:59.409493 23972 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.410248 24067 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.417986 23972 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.419667 22756 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.423596 22829 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.425093 24084 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.428273 22896 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.435878 22753 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.441375 22652 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:06:59.445585 24086 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:06:59.466180 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 1879799f5d0c462c9d2c95bd8a88862e: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:06:59.514387 22756 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:59.516322 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:59.518041 24084 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.538339 22829 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:06:59.539817 24084 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.556945 22896 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.558354 24075 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.576514 22829 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.578025 24049 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.650763 22756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:06:59.651389 22756 raft_consensus.cc:606] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:06:59.653918 22829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:06:59.654428 22829 raft_consensus.cc:606] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:06:59.657335 22901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:06:59.657953 22901 raft_consensus.cc:606] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:06:59.660820 22756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:06:59.661386 22756 raft_consensus.cc:606] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:06:59.662657 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:06:59.663702 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:59.665920 24075 raft_consensus.cc:993] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:06:59.666379 24048 raft_consensus.cc:1081] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Signalling peer 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:06:59.667678 22829 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:43624
I20260709 13:06:59.668002 22829 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:59.668217 22829 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.672389 22829 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:59.674239 22829 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:06:59.674291 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:59.674849 22756 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:59.674751 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:06:59.675194 22756 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:06:59.675391 23449 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.675863 22756 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:59.676582 22756 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.683337 22756 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:06:59.684012 22901 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.684852 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:06:59.685499 23972 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.685925 23972 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:06:59.686681 23972 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:59.688890 24048 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.688918 22830 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:06:59.691772 23449 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:06:59.693182 24048 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:59.712141 22654 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 1 to 2, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:59.746661 24124 ts_tablet_manager.cc:933] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:59.760751 24124 tablet_copy_client.cc:323] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:59.762264 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 3833f41661774b918ddf80499fd85f2f from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:59.762768 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 3833f41661774b918ddf80499fd85f2f from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-3833f41661774b918ddf80499fd85f2f
I20260709 13:06:59.769766 22841 tablet_copy_source_session.cc:215] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:59.772823 24124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3833f41661774b918ddf80499fd85f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:59.787251 24124 tablet_copy_client.cc:806] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:59.787814 24124 tablet_copy_client.cc:670] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:59.791035 24124 tablet_copy_client.cc:538] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:59.798820 24124 tablet_bootstrap.cc:492] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
W20260709 13:06:59.809674 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08002cf440 after lost signal to thread 21632
W20260709 13:06:59.810722 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08002cf1a0 after lost signal to thread 22688
W20260709 13:06:59.811862 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08002ff440 after lost signal to thread 22780
W20260709 13:06:59.812903 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08003478c0 after lost signal to thread 22855
W20260709 13:06:59.818370 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08003651e0 after lost signal to thread 22929
W20260709 13:06:59.819761 23475 debug-util.cc:398] Leaking SignalData structure 0x7b0800334640 after lost signal to thread 23478
I20260709 13:06:59.823994 24124 tablet_bootstrap.cc:492] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:06:59.824822 24124 tablet_bootstrap.cc:492] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:06:59.825448 24124 ts_tablet_manager.cc:1403] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.005s
I20260709 13:06:59.828153 24124 raft_consensus.cc:359] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:59.828924 24124 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:06:59.829557 24124 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:06:59.834710 24124 ts_tablet_manager.cc:1434] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:06:59.836584 24084 raft_consensus.cc:993] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:59.837599 24065 raft_consensus.cc:1081] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:06:59.837957 22841 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-3833f41661774b918ddf80499fd85f2f received from {username='slave'} at 127.0.0.1:47110
I20260709 13:06:59.838375 22841 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-3833f41661774b918ddf80499fd85f2f on tablet 3833f41661774b918ddf80499fd85f2f with peer c45e1805f049467697a7358fb566de2f
I20260709 13:06:59.840410 23453 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:06:59.840903 23453 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:59.841217 23453 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.851886 23453 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:59.858985 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:06:59.859630 22756 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.881456 24124 ts_tablet_manager.cc:933] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:59.882799 23453 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:06:59.884922 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:06:59.885721 22901 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:59.886102 22901 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:06:59.887112 22901 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:59.900961 22901 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:59.921243 24127 ts_tablet_manager.cc:933] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:06:59.922566 24124 tablet_copy_client.cc:323] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:59.923102 22756 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:59.924152 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:06:59.924933 24083 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:59.925374 24083 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:06:59.925971 22901 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:06:59.926172 24083 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:06:59.927498 24127 tablet_copy_client.cc:323] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:06:59.930003 22841 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 82d0d73de20d49d3a64fef8b052288e5 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:59.930537 22841 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 82d0d73de20d49d3a64fef8b052288e5 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-82d0d73de20d49d3a64fef8b052288e5
I20260709 13:06:59.938107 22840 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 27b79b7fd4bf4e90a1a7f357a253688d from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:06:59.938455 22841 tablet_copy_source_session.cc:215] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:59.938699 22840 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 27b79b7fd4bf4e90a1a7f357a253688d from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-27b79b7fd4bf4e90a1a7f357a253688d
I20260709 13:06:59.942402 24124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82d0d73de20d49d3a64fef8b052288e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:59.945390 22840 tablet_copy_source_session.cc:215] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:59.948396 24127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b79b7fd4bf4e90a1a7f357a253688d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:59.960736 22654 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.974464 24127 tablet_copy_client.cc:806] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:59.975030 24127 tablet_copy_client.cc:670] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:59.978581 24127 tablet_copy_client.cc:538] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:59.984849 24124 tablet_copy_client.cc:806] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:59.987171 24124 tablet_copy_client.cc:670] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:59.993614 24124 tablet_copy_client.cc:538] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:00.003123 24124 tablet_bootstrap.cc:492] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:07:00.005034 24127 tablet_bootstrap.cc:492] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:07:00.031550 24127 tablet_bootstrap.cc:492] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.032817 24127 tablet_bootstrap.cc:492] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:00.033593 24127 ts_tablet_manager.cc:1403] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.029s	user 0.027s	sys 0.001s
I20260709 13:07:00.036537 24127 raft_consensus.cc:359] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:00.037446 24127 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:00.038204 24127 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:00.040885 24127 ts_tablet_manager.cc:1434] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:00.044831 22840 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-27b79b7fd4bf4e90a1a7f357a253688d received from {username='slave'} at 127.0.0.1:47110
I20260709 13:07:00.045210 22840 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-27b79b7fd4bf4e90a1a7f357a253688d on tablet 27b79b7fd4bf4e90a1a7f357a253688d with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:00.065996 24124 tablet_bootstrap.cc:492] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:00.066931 24124 tablet_bootstrap.cc:492] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:00.067587 24124 ts_tablet_manager.cc:1403] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.015s
I20260709 13:07:00.070837 24124 raft_consensus.cc:359] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:00.071708 24124 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:00.072821 24124 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:00.075845 24124 ts_tablet_manager.cc:1434] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:00.115208 22840 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-82d0d73de20d49d3a64fef8b052288e5 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:07:00.116616 22840 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-82d0d73de20d49d3a64fef8b052288e5 on tablet 82d0d73de20d49d3a64fef8b052288e5 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:00.174091 23452 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:00.186172 24081 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:00.226925 23931 raft_consensus.cc:993] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:00.228273 23972 raft_consensus.cc:1081] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:00.232034 22753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38488
I20260709 13:07:00.232738 22753 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:00.233198 22753 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.238833 24048 raft_consensus.cc:993] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:00.239473 24048 raft_consensus.cc:1081] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:00.242230 22896 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:07:00.242888 22896 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:00.243302 22896 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.243929 22753 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:00.254240 23450 raft_consensus.cc:1217] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:00.270746 22896 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:00.293545 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:00.294232 22829 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:00.294587 22829 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:00.295161 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:00.297991 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:00.327355 22896 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:00.345263 22829 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:00.347992 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:00.298731 22903 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.354107 22756 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:00.294349 22753 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:00.357944 22756 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:00.361814 22825 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.378000 22825 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:00.393000 22753 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:00.396178 22829 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.401228 22756 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:00.439574 22756 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:00.462149 22903 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:00.487910 22756 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
W20260709 13:07:00.467458 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 68876.1us
I20260709 13:07:00.494166 22829 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:00.509243 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:00.510608 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 88106.7us
I20260709 13:07:00.513173 24086 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:00.513703 24086 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:00.514773 24086 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:00.520516 24140 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:00.537209 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:00.542510 22752 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:00.551574 22903 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:00.555743 24138 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:00.564851 22829 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:00.563669 24083 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:07:00.571923 24083 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:00.573449 23452 raft_consensus.cc:1217] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:00.585500 24138 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:00.607095 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48237.1us
I20260709 13:07:00.616734 23449 raft_consensus.cc:1217] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:00.625593 24138 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:00.673684 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:00.799612 22651 catalog_manager.cc:5697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 2, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:00.842144 22652 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:00.864357 24149 raft_consensus.cc:1064] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:00.867115 24149 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:00.911396 24146 raft_consensus.cc:1064] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:00.914270 24146 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:00.939088 23452 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:00.940284 22903 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:00.942282 24140 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:00.943984 24147 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:00.944582 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:00.981918 22754 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:00.984041 23449 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:00.984236 22901 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:00.986101 24140 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:00.992956 24140 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:00.994217 24149 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:00.996676 23972 raft_consensus.cc:1064] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:00.999177 23972 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:01.016769 24158 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.027252 24149 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.028122 22748 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.038956 23452 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.043047 22652 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:01.054205 22653 catalog_manager.cc:5697] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.056517 22901 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.054255 22903 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.056310 22749 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:01.066927 24146 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:01.094420 23452 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:01.094817 22749 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:01.095911 22903 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:01.103071 24149 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:01.104552 24146 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:01.106575 24149 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:01.103132 22754 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.109165 22825 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:01.111260 24042 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:01.121455 22825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:01.122058 22825 raft_consensus.cc:606] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:01.130870 22901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:07:01.131351 22901 raft_consensus.cc:606] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:01.132071 22748 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:01.133589 24042 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:01.134536 23449 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.145731 22756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:07:01.146275 22756 raft_consensus.cc:606] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:01.149034 22826 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:01.157043 22904 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:01.159349 22904 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:01.161528 23972 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:01.167994 22825 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:01.169392 23452 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.167137 23972 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.184602 24138 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:01.186064 24138 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:01.187157 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:01.200666 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:01.224825 22652 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.228310 24158 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.235317 22903 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.249089 23453 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:01.257694 22904 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.262995 22749 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.272235 23972 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:01.286643 23453 raft_consensus.cc:2955] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.311347 22653 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.324772 22825 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:01.330971 23449 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:01.333987 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 27b79b7fd4bf4e90a1a7f357a253688d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:01.334918 22754 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:01.337944 24149 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:01.343856 24134 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:01.348059 23972 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.355480 24065 raft_consensus.cc:993] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:01.355988 24065 raft_consensus.cc:1081] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:01.357414 22749 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:07:01.367182 22749 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:01.367626 22749 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:01.367422 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 27b79b7fd4bf4e90a1a7f357a253688d with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:07:01.363590 22752 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.371580 23449 raft_consensus.cc:2955] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.396286 24181 tablet_replica.cc:333] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:07:01.397495 24181 raft_consensus.cc:2243] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.398378 24181 raft_consensus.cc:2272] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.407481 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 82d0d73de20d49d3a64fef8b052288e5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:01.408542 24181 ts_tablet_manager.cc:1916] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:01.409257 24048 raft_consensus.cc:993] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:01.409798 24048 raft_consensus.cc:1081] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:01.409659 22653 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.411454 22749 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:01.411963 22903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:07:01.412490 22903 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:01.412830 22903 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.415923 22749 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:01.416301 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:01.416816 22825 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:01.419096 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:01.419574 22904 raft_consensus.cc:3055] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:01.419852 22904 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:01.420475 22904 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:01.423022 22825 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:01.424177 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:01.426612 24049 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:01.427045 24049 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:01.427855 24049 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:01.430827 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 82d0d73de20d49d3a64fef8b052288e5 with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:07:01.430981 22904 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:01.460988 24158 raft_consensus.cc:993] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:01.461615 24158 raft_consensus.cc:1081] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:01.471828 22749 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38488
I20260709 13:07:01.472429 22749 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:01.472832 22749 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.490301 24182 tablet_replica.cc:333] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:07:01.495875 22904 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:01.496786 24182 raft_consensus.cc:2243] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.490718 22903 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:01.506130 24182 raft_consensus.cc:2272] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.507553 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:01.507874 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:01.508102 22752 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:01.508384 22825 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.508407 22752 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:01.508826 22749 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:01.494819 24181 ts_tablet_manager.cc:1929] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:01.509656 24182 ts_tablet_manager.cc:1916] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:01.509255 22752 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:01.510056 24181 log.cc:1199] T 27b79b7fd4bf4e90a1a7f357a253688d P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/27b79b7fd4bf4e90a1a7f357a253688d
I20260709 13:07:01.510586 22752 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.512801 22749 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:01.513876 22903 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:01.514529 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 27b79b7fd4bf4e90a1a7f357a253688d (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:01.514879 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:01.515446 23449 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.517202 22825 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:01.520675 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:01.522300 24089 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:01.522735 24089 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:01.523487 23449 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:01.523533 24089 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:01.525136 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:01.525475 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:01.525956 22825 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:01.526417 22825 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:01.527333 22825 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:01.528481 22825 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:01.525961 24075 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:01.523734 22752 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:01.539228 24075 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:01.540239 24075 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:01.548228 22830 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:01.555881 22651 catalog_manager.cc:5697] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:01.559892 23449 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:01.560793 22825 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:01.562215 24158 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:01.570600 22752 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:01.572124 24158 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:01.573449 22653 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 3, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:01.582744 24134 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.585009 24182 ts_tablet_manager.cc:1929] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:01.585590 24182 log.cc:1199] T 82d0d73de20d49d3a64fef8b052288e5 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/82d0d73de20d49d3a64fef8b052288e5
I20260709 13:07:01.588806 22749 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.591203 23449 raft_consensus.cc:2955] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.607568 22653 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 3, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 3 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:01.610178 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 82d0d73de20d49d3a64fef8b052288e5 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:01.611979 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3833f41661774b918ddf80499fd85f2f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:01.613245 22652 catalog_manager.cc:5697] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 1 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:01.624286 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 3833f41661774b918ddf80499fd85f2f with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43992
I20260709 13:07:01.625176 24182 tablet_replica.cc:333] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:07:01.626338 24182 raft_consensus.cc:2243] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:01.627012 24182 raft_consensus.cc:2272] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:01.630873 24182 ts_tablet_manager.cc:1916] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:01.652222 24182 ts_tablet_manager.cc:1929] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:01.652618 24182 log.cc:1199] T 3833f41661774b918ddf80499fd85f2f P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/3833f41661774b918ddf80499fd85f2f
I20260709 13:07:01.654420 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 3833f41661774b918ddf80499fd85f2f (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:01.895241 22830 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:01.896836 24075 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:01.916347 22903 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:01.918111 24048 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:02.019622 22825 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:02.023381 24143 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:02.042212 23449 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:02.043658 24143 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:02.090381 22830 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:02.091873 24086 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:02.206425 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:02.931778 22752 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:07:03.008744 22868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.001s	sys 0.023s
W20260709 13:07:03.009261 22868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.002s	sys 0.024s
I20260709 13:07:03.021286 24196 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:07:03.114421 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:03.190107 22903 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:07:03.226882 22868 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac8ca58483824d0bbd1e56d06088b4da. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.228346 22752 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:03.233534 22829 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:03.234889 24042 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:03.236406 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f881fed251c4393ac053086bd572549"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:07:03.237007 22903 raft_consensus.cc:606] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:03.242014 22752 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:07:03.242738 22752 raft_consensus.cc:606] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:07:03.241185 24196 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:07:03.245824 22752 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:07:03.246424 22752 raft_consensus.cc:606] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:03.249871 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:07:03.250370 22903 raft_consensus.cc:606] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:03.251705 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:07:03.260933 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:03.260475 24224 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.267040 22829 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.276070 22903 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:03.278851 22653 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.287918 22756 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.291911 22829 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:07:03.292639 22868 consensus_peers.cc:597] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41fb40751eb74da580d83c9bb69ae58a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.293402 24196 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:03.296856 22752 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:03.298532 24224 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:03.324574 24196 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.328724 22825 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.347033 22752 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:03.347383 22756 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.350986 22653 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.365424 22903 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:07:03.365738 22719 consensus_peers.cc:597] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 264d1889b1f144c5897b33c834b1beef. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.369342 22825 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:07:03.374691 24203 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:03.378556 24206 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:03.392819 24206 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.396836 22826 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.406591 22903 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.415510 22901 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:03.417656 22651 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.434439 24196 raft_consensus.cc:993] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:07:03.434911 24196 raft_consensus.cc:1081] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Signalling peer 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:07:03.436623 22826 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f881fed251c4393ac053086bd572549"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:07:03.437177 22826 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:03.437573 22826 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:03.439252 22868 consensus_peers.cc:597] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db154be77d6e4b5388f9a3622f91a4ba. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.440860 22829 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:03.441989 22756 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:03.443231 24196 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:03.444172 24042 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:03.447450 24201 raft_consensus.cc:993] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:03.447981 24201 raft_consensus.cc:1081] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:03.453148 22826 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:03.455572 22826 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:03.457103 22901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:07:03.457391 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:03.458041 22754 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:03.457122 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:03.457655 22901 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:03.463341 22903 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:03.463459 22901 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:03.464000 22903 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:03.464193 22754 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:03.464890 22903 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:03.465448 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:03.466126 22903 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:03.466172 24221 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:03.466802 24221 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:03.467510 24221 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:03.472218 22903 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:03.473172 24196 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.477500 22901 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:03.479346 22653 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 2 to 3, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.480531 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:03.483455 22754 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:03.483831 22754 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:03.484654 22754 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:03.485863 22754 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:03.498668 22754 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:03.481717 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:03.500722 22826 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:03.481029 22901 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:03.510468 22826 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:03.514132 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:03.514156 24143 raft_consensus.cc:993] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:07:03.515053 24204 raft_consensus.cc:1081] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:07:03.515309 24195 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:03.517740 24195 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:03.518553 24195 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:03.519294 23450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38074
I20260709 13:07:03.519778 23450 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:03.520080 23450 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:03.515720 22754 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.537878 23450 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:03.541388 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:03.542131 22756 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:03.542548 22756 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:03.540337 22826 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:03.544863 22756 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:03.550386 22756 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:03.553702 24042 raft_consensus.cc:993] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:07:03.554188 24196 raft_consensus.cc:1081] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:07:03.555723 23450 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:03.556293 22829 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:07:03.556795 22829 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:03.557124 22829 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:03.561885 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:03.562371 22831 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:03.567406 22829 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:03.550042 22653 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 2, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.569967 22829 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:03.571007 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:03.581607 22903 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:03.581918 22903 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:03.582597 22903 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:03.594465 22903 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:03.576840 22756 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:03.602067 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:03.602747 24144 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:03.603206 24144 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:03.604110 24144 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:03.607260 22651 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 3, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:03.641934 22903 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:03.578573 22831 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:03.574822 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:03.648108 23450 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:03.648135 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:03.653028 24210 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:03.653590 24210 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:03.654476 24210 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:03.658331 23450 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:03.666041 22653 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 3 to 4, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:03.674582 22651 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:03.676169 21634 debug-util.cc:398] Leaking SignalData structure 0x7b080000f0c0 after lost signal to thread 22868
I20260709 13:07:03.699052 24248 ts_tablet_manager.cc:933] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:03.700804 24248 tablet_copy_client.cc:323] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:07:03.702577 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 41fb40751eb74da580d83c9bb69ae58a from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:07:03.703126 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 41fb40751eb74da580d83c9bb69ae58a from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-41fb40751eb74da580d83c9bb69ae58a
I20260709 13:07:03.712620 22914 tablet_copy_source_session.cc:215] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:03.716187 24248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41fb40751eb74da580d83c9bb69ae58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:03.732074 24248 tablet_copy_client.cc:806] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:03.732606 24248 tablet_copy_client.cc:670] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:03.737054 24248 tablet_copy_client.cc:538] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:03.749485 24248 tablet_bootstrap.cc:492] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
W20260709 13:07:03.763619 22777 debug-util.cc:398] Leaking SignalData structure 0x7b08000aafc0 after lost signal to thread 21632
W20260709 13:07:03.764658 22777 debug-util.cc:398] Leaking SignalData structure 0x7b08001e9a40 after lost signal to thread 22688
W20260709 13:07:03.765897 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800352500 after lost signal to thread 22780
W20260709 13:07:03.766995 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800377aa0 after lost signal to thread 22855
W20260709 13:07:03.768055 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800317800 after lost signal to thread 22929
W20260709 13:07:03.769389 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800378800 after lost signal to thread 23478
W20260709 13:07:03.769871 22777 debug-util.cc:398] Leaking SignalData structure 0x7b080031b560 after lost signal to thread 24153
W20260709 13:07:03.770337 22777 debug-util.cc:398] Leaking SignalData structure 0x7b08003786e0 after lost signal to thread 24186
W20260709 13:07:03.770608 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800378a20 after lost signal to thread 24189
I20260709 13:07:03.787477 24250 ts_tablet_manager.cc:933] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:03.797040 24248 tablet_bootstrap.cc:492] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.798483 24248 tablet_bootstrap.cc:492] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:03.799347 24248 ts_tablet_manager.cc:1403] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.051s	user 0.023s	sys 0.004s
I20260709 13:07:03.802835 24250 tablet_copy_client.cc:323] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:07:03.802830 24248 raft_consensus.cc:359] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:03.804212 24248 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:03.804613 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet ac8ca58483824d0bbd1e56d06088b4da from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:07:03.805174 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet ac8ca58483824d0bbd1e56d06088b4da from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-ac8ca58483824d0bbd1e56d06088b4da
I20260709 13:07:03.804893 24248 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:03.808342 24248 ts_tablet_manager.cc:1434] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:07:03.811712 22915 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-41fb40751eb74da580d83c9bb69ae58a received from {username='slave'} at 127.0.0.1:42490
I20260709 13:07:03.812196 22915 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-41fb40751eb74da580d83c9bb69ae58a on tablet 41fb40751eb74da580d83c9bb69ae58a with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:03.822381 24248 ts_tablet_manager.cc:933] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:03.837916 22914 tablet_copy_source_session.cc:215] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:03.869124 24250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac8ca58483824d0bbd1e56d06088b4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:03.912056 24253 ts_tablet_manager.cc:933] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:03.920226 24248 tablet_copy_client.cc:323] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:07:03.924731 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet db154be77d6e4b5388f9a3622f91a4ba from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:07:03.925264 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet db154be77d6e4b5388f9a3622f91a4ba from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-db154be77d6e4b5388f9a3622f91a4ba
I20260709 13:07:03.925837 24253 tablet_copy_client.cc:323] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.65:37213
I20260709 13:07:03.938455 22765 tablet_copy_service.cc:140] P 6a5191aaff76469bbf6c4735dc9e6509: Received BeginTabletCopySession request for tablet 264d1889b1f144c5897b33c834b1beef from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:56780)
I20260709 13:07:03.938982 22765 tablet_copy_service.cc:161] P 6a5191aaff76469bbf6c4735dc9e6509: Beginning new tablet copy session on tablet 264d1889b1f144c5897b33c834b1beef from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:56780: session id = c45e1805f049467697a7358fb566de2f-264d1889b1f144c5897b33c834b1beef
I20260709 13:07:03.952118 24250 tablet_copy_client.cc:806] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:03.952705 24250 tablet_copy_client.cc:670] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:03.958282 24250 tablet_copy_client.cc:538] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:03.962456 22765 tablet_copy_source_session.cc:215] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:03.936702 22914 tablet_copy_source_session.cc:215] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:03.966656 24248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db154be77d6e4b5388f9a3622f91a4ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:03.967344 24253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 264d1889b1f144c5897b33c834b1beef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:03.994505 22754 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:03.996007 24144 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:04.019057 22756 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:04.020665 24042 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:04.035346 22752 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:04.036702 24210 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:04.019547 22905 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:04.041637 24210 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:04.048382 22831 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:04.052500 24196 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:04.061874 24250 tablet_bootstrap.cc:492] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
W20260709 13:07:04.164582 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 59933.1us
I20260709 13:07:04.262009 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:04.252944 24253 tablet_copy_client.cc:806] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
W20260709 13:07:04.357422 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 172573us
I20260709 13:07:04.397619 22905 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:04.294512 24253 tablet_copy_client.cc:670] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:04.408313 23451 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:04.441995 24248 tablet_copy_client.cc:806] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:04.442590 24248 tablet_copy_client.cc:670] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
W20260709 13:07:04.440588 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 57091.2us
W20260709 13:07:04.470815 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 56509.4us
I20260709 13:07:04.471776 23451 raft_consensus.cc:1217] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:07:04.481609 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 317144us
W20260709 13:07:04.566046 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 90409.3us
W20260709 13:07:04.598107 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 120236us
W20260709 13:07:04.598987 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 199477us
I20260709 13:07:04.624886 24253 tablet_copy_client.cc:538] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:04.678094 22831 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:07:04.679742 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 43646.4us
W20260709 13:07:04.765926 24258 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:07:04.698593 24283 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:07:04.761546 24248 tablet_copy_client.cc:538] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:07:04.775810 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 73051.3us
I20260709 13:07:04.813243 24253 tablet_bootstrap.cc:492] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:07:04.797714 24287 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 13:07:04.867362 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 58241.5us
I20260709 13:07:04.878044 24250 tablet_bootstrap.cc:492] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.930610 24268 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:07:04.973966 24250 tablet_bootstrap.cc:492] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:04.975000 24250 ts_tablet_manager.cc:1403] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.963s	user 0.027s	sys 0.023s
I20260709 13:07:04.980996 24250 raft_consensus.cc:359] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:04.983551 24250 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:04.984390 24250 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:04.999146 24248 tablet_bootstrap.cc:492] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
W20260709 13:07:05.025538 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 223406us
I20260709 13:07:05.054037 23447 raft_consensus.cc:1217] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:05.097663 24276 raft_consensus.cc:1064] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:05.115147 24276 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:05.091868 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:05.228950 24250 ts_tablet_manager.cc:1434] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.254s	user 0.009s	sys 0.002s
W20260709 13:07:05.243093 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 61468.9us
I20260709 13:07:05.260538 22914 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-ac8ca58483824d0bbd1e56d06088b4da received from {username='slave'} at 127.0.0.1:42490
I20260709 13:07:05.260994 22914 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-ac8ca58483824d0bbd1e56d06088b4da on tablet ac8ca58483824d0bbd1e56d06088b4da with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:05.278209 22748 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:05.282222 22829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:05.282812 22829 raft_consensus.cc:606] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:05.297713 22826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84ea43361e534fbebad704327c9f6367"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:05.298246 22826 raft_consensus.cc:606] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:05.311189 24253 tablet_bootstrap.cc:492] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.312109 24253 tablet_bootstrap.cc:492] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:05.312800 24253 ts_tablet_manager.cc:1403] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.500s	user 0.046s	sys 0.034s
I20260709 13:07:05.316803 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:07:05.317384 22903 raft_consensus.cc:606] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:05.318625 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:05.329561 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:05.331666 23444 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:05.332294 24253 raft_consensus.cc:359] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:05.333072 24253 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:05.333353 22829 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:05.333704 24253 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:05.336030 24253 ts_tablet_manager.cc:1434] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.000s
I20260709 13:07:05.343554 22765 tablet_copy_service.cc:342] P 6a5191aaff76469bbf6c4735dc9e6509: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-264d1889b1f144c5897b33c834b1beef received from {username='slave'} at 127.0.0.1:56780
I20260709 13:07:05.344282 22765 tablet_copy_service.cc:434] P 6a5191aaff76469bbf6c4735dc9e6509: ending tablet copy session c45e1805f049467697a7358fb566de2f-264d1889b1f144c5897b33c834b1beef on tablet 264d1889b1f144c5897b33c834b1beef with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:05.336127 24295 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:07:05.359236 24276 raft_consensus.cc:993] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:05.359679 24276 raft_consensus.cc:1081] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:05.382248 22748 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:07:05.382759 22748 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:05.383092 22748 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.391139 24288 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:05.414997 22748 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:05.413190 24284 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:07:05.425900 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:05.426416 22903 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:05.426671 22903 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:05.427351 22903 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:05.428426 22903 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.420996 22748 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:05.421865 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:05.437525 23444 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.444590 24248 tablet_bootstrap.cc:492] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.459750 24288 raft_consensus.cc:1064] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:05.468010 22903 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:05.470137 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:05.471159 24307 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:05.471832 24307 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:05.472720 24307 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:05.492707 24248 tablet_bootstrap.cc:492] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:05.473580 23444 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:05.493497 24248 ts_tablet_manager.cc:1403] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.495s	user 0.024s	sys 0.038s
I20260709 13:07:05.506017 24283 raft_consensus.cc:993] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:05.508002 24283 raft_consensus.cc:1081] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:05.510303 22756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38488
I20260709 13:07:05.510793 22756 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:05.511121 22756 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.511761 24248 raft_consensus.cc:359] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:05.512528 24248 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:05.519593 24288 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:05.521857 22756 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:05.523545 22756 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:05.521097 22654 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:05.529934 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:05.530565 23447 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.513120 24248 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:05.537765 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:05.542295 22829 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:05.542588 22829 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:05.553740 23447 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:05.554730 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:05.555665 24307 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:05.541731 24248 ts_tablet_manager.cc:1434] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 13:07:05.556193 24307 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:05.558768 24307 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:05.563138 22829 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:05.564447 22829 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.568630 22914 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-db154be77d6e4b5388f9a3622f91a4ba received from {username='slave'} at 127.0.0.1:42490
I20260709 13:07:05.569087 22914 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-db154be77d6e4b5388f9a3622f91a4ba on tablet db154be77d6e4b5388f9a3622f91a4ba with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:05.573547 24196 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.591631 22829 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:05.590507 23450 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.614889 23444 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:05.618240 24278 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:05.620949 22752 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.618474 22825 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:05.625447 24278 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:05.628134 22825 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.664417 24269 raft_consensus.cc:993] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:05.665000 22752 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:05.665019 24304 raft_consensus.cc:1081] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:05.667892 22903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84ea43361e534fbebad704327c9f6367"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43998
I20260709 13:07:05.668380 22903 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:05.668694 22903 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.671430 24288 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:05.678095 22903 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:05.683012 22903 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:05.683883 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:05.685218 22831 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:05.685529 22831 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:05.686184 22831 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:05.687295 22831 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.683482 22651 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:05.698994 22831 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:05.700197 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:05.701154 24262 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:05.701642 24262 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:05.702456 24262 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:05.704334 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:05.704960 22748 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:05.713289 23450 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.706872 24296 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.734553 22748 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:05.764572 22756 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.890735 22831 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.894996 23450 raft_consensus.cc:1217] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:07:05.895273 23444 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:05.900655 24049 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:05.914767 22903 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:05.917861 24291 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:05.939103 23450 raft_consensus.cc:1217] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:05.941569 22904 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:05.949301 22831 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:05.952270 23450 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:05.952733 24300 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:05.957423 22653 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.961839 24288 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:05.962874 22829 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.963588 24288 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.973600 23450 raft_consensus.cc:2955] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:05.981413 23447 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:05.983260 24307 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:07:06.010893 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 41fb40751eb74da580d83c9bb69ae58a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:06.013008 22653 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.022048 22651 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.122752 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 41fb40751eb74da580d83c9bb69ae58a with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:07:06.124348 22831 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:06.137251 24307 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:06.139608 22651 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:06.158337 24349 tablet_replica.cc:333] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:07:06.161669 22748 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:06.171375 24300 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:07:06.176522 22900 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:06.190817 24349 raft_consensus.cc:2243] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.191589 24349 raft_consensus.cc:2272] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.200680 24349 ts_tablet_manager.cc:1916] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:06.210984 22748 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:06.212421 23444 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:06.213871 24296 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:06.216504 24296 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:06.220886 24300 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.222411 23452 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.229413 22752 raft_consensus.cc:2955] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.241146 22651 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 3 to 4, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.243726 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ac8ca58483824d0bbd1e56d06088b4da with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:06.244454 22831 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:06.247395 24196 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:06.258512 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet ac8ca58483824d0bbd1e56d06088b4da with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43622
I20260709 13:07:06.269893 24349 ts_tablet_manager.cc:1929] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:06.270325 24349 log.cc:1199] T 41fb40751eb74da580d83c9bb69ae58a P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/41fb40751eb74da580d83c9bb69ae58a
I20260709 13:07:06.273720 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 41fb40751eb74da580d83c9bb69ae58a (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:06.282685 24354 tablet_replica.cc:333] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:07:06.285753 24354 raft_consensus.cc:2243] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.286487 24354 raft_consensus.cc:2272] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.302728 24307 raft_consensus.cc:1064] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:06.303866 24354 ts_tablet_manager.cc:1916] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:06.305779 24307 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:06.329540 24354 ts_tablet_manager.cc:1929] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:06.330039 24354 log.cc:1199] T ac8ca58483824d0bbd1e56d06088b4da P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/ac8ca58483824d0bbd1e56d06088b4da
I20260709 13:07:06.332342 23452 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:07:06.334795 22903 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:07:06.335590 22831 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:07:06.335461 24291 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:06.336881 24337 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:06.340989 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet ac8ca58483824d0bbd1e56d06088b4da (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:06.350399 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:06.360204 24049 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:06.455323 24337 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.470444 22904 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.468763 22831 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.486939 24196 raft_consensus.cc:1064] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:06.490481 24196 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:06.497309 22653 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.502072 23452 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.520193 23444 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:06.529597 22831 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:06.531606 22752 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:06.532985 24300 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:06.541096 24284 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:06.548739 24196 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:06.558382 22756 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:06.570875 23444 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:06.573452 24291 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:06.578462 22829 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:06.586169 24337 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:06.597347 24284 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.598855 23452 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.602934 22829 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.608237 22756 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.599164 24307 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.637369 23444 raft_consensus.cc:2955] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.637955 22652 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:06.657148 22825 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.694280 22641 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 264d1889b1f144c5897b33c834b1beef with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:06.697595 22654 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 4 to 5, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.731101 22904 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:06.737787 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:06.739609 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 264d1889b1f144c5897b33c834b1beef with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43992
I20260709 13:07:06.743769 23450 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:06.748765 24278 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:06.753221 22829 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:06.755115 24363 tablet_replica.cc:333] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:07:06.756430 24363 raft_consensus.cc:2243] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.758421 24363 raft_consensus.cc:2272] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.754783 24278 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:06.769445 24363 ts_tablet_manager.cc:1916] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:06.779011 24262 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.800788 22829 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.809150 24363 ts_tablet_manager.cc:1929] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:07:06.817525 24363 log.cc:1199] T 264d1889b1f144c5897b33c834b1beef P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/264d1889b1f144c5897b33c834b1beef
I20260709 13:07:06.819238 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 264d1889b1f144c5897b33c834b1beef (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:06.878937 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet db154be77d6e4b5388f9a3622f91a4ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:06.889247 23450 raft_consensus.cc:2955] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:06.889564 22652 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 3 to 4, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 1 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:06.966410 22756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:07:06.966957 22756 raft_consensus.cc:606] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:07:06.968922 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet db154be77d6e4b5388f9a3622f91a4ba with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38476
I20260709 13:07:06.972978 24364 tablet_replica.cc:333] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:07:06.974052 24364 raft_consensus.cc:2243] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:06.974702 24364 raft_consensus.cc:2272] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:06.977664 22900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:07:06.978257 22900 raft_consensus.cc:606] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:06.982168 22756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:07:06.982705 22756 raft_consensus.cc:606] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:06.983618 24364 ts_tablet_manager.cc:1916] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:06.988209 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:06.989773 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:07.002790 24364 ts_tablet_manager.cc:1929] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:07.003607 24364 log.cc:1199] T db154be77d6e4b5388f9a3622f91a4ba P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/db154be77d6e4b5388f9a3622f91a4ba
I20260709 13:07:07.006067 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet db154be77d6e4b5388f9a3622f91a4ba (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:07.067474 24307 raft_consensus.cc:993] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:07:07.068791 24307 raft_consensus.cc:1081] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:07:07.072304 23450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38074
I20260709 13:07:07.073558 23450 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:07.073902 23450 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:07.078523 23450 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:07.080525 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:07.080772 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:07.081034 22754 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:07.081286 22829 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:07.081398 22754 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:07.082136 22754 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:07.083114 22754 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:07.086375 23450 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:07.088593 22829 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:07.089880 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:07.090611 24144 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:07.090859 22754 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:07.091094 24144 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:07.092689 24144 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:07.107802 22652 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 3, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 3 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:07.150055 24262 raft_consensus.cc:993] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:07:07.150589 24262 raft_consensus.cc:1081] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Signalling peer 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:07:07.152431 22825 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:07:07.153115 22825 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:07.153533 22825 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:07.160413 22825 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:07.162276 22825 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:07.163280 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:07.163836 22904 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:07.164104 22904 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:07.164187 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:07.164701 22754 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:07.164857 22904 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:07.165874 22904 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:07.171607 22754 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:07.171607 22904 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:07.172931 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:07.173745 24269 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:07.174237 24269 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:07.175098 24269 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:07.187070 22652 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 2 to 3, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:07.292620 24291 raft_consensus.cc:993] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:07.293160 24291 raft_consensus.cc:1081] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:07.294870 22904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43996
I20260709 13:07:07.295390 22904 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:07.295717 22904 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.307497 22904 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:07.309922 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:07.310479 22754 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:07.310760 22754 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:07.311455 22754 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:07.312518 22754 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.314200 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:07.314797 22829 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:07.319337 22754 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:07.326138 22871 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:07.326936 24284 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:07.326998 22829 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:07.327479 24284 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:07.328543 22904 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:07.328365 24284 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:07.344552 22652 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 2, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:07.648366 22903 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:07.649621 24269 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:07.670492 22756 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:07.671825 24144 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:07.683439 22825 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:07.688427 24347 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:07.720145 22752 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:07.741904 24304 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:07.786170 22829 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:07.794979 24300 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:07.852536 22754 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:07.857178 24284 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:07.990612 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:08.133178 22756 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:07:08.144888 22719 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70588fa182854c9dbc786bdd4f807fc9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.146397 22904 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:07:08.146716 22829 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:07:08.148540 24337 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:08.149668 24291 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:08.178488 24307 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.180253 22904 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.194325 22831 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:08.199254 22652 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:08.212078 22794 consensus_peers.cc:597] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 469a8bc68a8b4e62b19f382c167cf50c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.233784 22900 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:08.235316 24304 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:08.251992 24283 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.272917 22903 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.281661 22754 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:08.288283 22829 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.297717 24370 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:08.304647 22904 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:08.307547 22654 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:08.315948 22754 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.333173 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:08.336582 22868 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a28fec9b1516463db9c7feac432f9ad5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.339104 22829 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:08.340541 24300 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:08.351715 22756 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:08.355142 24300 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:08.380273 24284 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.393584 22829 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.401868 22754 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.406808 22831 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:07:08.427480 22794 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f881fed251c4393ac053086bd572549. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:08.432714 22903 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:08.433917 22754 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:08.439643 24304 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:08.444052 24304 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:08.451575 22653 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.477059 24370 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.485911 22903 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.491196 22754 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.494661 22904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:07:08.495206 22904 raft_consensus.cc:606] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:08.524652 22653 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 4, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:08.523304 22831 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f765d75643f47dd939002936724a315"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:08.536875 22831 raft_consensus.cc:606] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:08.540561 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:08.542107 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:08.599120 24399 ts_tablet_manager.cc:933] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:08.615430 24399 tablet_copy_client.cc:323] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.65:37213
I20260709 13:07:08.617869 22765 tablet_copy_service.cc:140] P 6a5191aaff76469bbf6c4735dc9e6509: Received BeginTabletCopySession request for tablet 70588fa182854c9dbc786bdd4f807fc9 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:56780)
I20260709 13:07:08.618474 22765 tablet_copy_service.cc:161] P 6a5191aaff76469bbf6c4735dc9e6509: Beginning new tablet copy session on tablet 70588fa182854c9dbc786bdd4f807fc9 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:56780: session id = c45e1805f049467697a7358fb566de2f-70588fa182854c9dbc786bdd4f807fc9
I20260709 13:07:08.634512 22765 tablet_copy_source_session.cc:215] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:08.642910 24399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70588fa182854c9dbc786bdd4f807fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:08.654227 24401 ts_tablet_manager.cc:933] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:08.668037 24401 tablet_copy_client.cc:323] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:07:08.675086 22840 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 469a8bc68a8b4e62b19f382c167cf50c from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:07:08.675585 22840 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 469a8bc68a8b4e62b19f382c167cf50c from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-469a8bc68a8b4e62b19f382c167cf50c
I20260709 13:07:08.682755 22840 tablet_copy_source_session.cc:215] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:08.686338 24401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 469a8bc68a8b4e62b19f382c167cf50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:08.704913 24399 tablet_copy_client.cc:806] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:08.705559 24399 tablet_copy_client.cc:670] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:08.710764 24399 tablet_copy_client.cc:538] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:08.733248 24370 raft_consensus.cc:993] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:08.733757 24370 raft_consensus.cc:1081] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:08.736404 22903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f765d75643f47dd939002936724a315"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43998
I20260709 13:07:08.736922 22903 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:08.737258 22903 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:08.753151 22903 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:08.758729 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:08.759265 22831 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:08.759538 22831 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:08.760224 22831 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:08.769399 22831 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:08.784843 22831 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:08.790680 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:08.797735 23452 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:08.810462 23452 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:08.811597 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:08.812418 24284 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:08.812870 24284 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:08.813660 24284 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:08.819875 24403 ts_tablet_manager.cc:933] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:08.795732 22903 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:08.847812 24403 tablet_copy_client.cc:323] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:07:08.856472 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet a28fec9b1516463db9c7feac432f9ad5 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:07:08.856978 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet a28fec9b1516463db9c7feac432f9ad5 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-a28fec9b1516463db9c7feac432f9ad5
I20260709 13:07:08.861186 24399 tablet_bootstrap.cc:492] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:07:08.862336 24401 tablet_copy_client.cc:806] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:08.862874 24401 tablet_copy_client.cc:670] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:08.872241 24401 tablet_copy_client.cc:538] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:08.874648 22914 tablet_copy_source_session.cc:215] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:08.875468 22650 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:08.880003 24403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a28fec9b1516463db9c7feac432f9ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:08.880511 24284 raft_consensus.cc:993] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:08.881006 24394 raft_consensus.cc:1081] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:08.882699 22754 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:07:08.883273 22754 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:08.883646 22754 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:08.901492 22754 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:08.903839 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:08.904368 22831 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:08.908806 24401 tablet_bootstrap.cc:492] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:07:08.912117 22831 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:08.930574 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:08.931366 24291 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:08.931859 24291 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:08.932875 24291 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:08.942266 22754 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:08.948236 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:08.948935 22903 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:08.949285 22903 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:08.950359 22903 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:08.951874 22903 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:08.954480 24407 ts_tablet_manager.cc:933] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:08.963685 24403 tablet_copy_client.cc:806] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:08.964278 24403 tablet_copy_client.cc:670] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:08.967034 22903 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:08.969821 24403 tablet_copy_client.cc:538] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:08.975221 24399 tablet_bootstrap.cc:492] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:08.976095 24399 tablet_bootstrap.cc:492] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:08.976737 24399 ts_tablet_manager.cc:1403] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.116s	user 0.027s	sys 0.012s
I20260709 13:07:08.979326 24403 tablet_bootstrap.cc:492] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:07:08.980041 24399 raft_consensus.cc:359] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:08.980904 24399 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:08.981599 24399 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:08.984063 24399 ts_tablet_manager.cc:1434] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:08.992990 22765 tablet_copy_service.cc:342] P 6a5191aaff76469bbf6c4735dc9e6509: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-70588fa182854c9dbc786bdd4f807fc9 received from {username='slave'} at 127.0.0.1:56780
I20260709 13:07:08.993424 22765 tablet_copy_service.cc:434] P 6a5191aaff76469bbf6c4735dc9e6509: ending tablet copy session c45e1805f049467697a7358fb566de2f-70588fa182854c9dbc786bdd4f807fc9 on tablet 70588fa182854c9dbc786bdd4f807fc9 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:08.995832 22653 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 3, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 3 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:09.006630 24407 tablet_copy_client.cc:323] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:07:09.008719 22840 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 1f881fed251c4393ac053086bd572549 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:07:09.009222 22840 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 1f881fed251c4393ac053086bd572549 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-1f881fed251c4393ac053086bd572549
I20260709 13:07:09.017035 22840 tablet_copy_source_session.cc:215] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:09.020588 24407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f881fed251c4393ac053086bd572549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:09.044593 24403 tablet_bootstrap.cc:492] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.052960 24403 tablet_bootstrap.cc:492] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:09.053622 24403 ts_tablet_manager.cc:1403] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.014s
I20260709 13:07:09.054502 24407 tablet_copy_client.cc:806] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:09.055065 24407 tablet_copy_client.cc:670] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:09.056448 24403 raft_consensus.cc:359] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:09.057236 24403 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:09.057881 24403 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:09.058681 24407 tablet_copy_client.cc:538] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:09.060225 24403 ts_tablet_manager.cc:1434] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:09.069751 22914 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-a28fec9b1516463db9c7feac432f9ad5 received from {username='slave'} at 127.0.0.1:42490
I20260709 13:07:09.070137 22914 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-a28fec9b1516463db9c7feac432f9ad5 on tablet a28fec9b1516463db9c7feac432f9ad5 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:09.074877 24407 tablet_bootstrap.cc:492] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:07:09.135270 23452 raft_consensus.cc:1217] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:07:09.153972 24407 tablet_bootstrap.cc:492] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.154752 24401 tablet_bootstrap.cc:492] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:09.154844 24407 tablet_bootstrap.cc:492] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:09.170598 24407 ts_tablet_manager.cc:1403] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.096s	user 0.028s	sys 0.000s
I20260709 13:07:09.171454 24401 tablet_bootstrap.cc:492] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:09.173425 24407 raft_consensus.cc:359] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:09.180543 24407 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:09.181510 24407 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:09.183820 24407 ts_tablet_manager.cc:1434] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:07:09.230559 22840 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-1f881fed251c4393ac053086bd572549 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:07:09.231026 22840 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-1f881fed251c4393ac053086bd572549 on tablet 1f881fed251c4393ac053086bd572549 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:09.245172 22831 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:09.250034 24377 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:09.257476 24401 ts_tablet_manager.cc:1403] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.273s	user 0.032s	sys 0.010s
I20260709 13:07:09.290774 24401 raft_consensus.cc:359] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:09.297482 24401 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:09.298238 24401 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:09.300673 24401 ts_tablet_manager.cc:1434] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.001s
I20260709 13:07:09.313891 22840 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-469a8bc68a8b4e62b19f382c167cf50c received from {username='slave'} at 127.0.0.1:47110
I20260709 13:07:09.314298 22840 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-469a8bc68a8b4e62b19f382c167cf50c on tablet 469a8bc68a8b4e62b19f382c167cf50c with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:09.322980 23452 raft_consensus.cc:1217] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:07:09.366869 23452 raft_consensus.cc:1217] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:07:09.401603 23452 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:09.402889 24300 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:09.425786 22903 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:09.432277 24413 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:09.466248 22831 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:09.467820 24413 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:09.507870 23452 raft_consensus.cc:1217] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:07:09.561592 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:09.702504 24291 raft_consensus.cc:1064] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:09.704970 24291 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:09.727365 23452 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:07:09.728662 22904 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:07:09.730126 22829 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:07:09.732399 24307 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:09.734071 24307 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:09.734927 24413 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:09.783988 24427 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:09.785936 22904 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:09.805075 23450 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:09.814909 22653 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:09.793515 22829 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:09.886569 24377 raft_consensus.cc:1064] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:09.888980 24377 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:09.952569 24415 raft_consensus.cc:1064] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:09.953115 24370 raft_consensus.cc:1064] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:09.954948 24415 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:09.956254 24370 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:09.974063 22748 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:09.976696 24284 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:07:10.005534 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 65243.3us
I20260709 13:07:10.011087 23450 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:10.012156 22904 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:10.012993 22829 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:10.013252 23450 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:10.013368 22754 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:10.015492 24283 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:10.018669 24414 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:10.028816 22756 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:10.030558 23452 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:10.031021 24304 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:10.031064 22903 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:10.037550 24370 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:07:10.049019 24433 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:07:10.050334 24269 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:10.051265 24300 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:10.052619 24283 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:07:10.061144 24438 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:07:10.059772 24377 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:07:10.093665 24283 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:07:10.106881 24437 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:07:10.106463 22903 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.135931 23452 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.138113 22754 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:10.140713 24414 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.142225 22756 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.146664 23450 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.147523 24284 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.155299 23447 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.160725 23444 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:10.162148 24427 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:10.178617 22753 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.180474 24307 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.208770 23444 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.208343 22903 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:10.217195 22903 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:07:10.254734 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49337us
I20260709 13:07:10.257227 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:10.260784 22641 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 70588fa182854c9dbc786bdd4f807fc9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:10.281390 22652 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:10.275359 22650 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.350005 24427 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:07:10.365408 22753 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.370995 22653 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:10.388022 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 46738.7us
I20260709 13:07:10.392534 22831 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:07:10.399439 24333 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:10.430721 22904 raft_consensus.cc:2955] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.437947 22650 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 4 to 5, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 2 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:10.513798 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 63416.5us
W20260709 13:07:10.517681 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 104889us
I20260709 13:07:10.529737 22904 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:10.536543 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet 70588fa182854c9dbc786bdd4f807fc9 with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43622
I20260709 13:07:10.555123 23447 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:10.557454 24284 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:10.558610 22753 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:10.566383 22831 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:10.561523 24284 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:10.575775 24420 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.587486 23449 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.605131 22753 raft_consensus.cc:2955] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.611099 22756 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:10.613813 24370 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:10.615183 24457 tablet_replica.cc:333] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:07:10.628738 24454 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.632928 23447 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:10.636059 22756 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.652762 24370 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:10.656126 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a28fec9b1516463db9c7feac432f9ad5 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:10.662797 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1f881fed251c4393ac053086bd572549 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:10.674386 22650 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 5 to 6, VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) evicted. New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.681780 23449 raft_consensus.cc:2955] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:07:10.714619 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 76763.5us
I20260709 13:07:10.724436 22814 tablet_service.cc:1558] Processing DeleteTablet for tablet a28fec9b1516463db9c7feac432f9ad5 with delete_type TABLET_DATA_TOMBSTONED (TS 7aad11e70ca94fce8f672db05f1c28c3 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43622
I20260709 13:07:10.729843 22653 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 5 to 6, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:07:10.736285 22638 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 1f881fed251c4393ac053086bd572549 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 5 but the committed config has opid_index of 6
I20260709 13:07:10.736325 22824 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:10.744843 23449 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:10.746373 24370 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:10.759256 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f881fed251c4393ac053086bd572549 with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43992
I20260709 13:07:10.757078 22903 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:10.788998 24459 tablet_replica.cc:333] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:07:10.790668 24304 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:10.795289 24454 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.796607 24459 raft_consensus.cc:2243] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:10.796658 23449 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.797659 24459 raft_consensus.cc:2272] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:10.801791 24459 ts_tablet_manager.cc:1916] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:10.810441 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 469a8bc68a8b4e62b19f382c167cf50c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:10.826766 24457 raft_consensus.cc:2243] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:10.828411 24457 raft_consensus.cc:2272] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:10.832307 22654 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 5 to 6, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:10.855206 22900 raft_consensus.cc:2955] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:10.860146 22739 tablet_service.cc:1558] Processing DeleteTablet for tablet 469a8bc68a8b4e62b19f382c167cf50c with delete_type TABLET_DATA_TOMBSTONED (TS 6a5191aaff76469bbf6c4735dc9e6509 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38476
I20260709 13:07:10.861160 24457 ts_tablet_manager.cc:1916] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:10.888651 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:07:10.889460 22903 raft_consensus.cc:606] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:07:10.894459 22756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:07:10.894985 22756 raft_consensus.cc:606] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:10.896521 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:10.896796 24460 tablet_replica.cc:333] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:07:10.898250 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:10.897974 24460 raft_consensus.cc:2243] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:10.902247 24460 raft_consensus.cc:2272] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:10.909624 24459 ts_tablet_manager.cc:1929] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:07:10.910178 24459 log.cc:1199] T 1f881fed251c4393ac053086bd572549 P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/1f881fed251c4393ac053086bd572549
I20260709 13:07:10.911707 24457 ts_tablet_manager.cc:1929] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:07:10.912165 24457 log.cc:1199] T 70588fa182854c9dbc786bdd4f807fc9 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/70588fa182854c9dbc786bdd4f807fc9
I20260709 13:07:10.912442 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 1f881fed251c4393ac053086bd572549 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:10.914060 24457 tablet_replica.cc:333] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:07:10.913997 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet 70588fa182854c9dbc786bdd4f807fc9 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:10.915560 24457 raft_consensus.cc:2243] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:10.916239 24460 ts_tablet_manager.cc:1916] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:10.916378 24457 raft_consensus.cc:2272] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:10.919706 24457 ts_tablet_manager.cc:1916] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:10.933420 24457 ts_tablet_manager.cc:1929] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:07:10.933768 24457 log.cc:1199] T a28fec9b1516463db9c7feac432f9ad5 P 7aad11e70ca94fce8f672db05f1c28c3: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/wals/a28fec9b1516463db9c7feac432f9ad5
I20260709 13:07:10.934526 24460 ts_tablet_manager.cc:1929] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:07:10.935086 22638 catalog_manager.cc:5028] TS 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): tablet a28fec9b1516463db9c7feac432f9ad5 (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:10.935961 24460 log.cc:1199] T 469a8bc68a8b4e62b19f382c167cf50c P 6a5191aaff76469bbf6c4735dc9e6509: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/wals/469a8bc68a8b4e62b19f382c167cf50c
I20260709 13:07:10.937779 22641 catalog_manager.cc:5028] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): tablet 469a8bc68a8b4e62b19f382c167cf50c (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:07:11.006107 24291 raft_consensus.cc:993] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: : Instructing follower 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:07:11.006615 24291 raft_consensus.cc:1081] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Signalling peer 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:07:11.010697 22831 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:43624
I20260709 13:07:11.011356 22831 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:11.011714 22831 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:11.029937 22831 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:11.032963 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:11.033607 22756 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:11.033895 22756 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:11.034639 22756 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:11.035794 22756 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:11.063056 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:11.063706 23449 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:11.076292 22831 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:11.080812 22756 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:11.081538 23449 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:11.084724 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:11.085800 24304 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:11.086241 24304 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:11.087049 24304 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:11.121748 24435 raft_consensus.cc:993] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:07:11.122283 24435 raft_consensus.cc:1081] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:07:11.117897 22654 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 2 to 3, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:11.131901 23444 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:07:11.132424 23444 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:11.132787 23444 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.165508 23444 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:11.169606 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:11.170181 22756 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.181118 22756 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:11.183634 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:11.184739 23444 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:11.186161 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:11.186728 22900 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:11.187009 22900 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:11.187680 22900 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:11.188887 22900 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:11.190860 24456 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:11.192274 24456 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:11.196614 24456 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:11.427805 22900 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:11.919210 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:12.214658 24420 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.215422 24420 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:12.239108 24420 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:12.311506 24420 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.312237 24420 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.315636 24420 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.553484 24469 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.554513 24469 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.562605 24469 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.629678 24420 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.692992 24467 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.773684 24467 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.776630 24467 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.777191 24467 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:12.777904 24467 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.780274 24467 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.694434 24466 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:12.703240 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 315) took 1247 ms. Trace:
I20260709 13:07:12.781137 24466 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.781204 22900 rpcz_store.cc:276] 0709 13:07:11.455575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.702505 (+1246930us) service_pool.cc:224] Handling call
0709 13:07:12.702807 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.702961 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.702981 (+    20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:12.703018 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:12.703046 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.703093 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.703190 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:12.783681 24466 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.784435 24420 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.787017 24420 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.813549 24469 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.820713 24471 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.821797 24471 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.822240 24470 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.838274 24466 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.838716 24474 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.838958 24466 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.838873 24470 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.840016 24474 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.820880 24469 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.841689 24466 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.842260 24466 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.842342 24474 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.842911 24474 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.842797 24466 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.843325 24469 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:12.843415 24474 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.845070 24472 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.845402 24470 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:12.845711 24472 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.845921 24474 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.847715 24472 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.848098 24471 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:12.850240 24466 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.850708 24470 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.852437 24470 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.853478 24467 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.854513 24470 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.814429 24420 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.823635 24476 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.855731 24471 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.854099 24467 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.857156 24466 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:12.856066 24476 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.857733 24466 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.859709 24476 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:12.861450 24467 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.871363 24478 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.872054 24478 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.875344 24478 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.870110 24472 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:12.882094 24472 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.883960 24472 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.855335 24420 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.885747 24420 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:12.867537 24471 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.888391 24471 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.888827 24471 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.891570 24470 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.892088 24470 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.894174 24470 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:12.894809 23419 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.462s	user 0.019s	sys 0.019s
W20260709 13:07:12.895076 23419 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.463s	user 0.019s	sys 0.019s
I20260709 13:07:12.899353 24476 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.889261 24471 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.899912 24476 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.901475 24471 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.901955 24476 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.902105 24466 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.906666 24467 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.907137 24467 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.908936 24467 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.913669 24478 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:12.914179 24478 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.924922 24476 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.925501 24476 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.923283 24466 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.917567 24467 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.916983 24478 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.927460 24476 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.927956 24467 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.924672 24472 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.929045 24481 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.929184 24472 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.929561 24481 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:12.929945 24467 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.931111 24472 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.931440 24481 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:12.927465 24466 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.943907 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 192) took 1749 ms. Trace:
I20260709 13:07:12.944739 24466 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.944372 22831 rpcz_store.cc:276] 0709 13:07:11.194246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.943001 (+1748755us) service_pool.cc:224] Handling call
0709 13:07:12.943341 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.943528 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.943564 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:12.943614 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:12.943667 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.943731 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.943849 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:12.947990 24476 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.948823 24476 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.951289 24476 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.965245 24484 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:12.970554 24484 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.966405 24472 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:12.971738 24472 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.973156 24483 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.973918 24473 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.973690 24483 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.974071 24472 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.974442 24473 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.976217 24483 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.977605 24471 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.977813 24485 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.976691 24473 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.978124 24471 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.978369 24485 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.966704 24478 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:12.979806 24478 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.980242 24471 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:12.980479 24485 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.980710 24471 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.980990 24485 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.971081 24482 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.978781 24473 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:12.987329 24485 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.965930 24466 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.988374 24466 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.981860 24478 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.985538 24479 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:12.989241 24485 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.989657 24479 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.990924 24466 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.966645 24481 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:12.991644 24479 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.991587 24481 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.987483 24473 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.987406 24482 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.993803 24475 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:12.994272 24475 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:12.995445 24482 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.995970 24481 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:12.996604 24475 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:12.981163 24471 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:12.998485 24471 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.000739 24475 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.001684 24475 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.002715 24476 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.004575 24475 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.002460 24462 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.004777 24476 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.005594 24462 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.007375 24476 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.007905 24478 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.017454 24476 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.019572 24484 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.020802 24476 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.022043 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.025064 22900 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:13.023739 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.008769 24478 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.029417 24464 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.034744 24464 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.038616 24464 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.041739 24473 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.040104 24465 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.047050 24480 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.047109 24465 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.047282 22654 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:13.041507 24458 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.047956 24480 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.039476 24456 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.045598 24462 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.046416 24468 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.048130 24485 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.041244 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.052330 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.043860 24478 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.083374 24420 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.083765 24455 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.083979 24496 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.085111 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.085459 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.085758 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.085951 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.086839 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.086923 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.087159 24471 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.087764 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.089071 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.090436 23449 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:13.090481 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.090732 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.091434 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.082888 24477 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.092523 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.093622 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.093078 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.093943 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.095058 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.095347 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.094542 24488 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.098544 24484 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.099088 24467 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.099655 24481 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.100214 24472 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.118608 24483 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.105867 22900 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:13.120772 24464 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.138438 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.119758 24490 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.100173 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.075356 24473 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.114847 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.096436 24495 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.102181 24482 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.125567 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.129253 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.124120 22750 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:13.126446 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.127887 22749 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:07:13.005228 24475 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.136409 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.048732 24458 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.129961 22755 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:13.114394 24462 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.119585 24471 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.129588 24491 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.144208 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.144593 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.103304 24456 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.145556 24497 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.120419 24420 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.124715 23444 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:07:13.134261 24484 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.150877 22897 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:07:13.122005 24468 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.136984 24467 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.137619 24476 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.120286 24485 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.225098 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.136490 24492 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.146790 24480 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.219448 22902 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:13.149765 24483 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.150166 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.157577 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.118531 24496 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.124285 24455 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.135637 24477 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.298677 22905 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:07:13.150693 24464 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.334776 24499 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.468132 22901 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:13.478623 24501 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.478698 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.479074 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.471194 24480 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.478816 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.493880 22747 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:13.240543 24479 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.320293 24489 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.460904 24502 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.365710 24471 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.157912 24472 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.464898 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.470206 24494 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.454835 24487 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.159165 24473 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.226616 24481 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.479161 24501 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.240005 24475 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.144306 24488 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.277696 24462 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.366611 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.327391 24482 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.561285 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.596079 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.596556 22901 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:13.577508 22898 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:07:13.555524 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.606969 22896 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:13.583837 24481 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.555364 24503 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.547319 24480 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.579073 24473 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.560513 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.365989 24491 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.581310 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.226320 24490 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.239882 24465 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.584496 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.585723 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.586339 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.589439 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.517289 24499 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.594858 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.596959 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.584718 24479 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.601902 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:13.602031 24456 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.602166 24484 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.602876 24458 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.603194 24420 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.603681 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.639660 24488 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.640580 24455 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.640992 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.641103 24455 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.604194 23449 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:07:13.641382 24504 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.641685 24455 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.641944 24504 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.643117 24491 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.643666 24491 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.605157 24462 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.605695 24475 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.643953 24504 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.608227 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.565964 24507 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.543838 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.589999 24489 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.617105 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.618033 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.577347 24494 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.654548 24472 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.619598 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.603695 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.304155 24495 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.605728 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.661541 24496 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.661953 24501 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.662460 24494 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.628439 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.662925 24480 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.628665 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.663899 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.664417 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.629577 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.620450 24503 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.655114 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.665637 22748 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:13.600471 24482 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.446908 24497 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.635416 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.636009 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.577987 24487 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.639632 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.644644 22754 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:13.606123 22753 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:13.647418 23450 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:07:13.634577 24465 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.626893 24509 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.686936 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.638356 24481 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:13.643766 24455 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.662765 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.633906 22904 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:13.667372 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.624029 24473 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.675312 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.674237 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.681687 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.682128 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.492015 24492 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.687913 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.644158 24491 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.579677 24502 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.687512 22751 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:13.692677 24465 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.674176 24490 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:13.699265 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 62041.4us
I20260709 13:07:13.700186 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.661551 24468 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.701649 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.680752 24477 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.699331 22752 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:13.698354 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.682636 24485 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.685158 24483 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.704089 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.704082 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.705302 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.706732 24490 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.698372 24473 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.693444 24509 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.706854 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.707248 24468 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.682480 22827 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:13.711172 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.717455 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:13.695181 24481 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.715093 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.717954 23453 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:07:13.720470 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.721292 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.721601 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.723440 24502 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.693392 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.724504 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.725134 23453 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
W20260709 13:07:13.715982 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 77962.3us
I20260709 13:07:13.725605 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.725875 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.696765 23447 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:07:13.728912 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.726153 23451 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:13.729779 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.717296 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.730508 22755 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:13.707938 24497 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.732009 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.731771 24468 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.732553 22750 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:13.733567 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.733783 24468 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.717083 24490 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.734532 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.735051 22750 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:13.735843 24490 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.735867 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.736405 24473 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.739295 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.740056 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.718274 23452 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:13.724504 24481 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:13.719703 22826 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:13.750995 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.751519 23449 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:13.654663 24507 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.727034 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:13.701854 24479 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.715283 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.753446 22753 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:13.754171 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.716290 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.744601 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:13.744881 24487 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.745048 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:13.745411 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.747602 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:13.755959 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66216.7us
I20260709 13:07:13.718276 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:13.749522 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.749374 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.724671 24486 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:13.748179 24467 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.756187 22751 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:13.773562 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:13.773293 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 55088.2us
I20260709 13:07:13.774672 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:13.774085 24471 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:13.778257 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.779700 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.775310 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.775959 24509 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.779769 22752 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:13.778155 24503 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.805737 24464 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.802955 22904 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:13.741081 24497 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:13.777673 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:13.801730 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.844067 24487 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:13.844431 24487 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.844792 24487 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:13.842628 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.805018 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:13.845153 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 67542.9us
I20260709 13:07:13.775744 24479 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:13.848138 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:07:13.848712 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 43797.4us
I20260709 13:07:13.843533 24489 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.778981 24477 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:13.806727 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.834141 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.845178 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.825490 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.854003 22750 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:13.846318 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.843133 24497 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:13.822712 23446 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:13.861644 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.862850 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.863425 22755 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:13.806165 24465 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.775674 24471 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.863930 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.864257 24471 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:13.793999 24472 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:13.762120 24486 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.776207 24473 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.841785 24499 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.857249 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:13.899150 24492 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.900847 24495 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:13.870594 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.870852 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.871097 24513 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:13.876796 24491 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.879698 24472 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.906170 24472 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:13.851707 22753 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:13.852622 24467 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:13.914280 24467 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.914754 24467 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:13.915359 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 58149.6us
I20260709 13:07:13.915668 24499 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:13.917337 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.898507 24507 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.900496 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:13.903934 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:13.904443 22754 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
I20260709 13:07:13.905581 24513 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:13.905333 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.919219 24513 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:13.914687 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.897878 24486 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:13.917481 24491 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:13.923404 24491 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.923844 24491 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.919193 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.913566 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.898300 24473 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:13.927217 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:13.855046 24477 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:13.949265 24477 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:13.949476 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 195175us
I20260709 13:07:13.947091 24515 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:13.951823 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.952211 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.952452 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.953205 22752 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:13.954360 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:13.955552 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:13.952299 24497 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:13.953872 22755 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:13.961136 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.957623 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.963349 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.958698 24488 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:13.965904 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.958119 24420 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:13.957710 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.973573 24519 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:13.976679 24519 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:13.977064 24519 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.976699 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.968515 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.978813 23452 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:13.968241 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.970350 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.979315 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.989563 24473 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:13.991645 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.993671 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:13.994246 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.996531 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:13.997814 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:13.967648 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.000886 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:14.003635 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08000e70c0 after lost signal to thread 21632
W20260709 13:07:14.004637 23475 debug-util.cc:398] Leaking SignalData structure 0x7b080009f360 after lost signal to thread 22688
W20260709 13:07:14.006088 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08002a6500 after lost signal to thread 22780
W20260709 13:07:14.007416 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08002ceda0 after lost signal to thread 22855
W20260709 13:07:14.008720 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08001a7ac0 after lost signal to thread 22929
W20260709 13:07:14.010291 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08002c1ae0 after lost signal to thread 23478
I20260709 13:07:14.022176 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.024034 24471 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.024174 24486 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.004423 24513 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.030296 22747 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.032076 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:13.998363 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:14.039901 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:14.040755 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 81509.9us
I20260709 13:07:14.041483 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:14.041733 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.042084 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:14.030242 24481 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.037391 24473 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:14.047209 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.045833 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.050153 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:14.050652 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 199507us
I20260709 13:07:14.046214 24473 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:14.051499 24482 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.043942 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.062670 24486 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.063059 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.045444 24420 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.064052 24479 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:14.064292 24487 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.065012 24513 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.065537 22748 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.066532 24487 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.067965 24491 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.070220 24471 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.072266 24491 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.061605 24473 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.052862 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.075453 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.054117 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:14.056141 24518 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.076331 24518 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:14.056237 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.077250 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:14.052313 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 97633.9us
I20260709 13:07:14.079100 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.080133 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:14.077087 22901 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:14.081239 24491 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:14.081490 24465 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:14.081673 24491 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.081880 24465 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.082099 24491 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.082260 24465 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.091274 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.091166 24464 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:14.095325 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.094645 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.076743 24518 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.098802 22871 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.101926 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:07:14.099327 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46573.9us
I20260709 13:07:14.098796 22797 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:14.091645 24482 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.085157 24471 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.088742 24467 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.103693 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.104039 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:14.112310 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.119015 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.102473 24484 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:14.119740 22900 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:14.102517 24420 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.106034 24520 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.128448 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.107268 24482 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:14.130558 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.108505 24488 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.123329 24479 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.131124 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.094138 24507 consensus_queue.cc:1059] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:07:14.126180 24484 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.142529 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.143255 22898 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.143581 24507 raft_consensus.cc:3055] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.142776 24484 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:14.128892 24471 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.133565 24482 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.147847 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.137748 24522 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.143954 24507 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:14.137756 24521 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.133440 24488 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.139806 24515 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.139649 24520 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.135469 24479 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:14.146940 24471 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.138322 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.153662 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.150583 24490 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:14.154654 24513 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.164165 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:14.153899 24481 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.157301 24488 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.156136 24522 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:14.159291 24520 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:14.168152 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.157404 24515 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:14.169291 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.171787 24515 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.160308 24479 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.159569 24521 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:14.171092 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.160857 24471 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.162945 24489 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.184396 24521 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.170317 24522 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.169587 24472 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.226076 24467 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.182978 24507 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:14.263595 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 164857us
I20260709 13:07:14.267639 24484 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.270231 24484 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.273947 22797 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:14.275828 24491 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.277467 24491 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.279474 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:14.280341 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.281291 24467 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:14.281700 24467 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.282050 24467 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.261910 24507 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.290652 24472 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.290886 24525 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.291826 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:14.293639 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.289489 24484 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:14.300484 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:14.302213 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:14.294776 24481 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:14.306403 24527 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:14.295358 24525 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:14.325333 24524 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:07:14.300215 24484 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:14.307446 24481 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:14.309387 24527 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:14.342695 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 230433us
I20260709 13:07:14.340329 24484 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.305667 24479 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:14.347465 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45314.9us
I20260709 13:07:14.342568 24481 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.349016 22871 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.363382 24520 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.343459 22898 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:14.343741 24527 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.366592 22903 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.370558 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.349457 24472 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.363937 24488 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.379421 24519 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.380244 24499 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.380780 24499 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:14.381688 24499 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.382050 24472 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:14.382719 24472 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.382895 24499 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.340994 24471 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.384626 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.358450 24518 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.373262 24520 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.356954 24473 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:14.390161 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 82707.6us
W20260709 13:07:14.395763 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 46960.9us
I20260709 13:07:14.399699 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.400717 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.400853 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.358302 24477 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.401970 22871 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.404632 22797 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:14.406318 22904 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.380776 24488 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.412478 24522 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.414065 24481 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.402563 24531 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:14.415026 24531 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.415380 24531 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:14.408005 24520 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.414458 24522 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.403429 24526 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.414151 24482 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.401671 24519 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.421514 24526 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:14.421931 24467 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.423921 24482 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.402851 24515 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.420975 24481 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.424000 24467 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.421948 24526 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.424880 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.425398 24518 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.426261 24479 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:14.426451 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.426641 24477 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.399240 24465 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.402281 24476 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.436496 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.418478 24520 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:14.418731 24521 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.447386 24471 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.441251 24486 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:07:14.449863 24473 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.450812 24465 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:14.444419 24520 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.455050 23450 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:14.457778 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.458561 22748 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:14.444499 24481 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.458978 24472 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.461973 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.456933 24484 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.465801 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:14.473284 22719 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d8a8f3cf129492eb679da24703f5b84. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:14.473716 24515 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.475471 24521 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.448320 24467 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.493278 24484 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.490967 24533 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.496500 24533 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:14.492841 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 85465us
I20260709 13:07:14.506327 24472 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.483649 24481 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.508536 24481 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.484119 24467 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:14.509862 24467 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.514158 24527 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:14.517791 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 114617us
I20260709 13:07:14.522161 23452 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:14.498620 24533 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:14.545830 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 141261us
I20260709 13:07:14.553637 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.554163 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:14.555616 22871 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.557250 24420 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.557658 24490 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:14.557677 24420 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.558353 24420 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.558388 24490 raft_consensus.cc:697] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:14.559434 24490 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.561930 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.563315 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.564932 24482 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.562693 24533 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:07:14.566452 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.567466 24519 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.568228 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.571611 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.573093 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.573453 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.574748 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.575340 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.577728 24519 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.578047 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.579097 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.577284 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.563094 24520 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.581768 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.582480 24520 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:14.582943 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.583138 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:14.583614 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.583978 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.584700 22749 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 3: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:14.584604 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.588408 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.589967 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.538777 24526 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.592097 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.592696 22753 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:07:14.593801 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.594506 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.594741 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.595117 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.563268 24488 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:14.597256 24509 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.599330 24527 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.597141 24471 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.600495 24526 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:14.600883 24471 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.602972 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.586331 24464 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:07:14.605584 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.606415 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.607281 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.597506 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.598062 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.586954 24528 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.608753 24464 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.598313 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:14.608801 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.610996 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.612102 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.612283 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.612939 24514 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:14.613184 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.613462 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.613628 24499 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:14.614444 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.614574 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.615711 22903 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:14.615770 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.616631 24499 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.617043 24499 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:14.617390 24499 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.617709 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.618712 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.598380 24530 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.619762 24509 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.620064 24530 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:14.620532 24530 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.621955 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.624425 22896 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.625619 22829 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:14.625645 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.625151 24521 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.627794 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.627966 22827 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.628480 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.568385 24531 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.629278 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.609201 24528 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:14.631102 22826 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:14.632169 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.625636 24522 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.635228 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.633921 22826 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.634464 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.625649 24503 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.635469 24507 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:14.635708 24522 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:14.645750 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.645956 24522 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.647307 24490 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.627990 24521 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.636193 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.654089 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.655071 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.655316 24490 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.656437 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.638427 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.657793 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.638722 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.640388 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.640116 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.640725 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.661334 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:14.662271 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.639006 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.642040 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.636591 22824 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:14.642659 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.663482 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.663875 22831 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:14.666610 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.666868 22896 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.668680 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:14.670256 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.643625 22828 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.671119 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.671957 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.641429 24507 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.672160 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.640650 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.672693 22823 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:14.673013 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.640911 24536 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.673391 24518 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.613487 24514 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.639063 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.670153 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.674762 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.675691 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.666246 24471 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.632653 24528 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.678546 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.679551 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.680847 22824 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:07:14.653144 24521 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.671933 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.678277 24520 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.682852 24513 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:14.683843 24473 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:14.684211 24473 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:14.685029 24473 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.686236 22823 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.686318 24473 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.677660 24490 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.688606 24514 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:14.677117 24538 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.689704 24538 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:14.690055 24538 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.691552 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.677871 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.698120 22756 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:14.693130 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.692317 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.693758 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.692408 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.674590 24533 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.705763 22824 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.706590 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.702926 24480 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:07:14.707903 24507 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:14.705478 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.714881 24536 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.715853 24489 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.685096 24420 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.716480 24471 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.718655 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:14.719231 24420 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.719419 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:14.717211 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.719944 22903 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:14.713670 24491 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.703540 24521 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.721874 24521 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:14.722035 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.697911 24513 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.691604 24541 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:14.724292 24541 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:14.716251 24489 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:14.726912 24520 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:14.722270 24521 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.698341 24518 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.709358 24533 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:14.703917 24485 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:14.734179 24533 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.727766 24489 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.730906 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.732872 22825 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:14.733566 22828 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.735193 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.726845 24480 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.740751 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.742739 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.746570 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.729629 24541 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.747442 22823 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.749980 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.750247 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.752054 24482 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:14.753486 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.746577 24507 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:07:14.754103 24490 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.754554 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.755038 24420 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:14.755961 24420 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.756175 22902 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:07:14.756371 24420 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:14.757758 24471 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:14.758096 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.758409 24519 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:14.760514 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.761904 24531 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:14.763046 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.721202 24530 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.763773 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.764364 24497 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:14.763972 22653 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 3, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 3 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.759964 24526 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.758229 24471 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:14.765231 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.752512 24489 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.754432 24507 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.767302 24536 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:14.767717 24521 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.768635 24539 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:14.780686 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.780964 24539 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:14.781279 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.769933 24488 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:14.780041 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.783473 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:14.782749 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.770061 24528 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.786317 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.792908 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.793928 22824 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.794461 24497 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.794498 24520 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:14.795008 24521 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.763613 24467 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.795639 24520 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.796307 24520 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.796738 24520 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:14.796839 22829 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.797071 24467 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.803399 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.775372 22750 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.777275 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.777705 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.779121 24536 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.778873 24526 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.770247 24538 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.813464 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.812809 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.785475 22897 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.814931 24538 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.810068 22896 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.816154 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.818547 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.820163 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.821280 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:14.762831 24420 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.784879 24488 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:14.786948 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.787928 24530 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:14.788877 22826 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:14.775204 24507 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:14.790078 22827 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.790807 22825 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:14.826059 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.781857 24539 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.829852 23451 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:07:14.830088 24528 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:14.830868 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:14.831271 24533 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.832405 24507 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.833366 24533 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.833609 24539 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.791359 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.770118 24499 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.792083 24522 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.796212 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.773329 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.773877 24471 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.796229 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.797848 24520 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.798640 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:14.799149 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.799245 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.840549 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.806527 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.808609 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:14.841490 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.810043 24544 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:07:14.812772 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.820633 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.823765 22902 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.823989 22896 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 2: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:14.825995 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:14.801337 24541 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.836724 22831 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.846235 24520 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.846458 24471 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.847151 24499 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.848924 24544 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.842482 24480 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.849282 24544 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:14.835035 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.842957 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.835978 22898 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:14.851861 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.856926 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.852916 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.852988 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.849947 24497 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:07:14.859402 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.860601 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.860689 24485 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.861452 24514 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.858377 24526 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:07:14.862269 24514 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:14.862329 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.862584 24526 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:14.840359 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.850337 24476 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.862797 24514 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.864671 24497 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.855307 24473 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.877529 24497 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:14.878441 24497 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.879643 24497 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.865209 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:14.865159 24481 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:14.851673 22822 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.881897 24521 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.885152 22828 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:14.885576 24521 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:14.885775 22828 raft_consensus.cc:606] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:14.886459 24521 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.887598 24521 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.887735 24541 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.861261 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.863938 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.890277 24476 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.890647 24514 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.858611 24507 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.891600 24518 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.883263 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:14.892622 24482 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.892980 24482 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:14.893803 24482 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.895177 24482 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.896531 24519 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:14.898036 24519 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.898365 24519 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:14.899127 24519 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.869879 24513 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.870256 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.900410 24519 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.900943 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.893128 22896 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:14.902004 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.870836 24464 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:14.869880 24456 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:14.902719 24464 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.902827 24456 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.904712 24464 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:14.904963 24456 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:14.905241 24503 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.905556 24456 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.905934 24503 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.907203 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.872256 24545 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.872354 24522 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:14.907850 24499 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.861274 24531 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:14.869657 24420 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.849715 24488 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.842432 24530 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.862988 24526 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:14.883248 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.883508 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:14.884284 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.911916 22831 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 2: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:14.854611 24544 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.883555 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.884975 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:14.899562 22822 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:14.866816 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.841894 24467 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:07:14.909013 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.913534 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.914880 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.916326 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.917536 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.927085 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.918833 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:14.927659 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60871.1us
I20260709 13:07:14.928316 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.928989 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.930163 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.930677 22755 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:14.914234 24533 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.933270 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.933377 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:14.934190 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:14.935643 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:14.953905 24490 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.916846 24544 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.932922 24485 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:14.941780 24488 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:14.918648 24530 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:07:14.991108 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57820.2us
I20260709 13:07:14.934715 24473 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.927237 24518 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.931685 24513 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:14.935746 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:15.003952 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68246.1us
I20260709 13:07:14.918956 24502 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:14.958426 24536 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:14.933884 24489 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.933017 24464 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:14.998624 24540 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:14.933211 24503 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.959214 24491 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:14.960250 23419 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:14.944494 24533 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:15.014184 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:15.016083 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60045.3us
I20260709 13:07:14.991209 24485 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:14.926023 24526 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:14.998842 24531 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:07:14.936172 24499 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.001981 24481 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:14.936128 24467 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.014122 24465 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.018563 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:15.023820 24533 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.027280 24485 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.059309 24484 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:15.061199 24472 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.023872 24522 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:07:15.066890 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.038091 24481 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.037181 24464 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:15.066543 24513 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:07:15.061606 24472 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.081944 24481 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.083334 24536 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:15.043833 24491 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:07:15.085235 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:15.088366 24526 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.038180 24490 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:07:15.075743 24522 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:07:15.088943 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 61845.2us
I20260709 13:07:15.094671 22747 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:07:15.105422 22900 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
W20260709 13:07:15.110671 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 150487us
I20260709 13:07:15.113039 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.114897 24536 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.115245 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.116648 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.116997 24536 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:15.118219 24536 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.118561 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.119021 24536 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:15.120035 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.111197 24549 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:15.105655 24481 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.129552 24472 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.130743 24472 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.131640 24482 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.131995 24482 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:15.122165 24540 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.132797 24482 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.135736 24482 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.136675 24490 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.137038 24490 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:15.140928 24484 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.111430 24522 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.141248 22822 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:15.143087 24527 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.143469 24527 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.143549 24522 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.144394 24527 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.144889 24522 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.143085 24532 raft_consensus.cc:993] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:15.146947 24491 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:15.147472 24532 raft_consensus.cc:1081] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
W20260709 13:07:15.147491 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.142306 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.093415 24464 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:15.149551 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.153250 22904 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:15.142004 22903 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:15.158694 24540 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.161811 23450 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:15.162869 22897 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:15.155236 24538 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.165812 24538 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:15.165840 24464 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.166383 24538 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.169533 22905 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:15.143573 22898 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:15.141278 24547 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:15.170521 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:15.142066 24480 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:15.168835 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:15.172851 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:15.172765 24548 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:07:15.173669 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:15.174664 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:07:15.195242 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:15.175030 24490 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.175410 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:15.175786 24527 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.171710 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:15.148208 24536 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.181787 24530 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:15.181957 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:15.161770 24507 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:07:15.184379 22904 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:15.181957 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:15.186602 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:15.187034 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:15.187243 24533 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.187647 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:15.187927 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:15.187654 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:15.188771 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.189162 22901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43998
I20260709 13:07:15.196835 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:15.210399 22755 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:15.212072 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.155232 24545 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:07:15.193743 24547 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:15.214097 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.215595 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:15.214010 22901 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.188681 24473 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:07:15.221114 22901 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.220944 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.221637 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:15.224169 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.224943 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.225875 24550 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.230253 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.205040 24502 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:07:15.234289 24502 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:15.234768 24502 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:15.235836 24502 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.237442 24502 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.217005 24536 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:15.209894 24544 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:07:15.228067 24521 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.222213 24514 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:15.263537 22902 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:15.229043 24481 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.218655 24552 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.243458 24507 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:15.274521 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 52948.1us
I20260709 13:07:15.249497 24522 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.275542 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:15.276804 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:15.220189 24492 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:07:15.261889 24503 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:15.278292 24514 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.279623 24552 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.252828 24538 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.290138 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.285130 24456 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:15.297706 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.299039 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.294845 24560 ts_tablet_manager.cc:933] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:15.302498 24507 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:15.308691 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:15.281147 24547 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.289214 24491 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:07:15.298202 24467 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.306944 24490 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.282035 24503 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.305020 22897 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:15.310454 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:15.294837 24554 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:15.275727 24549 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:07:15.286005 24509 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:07:15.296663 24456 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:15.308357 24532 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.324838 24485 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.327858 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.309626 24507 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:15.329473 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:15.331188 24549 raft_consensus.cc:3055] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:15.331547 24549 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:15.333300 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.313084 24552 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.362749 22754 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:15.323632 24497 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.335237 22901 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.322000 24555 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.368440 22903 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:15.327456 24420 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.329442 24489 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:15.370471 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.341773 24502 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.341359 24481 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:15.343876 24544 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:15.375854 22901 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:15.322892 24473 consensus_queue.cc:1175] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3b626c41600d4737a178ebab8a7ebfc2" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
I20260709 13:07:15.332288 24549 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:15.369230 24554 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:15.365511 24497 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:15.405040 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.409808 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:15.436012 23451 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.404520 24489 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:15.374238 24420 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:15.410050 24540 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.417115 24481 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.399441 24532 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 13:07:15.422698 24557 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:15.409859 24553 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:15.422112 24456 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.514331 24485 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:15.496270 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:15.497128 24484 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.505366 24467 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.437852 24507 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.538688 24467 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.539541 24467 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.531296 24481 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.542057 24550 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.541657 24489 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.544927 23451 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:15.545662 24533 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:15.546813 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 141831us
I20260709 13:07:15.547640 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.548333 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:15.532090 24560 tablet_copy_client.cc:323] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.65:37213
I20260709 13:07:15.532649 24484 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.537572 22899 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:15.509644 24420 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.409698 24558 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.184s
I20260709 13:07:15.516331 24553 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:15.549114 22822 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:15.566272 22822 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.553860 24497 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.567076 22822 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.567551 24497 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.568001 22822 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.555284 24530 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:15.571015 24530 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:15.571491 24484 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.571827 24530 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.573589 24480 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.555987 24533 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:07:15.557101 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 245155us
I20260709 13:07:15.574409 24533 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.574529 22765 tablet_copy_service.cc:140] P 6a5191aaff76469bbf6c4735dc9e6509: Received BeginTabletCopySession request for tablet 4d8a8f3cf129492eb679da24703f5b84 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:56780)
I20260709 13:07:15.575093 22765 tablet_copy_service.cc:161] P 6a5191aaff76469bbf6c4735dc9e6509: Beginning new tablet copy session on tablet 4d8a8f3cf129492eb679da24703f5b84 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:56780: session id = c45e1805f049467697a7358fb566de2f-4d8a8f3cf129492eb679da24703f5b84
I20260709 13:07:15.562238 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:15.578202 24533 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.573937 24480 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:15.580139 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.580171 22828 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:15.556684 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:15.593920 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:15.564790 24503 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:15.605295 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:15.600158 24554 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.618764 24548 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.618698 24554 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:15.615329 24568 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:07:15.619637 24480 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.621047 24480 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.621052 24456 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:15.619192 24548 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:15.606392 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:15.617130 22831 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:15.660981 24548 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.662498 24548 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.668035 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:07:15.677625 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 65636.4us
I20260709 13:07:15.678619 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.679935 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:15.697683 24536 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.705426 22897 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:15.682835 24545 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:15.683246 24490 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:07:15.712139 22755 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:15.708292 24536 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.706589 22822 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:15.715617 24527 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:15.716395 24547 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.718254 22901 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:15.719439 22902 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:15.721546 22750 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:15.744129 22755 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.761950 22755 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:15.730671 24536 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.760789 24514 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.741437 24562 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:07:15.762910 24465 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:07:15.779973 24456 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:15.780365 24456 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:15.780663 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.767328 24472 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:07:15.773926 24489 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.783536 24536 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.781190 24456 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.788763 24549 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.788960 24456 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.793588 24514 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.796360 24514 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:15.798543 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:15.799199 22747 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:15.801119 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:15.801486 24499 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:15.814474 24499 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:15.815584 24499 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.804051 24562 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:15.817760 24562 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.820406 24562 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.819470 24553 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.824234 22828 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:15.824633 24420 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.817134 24499 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:15.835249 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.833060 24490 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:07:15.843699 24490 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:15.844066 24490 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:15.845861 24467 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.850472 24507 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:15.844868 24490 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.859431 24490 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.858013 23448 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:15.862787 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:15.863495 22897 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 2: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:15.865487 22901 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:15.891270 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:15.888507 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:15.891147 22653 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 329) took 1149 ms. Trace:
I20260709 13:07:15.893210 22901 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:15.863507 24553 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.893816 22653 rpcz_store.cc:276] 0709 13:07:14.727037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.727188 (+   151us) service_pool.cc:224] Handling call
0709 13:07:15.876865 (+1149677us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:14.925713 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:14.954413 (+ 28700us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:14.954446 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:07:14.954462 (+    16us) tablet.cc:662] Decoding operations
0709 13:07:14.954796 (+   334us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:14.954864 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:14.954894 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:07:14.955057 (+   163us) tablet.cc:701] Row locks acquired
0709 13:07:14.955072 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:07:14.955199 (+   127us) write_op.cc:270] Start()
0709 13:07:14.955296 (+    97us) write_op.cc:276] Timestamp: P: 1783602434955164 usec, L: 0
0709 13:07:14.955315 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:07:15.034722 (+ 79407us) log.cc:844] Serialized 533 byte log entry
0709 13:07:15.361599 (+326877us) op_driver.cc:464] REPLICATION: finished
0709 13:07:15.613492 (+251893us) write_op.cc:301] APPLY: starting
0709 13:07:15.613592 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:15.613706 (+   114us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:15.613863 (+   157us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:15.614142 (+   279us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:15.614307 (+   165us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:07:15.614348 (+    41us) write_op.cc:312] APPLY: finished
0709 13:07:15.614553 (+   205us) log.cc:844] Serialized 23 byte log entry
0709 13:07:15.614726 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:15.614945 (+   219us) write_op.cc:454] Released schema lock
0709 13:07:15.614989 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:15.615470 (+   481us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:15.615502 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":251553,"apply.run_cpu_time_us":1988,"apply.run_wall_time_us":1988,"num_ops":1,"prepare.queue_time_us":139037,"prepare.run_cpu_time_us":9302,"prepare.run_wall_time_us":188299,"raft.queue_time_us":121387,"raft.run_cpu_time_us":37962,"raft.run_wall_time_us":262771,"replication_time_us":406121,"thread_start_us":419311,"threads_started":4,"wal-append.queue_time_us":167127}]]}
I20260709 13:07:15.907167 22822 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:15.907862 22765 tablet_copy_source_session.cc:215] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:15.896145 24541 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.911217 22899 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:15.911654 24560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d8a8f3cf129492eb679da24703f5b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:15.911492 22650 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 3, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 3 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:15.903085 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:15.911275 24541 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:15.895867 24538 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:07:15.915369 24541 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:15.896625 24521 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:07:15.917542 24503 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:15.917934 24503 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:15.913252 22652 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 3 to 4, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:15.918315 24503 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.920461 24545 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:15.955327 24562 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.958021 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:15.958611 22750 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.958844 22902 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:15.961838 24533 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:15.941852 24545 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:15.964248 24585 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:07:15.964478 24545 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:15.966096 24545 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.969692 22902 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:15.968667 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:15.964318 24562 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:15.978794 22828 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:15.977989 22898 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:15.967278 24544 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:07:15.975260 24538 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:07:16.009260 22750 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:16.027590 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:16.029568 24499 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:16.030177 24499 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:16.031076 24499 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.033056 22898 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:16.087023 24560 tablet_copy_client.cc:806] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:16.100833 24560 tablet_copy_client.cc:670] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:16.107869 22898 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:16.110848 24533 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:16.063758 24503 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.116917 23448 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:16.118731 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:16.122462 24530 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:16.125991 24533 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.133895 24533 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.136950 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:16.137666 22755 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:16.139252 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:16.140607 24533 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.141183 24533 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:16.141587 24533 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.141963 24533 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.146004 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:16.146906 22900 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 2: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:16.150144 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:16.150892 22900 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:16.159129 24490 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.159734 24490 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.160342 22900 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.162979 24490 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.164803 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:16.166828 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:16.166920 24564 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:16.167600 22822 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:16.169255 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:16.172664 23448 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.174338 22755 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:16.183055 22828 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:16.184312 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:16.186597 24544 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:16.187063 24544 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:16.187901 24544 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.188648 22822 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:16.173305 24490 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:16.190169 24490 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.190608 24490 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:16.194829 22750 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.227483 24420 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:16.228765 24530 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.179847 24503 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.229259 24530 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.232431 22754 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.236404 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:16.237630 24503 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.234968 24530 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:16.238088 24503 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.240170 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:16.240849 22822 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:16.242071 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:16.242692 22747 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:16.243891 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:16.244692 24464 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:16.245051 24464 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.245718 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:16.246719 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:16.247468 22900 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:16.250100 22754 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.256793 22750 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:16.259724 22755 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:16.249461 24464 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:16.249370 24503 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.255367 24420 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:16.274335 24562 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.257141 24484 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:07:16.263973 24541 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:07:16.286839 24560 tablet_copy_client.cc:538] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:16.287763 23448 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:16.259279 24564 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:07:16.274937 24420 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.305123 24420 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:16.289147 24507 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.316522 23450 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.278766 24562 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.304939 24521 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:07:16.392788 24482 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.401284 24482 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.402498 24518 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.403388 24482 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:16.403108 24518 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.330009 24507 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.405520 24518 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:16.411294 24548 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.411804 24548 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.413777 24548 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.418951 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:16.420114 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:16.424073 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:16.424599 22900 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:16.425705 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:16.420652 22756 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:16.428766 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:16.433959 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:16.434584 22831 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:16.439281 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:16.454368 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:16.454895 23450 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
W20260709 13:07:16.460497 22903 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.013s	sys 0.015s
W20260709 13:07:16.461197 22903 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.013s	sys 0.015s
I20260709 13:07:16.430624 24545 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:16.463999 24545 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.382308 24490 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.431586 24530 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:16.470731 24545 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.469877 24538 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.472199 24530 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.473295 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:16.498597 24467 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.498816 24507 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.499097 24467 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.500963 24467 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:16.496621 22822 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:16.492811 24530 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:16.503537 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:16.492117 24553 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.504124 22896 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:16.506394 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:16.500463 24464 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.507573 24467 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:16.492769 24538 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.517858 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:16.518602 22747 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:16.519992 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:16.523312 22896 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.505506 24420 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.514189 24467 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.528107 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:16.523947 24507 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:16.507966 24553 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.529680 24562 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.541237 24467 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:16.584966 24481 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.024s	sys 0.028s
I20260709 13:07:16.541438 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:16.586364 24507 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.529166 24484 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:07:16.650197 24507 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:16.651014 24596 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:07:16.652148 24601 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:07:16.653273 24594 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
W20260709 13:07:16.602388 24556 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:07:16.602674 24481 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.025s	sys 0.028s
I20260709 13:07:16.598907 24490 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.750852 24566 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:16.754680 24420 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:16.755249 24420 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.754988 24566 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.694820 24521 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.745878 22822 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:16.755728 24599 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.160s	user 0.001s	sys 0.007s
W20260709 13:07:16.797482 24600 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:07:16.751080 24548 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:16.830933 24553 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.790616 24521 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.843922 24464 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.847008 24566 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.755671 24420 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.922112 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:16.922348 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:16.850376 24467 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:16.809195 24533 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.624s	user 0.007s	sys 0.000s
I20260709 13:07:16.858493 24530 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.869580 24560 tablet_bootstrap.cc:492] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:07:16.828838 24507 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:16.883188 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 391924us
I20260709 13:07:16.704463 24549 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.889707 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:16.896068 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 436364us
W20260709 13:07:16.919291 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 113548us
I20260709 13:07:16.922811 22747 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:16.791652 24544 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:16.929836 24553 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.977990 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:16.978610 22756 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 1.
W20260709 13:07:16.980650 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 181861us
I20260709 13:07:16.931684 24522 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.969568 24566 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.920647 24548 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.987239 24522 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.977860 24544 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.970327 24553 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.979554 24616 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.001441 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.003080 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:16.982635 22905 raft_consensus.cc:2368] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5.
I20260709 13:07:16.935266 24533 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.007803 24467 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.008690 24533 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.004786 22829 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:17.011308 24616 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:16.977000 24549 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.013391 24616 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.013993 24614 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.014468 24614 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.003525 22797 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:17.010561 24553 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.017743 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.024915 22755 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:17.026177 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:17.027123 24616 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:17.027541 24616 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.027921 24616 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:17.030148 24464 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:17.030632 24464 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.032552 24464 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:16.976775 24545 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:16.997871 24503 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:17.041849 22822 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:17.043001 24530 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:16.998780 24620 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:07:17.043778 24598 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.301s	user 0.001s	sys 0.002s
W20260709 13:07:17.044477 24606 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:07:17.046041 24420 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.046708 24619 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:17.047339 24597 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 13:07:17.053001 22747 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:17.024574 24613 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.070708 24619 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.071139 24613 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.073285 24618 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.074124 24618 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.070735 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:17.056357 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 52887.8us
I20260709 13:07:17.078141 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:17.078858 22747 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.016502 24614 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:16.994829 24566 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.088677 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.089258 22750 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:17.059842 24562 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.034838 24549 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.050694 24612 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.070039 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:17.100618 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:17.070310 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1573) took 1150 ms. Trace:
I20260709 13:07:17.073606 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1577) took 1107 ms. Trace:
I20260709 13:07:17.082775 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.082942 22754 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.085760 24521 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.086040 24522 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.086531 24507 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.088162 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.109571 22753 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:17.073554 24533 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:17.072906 24619 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.072840 24621 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.056201 24503 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.104033 22755 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:17.106154 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.127506 22902 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:17.107964 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:17.108225 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.128765 22897 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:17.111862 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.129680 22900 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:17.130439 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.112908 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:17.101500 22903 rpcz_store.cc:276] 0709 13:07:15.920105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.920707 (+   602us) service_pool.cc:224] Handling call
0709 13:07:15.937658 (+ 16951us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.938271 (+   613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:15.938302 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:15.938712 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 13:07:16.462903 (+524191us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:16.462950 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.463005 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:17.043278 (+580273us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:17.066457 (+ 23179us) raft_consensus.cc:1685] finished
0709 13:07:17.066478 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:17.070228 (+  3750us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":12171520,"thread_start_us":523853,"threads_started":1,"wal-append.queue_time_us":599594}
I20260709 13:07:17.131594 22896 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.102738 22901 rpcz_store.cc:276] 0709 13:07:15.966163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.966305 (+   142us) service_pool.cc:224] Handling call
0709 13:07:15.966576 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.966909 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:15.966937 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:15.967339 (+   402us) log.cc:844] Serialized 26 byte log entry
0709 13:07:16.461868 (+494529us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:16.461910 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.461984 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:17.073401 (+611417us) raft_consensus.cc:1685] finished
0709 13:07:17.073420 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:17.073546 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":659,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"thread_start_us":489523,"threads_started":1,"wal-append.queue_time_us":617589}
I20260709 13:07:17.117889 22797 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:17.073082 24538 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.132400 22829 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:17.130944 22900 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:17.134431 22903 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:17.076190 24618 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.107394 24549 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:17.082186 24472 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.127470 22905 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:17.106176 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:17.144438 22756 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.094970 22747 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:17.128607 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:17.135651 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.099617 24614 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.107010 24518 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.117610 24553 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:17.066164 24620 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.088305 24616 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.133246 24522 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:17.146862 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.116998 22754 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:17.147737 22902 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:17.127944 24613 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.124918 24533 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.144138 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.107167 24562 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.161306 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.162060 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:17.182926 22828 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:17.184098 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:17.166729 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.167899 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.138057 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.185119 22830 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 2: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:17.168573 22829 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:17.180583 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:17.183148 22754 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:17.164738 24641 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:17.185986 22747 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
W20260709 13:07:17.184779 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76871.3us
I20260709 13:07:17.166106 24628 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.193600 23419 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:17.158584 23448 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:17.165555 24609 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:17.171738 24533 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.168373 24522 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.111373 24612 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.204936 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:17.181751 24530 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.148077 24549 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.158505 24614 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.150192 24472 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.181610 24553 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:17.245426 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56956.3us
I20260709 13:07:17.205825 24629 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:17.189404 24544 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.137679 24621 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.160816 24518 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.186579 24641 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.226414 22897 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:17.215734 22756 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:17.195118 24538 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.258142 24538 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.258765 24472 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.260145 24538 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.260643 24518 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.219745 24612 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:17.250433 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53195us
I20260709 13:07:17.210875 24522 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:17.245545 24545 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.205977 24640 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.195086 24507 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:17.245816 24613 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.218291 24616 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.286543 24621 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.217913 24548 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.251609 24629 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.286999 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:17.289000 24641 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.225029 24566 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:17.254237 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 67238.8us
I20260709 13:07:17.262662 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:17.205281 24609 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.267357 22896 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:17.253397 24420 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.252588 24625 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:17.256428 24544 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:17.220135 24530 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.261747 24627 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:17.273736 24640 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.284343 24635 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.284808 24613 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.287285 24629 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.217273 24503 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.203809 24622 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:07:17.210634 24624 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:07:17.205093 24628 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.255004 24484 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
W20260709 13:07:17.349030 24626 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
W20260709 13:07:17.379868 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 117296us
I20260709 13:07:17.380745 22719 leader_election.cc:400] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5.
I20260709 13:07:17.274645 24507 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.388000 22719 leader_election.cc:403] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:17.387873 24631 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.387876 24507 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:17.388742 24631 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.391422 24631 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:17.382361 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 136403us
W20260709 13:07:17.396643 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 187004us
W20260709 13:07:17.397861 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 151635us
I20260709 13:07:17.309573 24545 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:17.351130 24620 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.408982 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.422303 22896 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:17.409562 24503 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.414134 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.426694 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.427014 22756 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:17.427260 22755 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:17.375588 24420 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.373669 24641 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:17.370927 24609 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.370743 24621 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.446265 24472 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:17.369689 24544 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.446462 24621 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.446878 24544 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.446774 24472 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.448508 24472 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.350072 24549 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.449409 24481 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.449702 24549 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:07:17.450205 24640 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.351377 24562 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.364399 24541 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.425153 24499 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:07:17.453819 24541 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.456254 24541 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.326507 24614 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.373965 24625 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.457104 24625 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:17.425488 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:17.420221 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:17.458794 22902 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.446398 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.460681 22755 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:17.428421 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.461584 22747 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:17.428141 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:17.415448 24632 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:17.433204 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:17.470571 22750 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:07:17.380223 24627 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:07:17.442634 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 72859.3us
I20260709 13:07:17.443797 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:17.404435 24635 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.444020 24545 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:17.444361 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51583.8us
I20260709 13:07:17.482968 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.483608 22756 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:17.445952 24628 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.449880 24481 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.355363 24533 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.462710 22896 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.429627 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:17.423400 22899 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:07:17.443851 24605 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.664s	user 0.003s	sys 0.000s
I20260709 13:07:17.463961 24641 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.465586 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.467305 24420 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.469092 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.523716 22900 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:17.469094 24562 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:17.472186 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:17.473579 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.488420 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:17.489907 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:17.525566 22749 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:07:17.524940 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 51463.4us
I20260709 13:07:17.526371 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.490175 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.527096 22756 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:17.527856 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.495839 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.496016 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.496120 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.496693 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.496843 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.529655 22755 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:17.501142 24639 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.501128 24654 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:07:17.539354 22756 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:07:17.539875 22828 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:17.321419 24553 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.448508 24621 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.522194 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.531025 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.546571 22830 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:17.531522 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.532411 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:17.523036 22901 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:17.474005 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:17.472901 24631 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:17.500535 22902 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:17.522182 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.551398 22754 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:17.534868 22824 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:17.435317 24503 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.536134 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.556241 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:17.566284 24615 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.358s	user 0.002s	sys 0.000s
I20260709 13:07:17.566823 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.558753 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.540951 24636 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.568953 22752 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:17.570132 22900 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:17.550933 24631 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:17.571727 24631 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.536736 24628 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:17.534041 24623 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.478067 24644 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.548214 22896 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:17.592761 24644 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.593178 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.593779 22896 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:17.594561 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.594748 24644 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.595194 22896 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:17.595225 24629 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.546412 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.596441 22823 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:17.568097 24639 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.540803 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.556511 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.550450 22751 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:17.491362 23447 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:17.568882 22825 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:17.574854 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.604173 22824 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:17.576776 22827 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.583518 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.606297 22897 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:17.584355 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.585554 24622 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:17.585747 24481 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.610682 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:17.611348 22897 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.586400 24635 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.586081 24545 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.587833 24641 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.539463 24562 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.590746 24623 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.592268 24613 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.542618 24643 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.597594 22755 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:17.478976 24632 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.598502 22749 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:17.504247 24507 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.493178 24541 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.598719 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.597199 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.616364 24633 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:17.601599 22750 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:17.597664 22830 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:17.568090 24620 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:17.613436 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1585) took 1434 ms. Trace:
I20260709 13:07:17.654124 22826 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:17.597961 24628 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.627420 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.624157 24644 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.630788 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.632946 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.639273 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.639868 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:17.649421 22897 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:17.649789 22896 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:17.600925 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.641139 24632 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.604247 24522 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.658162 22756 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:17.628201 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.653904 22904 rpcz_store.cc:276] 0709 13:07:16.179458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.179674 (+   216us) service_pool.cc:224] Handling call
0709 13:07:16.180071 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.180438 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.180467 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:16.185921 (+  5454us) log.cc:844] Serialized 26 byte log entry
0709 13:07:16.192965 (+  7044us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b18002898a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:07:16.505548 (+312583us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:07:16.505586 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.505667 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:17.405785 (+900118us) spinlock_profiling.cc:243] Waited 893 us on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:17.613144 (+207359us) raft_consensus.cc:1685] finished
0709 13:07:17.613176 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:17.613365 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33372,"raft.queue_time_us":86,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":273,"spinlock_wait_cycles":11296256,"thread_start_us":312429,"threads_started":1,"wal-append.queue_time_us":411786}
I20260709 13:07:17.678838 22752 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:17.679847 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 670) took 1198 ms. Trace:
I20260709 13:07:17.642808 24629 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:17.667894 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 67066.2us
I20260709 13:07:17.641738 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:17.674767 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.652002 24530 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.681147 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:07:17.681763 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51044.7us
I20260709 13:07:17.680274 23450 rpcz_store.cc:276] 0709 13:07:16.480965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.481099 (+   134us) service_pool.cc:224] Handling call
0709 13:07:16.481397 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.481703 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.481733 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:16.482175 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:07:16.924921 (+442746us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:16.924957 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.925023 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:17.406626 (+481603us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:17.673476 (+266850us) raft_consensus.cc:1685] finished
0709 13:07:17.673496 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:17.679765 (+  6269us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3769,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":32486272,"thread_start_us":442403,"threads_started":1,"wal-append.queue_time_us":516357}
I20260709 13:07:17.684520 22902 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:17.666594 23448 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.664718 24628 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:17.636817 24614 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:17.608229 24636 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.667593 24633 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.696740 24655 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:17.698494 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.675531 24620 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:17.707928 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.649045 24625 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.702032 22822 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:17.706897 24648 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:17.709728 24530 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:17.723951 24530 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.724476 24530 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.712788 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.713956 24613 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:17.721956 24503 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.722720 24533 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.731391 24613 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.656843 24541 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:17.702430 24489 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:17.710762 24633 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.676579 24644 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:17.700345 24639 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.753803 22831 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:07:17.739109 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.747397 24634 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:17.610278 24627 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.736596 24613 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.742435 24641 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.761674 24635 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:17.765249 24641 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.765822 24635 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.766237 24635 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.767256 24648 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.767455 24641 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.767691 24648 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.768086 24634 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.740535 24489 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.733160 24655 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.741520 24549 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5.
I20260709 13:07:17.755466 22871 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.677331 24643 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:17.779551 24649 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:07:17.780418 24522 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.727929 24614 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.741624 24609 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:17.784220 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.770149 24634 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.770511 24489 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.762708 22827 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:17.770828 24655 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:17.806327 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.789434 24420 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:17.808569 24420 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.808969 24420 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.795372 24619 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:17.823997 24625 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.826887 22829 raft_consensus.cc:2368] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:17.830652 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.833120 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:17.834128 22902 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:17.792490 23445 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:17.807265 24562 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.847550 24623 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:17.850426 24541 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.824896 24619 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.851006 22753 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:17.851610 24619 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.822445 24507 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.812762 22899 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:17.784340 24620 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.791993 24643 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.881086 24636 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.882730 24644 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.828652 24544 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:17.895992 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 89707.7us
W20260709 13:07:17.899837 24602 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.014s	sys 0.075s
I20260709 13:07:17.911911 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:17.900352 24549 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:07:17.894856 24518 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.014s	sys 0.058s
I20260709 13:07:17.911803 24633 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:17.912086 24602 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.014s	sys 0.075s
I20260709 13:07:17.922214 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1150) took 1037 ms. Trace:
W20260709 13:07:17.923691 22748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.000s	sys 0.033s
W20260709 13:07:17.924988 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 495428us
I20260709 13:07:17.971971 23419 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:17.972857 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:17.973824 22748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.000s	sys 0.033s
I20260709 13:07:17.979003 23419 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:17.968688 24518 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.015s	sys 0.058s
I20260709 13:07:17.985715 22823 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:17.986286 24562 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:17.986606 24633 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:17.987177 24507 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:17.987524 24507 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.987936 24507 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:17.953646 24639 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:17.989338 24639 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.989714 24639 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:17.981490 24643 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:17.986666 24562 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.980350 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:17.993777 24643 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:17.994496 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.994328 24562 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.995123 22756 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:17.997103 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:17.995662 24614 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:17.997828 22756 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:17.998732 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:17.999260 22751 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:07:18.003154 24634 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.004937 24639 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:17.971679 24549 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:18.008778 22903 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.045s	sys 0.057s
I20260709 13:07:18.023177 22752 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:07:18.007220 22751 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:18.030851 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:18.025848 22903 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.045s	sys 0.059s
I20260709 13:07:18.034016 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:18.023106 24507 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.034934 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:18.035425 24614 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:18.035770 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:18.036278 23449 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:18.036620 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:18.037204 23446 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:18.037688 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:17.999606 24616 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:17.996333 24643 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:18.003801 24629 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.055397 24629 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.024457 24609 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:18.050184 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:18.058427 24499 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:18.058789 24499 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.059114 24499 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:18.063140 24631 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.064097 23447 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:18.064638 23448 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:18.068069 24499 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.068694 24635 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:18.005956 24628 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.070905 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:18.071650 24530 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:18.072860 24632 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:18.076086 22901 raft_consensus.cc:2368] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3.
I20260709 13:07:18.077901 24622 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:18.078402 24622 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.078820 24622 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:18.006134 24654 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:18.068872 24420 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:18.083734 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:18.083341 24616 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:18.097916 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:18.100121 24549 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:18.100687 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:18.104106 22905 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.002s	sys 0.000s
W20260709 13:07:18.104426 22905 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.002s	sys 0.000s
I20260709 13:07:17.970906 22899 rpcz_store.cc:276] 0709 13:07:16.879144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.882710 (+  3566us) service_pool.cc:224] Handling call
0709 13:07:17.286873 (+404163us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b5800211598. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6eee 0000000000375939 00007fcb79d4a1e0 00007fcb79e130c4 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:07:17.906272 (+619399us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b540034b7d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:17.916842 (+ 10570us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278979712}
I20260709 13:07:18.060003 24619 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:18.095196 24560 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.011s	sys 0.007s
I20260709 13:07:18.112568 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:18.131130 22752 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:18.082353 24624 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:07:18.084753 24613 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:18.122377 24530 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:18.103514 24630 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:18.150488 22900 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:18.155787 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:18.113150 22797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.002s	sys 0.015s
I20260709 13:07:18.156477 22752 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:18.089748 24654 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:18.139791 24560 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.011s	sys 0.007s
W20260709 13:07:18.149938 22719 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.544s (SENT) [suppressed 3 similar messages]
I20260709 13:07:18.155287 24530 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:18.161495 24530 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.161922 24530 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:18.162081 24631 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:18.148754 24634 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:18.164337 24639 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:18.165405 24616 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:18.156558 22797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.003s	sys 0.015s
I20260709 13:07:18.236315 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:18.241734 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 709326us
I20260709 13:07:18.242093 22756 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:07:18.206804 22751 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:18.243044 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:18.232187 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:18.158059 24630 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:18.242957 24632 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:18.230685 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 54221.3us
I20260709 13:07:18.251600 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.247480 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:18.252382 22748 raft_consensus.cc:2368] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:18.195245 24544 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:18.249739 23447 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:18.251811 24631 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:18.256191 24631 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.256644 24631 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:18.194343 24499 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:18.169517 24628 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:18.259976 22797 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:18.203917 24507 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:18.236162 24654 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:18.266395 23416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.009s	sys 0.017s
W20260709 13:07:18.273581 23416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.009s	sys 0.017s
W20260709 13:07:18.281958 23418 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.011s	sys 0.045s
W20260709 13:07:18.282683 23418 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.011s	sys 0.045s
I20260709 13:07:18.281064 22756 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:18.249203 23419 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:18.230118 24660 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:18.169644 24643 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.215773 24648 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:18.324466 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 75330.5us
I20260709 13:07:18.324496 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:18.326618 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.327301 22748 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:07:18.160518 24655 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:18.330539 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 416794us
I20260709 13:07:18.274837 24541 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:18.348492 23450 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:18.346452 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:18.322707 24544 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:18.249897 24562 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.350728 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:18.274607 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 802437us
I20260709 13:07:18.319237 24660 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:18.321439 24420 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:18.325860 24545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.018s	sys 0.027s
I20260709 13:07:18.345856 24507 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:18.377595 23449 raft_consensus.cc:2368] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4.
I20260709 13:07:18.330130 24619 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:18.380519 23419 leader_election.cc:400] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:18.382550 23419 leader_election.cc:403] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:07:18.383157 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53328.7us
I20260709 13:07:18.383752 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:18.383926 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:18.376181 24681 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:07:18.350172 24541 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:18.418396 22897 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.008s	sys 0.004s
I20260709 13:07:18.418550 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:18.391835 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.348528 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:18.419368 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.413736 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:18.419675 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.014s	sys 0.013s
W20260709 13:07:18.420002 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.014s	sys 0.013s
W20260709 13:07:18.420840 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 845881us
I20260709 13:07:18.397640 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.422323 22751 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:07:18.418840 22897 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.008s	sys 0.004s
I20260709 13:07:18.390672 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:18.380326 24614 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:18.379324 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:18.436270 24629 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:18.391337 24654 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:18.420192 24553 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:07:18.452374 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55952.9us
I20260709 13:07:18.464203 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.451377 24541 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:18.411127 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:18.476605 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 86017.2us
W20260709 13:07:18.406632 23417 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.070s	sys 0.103s
I20260709 13:07:18.408284 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.531605 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:18.533663 22754 raft_consensus.cc:2368] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
W20260709 13:07:18.382073 24545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.018s	sys 0.027s
I20260709 13:07:18.379092 24544 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.450712 24507 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:18.391067 24464 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:18.545243 23417 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.071s	sys 0.104s
I20260709 13:07:18.545650 22901 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:07:18.546303 22750 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:07:18.556530 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:18.591173 22902 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:18.441299 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:18.592720 22896 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.015s	sys 0.048s
W20260709 13:07:18.593043 22870 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.020s	sys 0.038s
W20260709 13:07:18.593348 22870 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.020s	sys 0.038s
I20260709 13:07:18.593683 24633 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:18.594311 24633 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:18.596798 24633 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:18.593729 22896 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.015s	sys 0.048s
W20260709 13:07:18.559257 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 137844us
I20260709 13:07:18.540321 22755 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:18.611802 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.612632 22755 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:07:18.526542 24518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.009s	sys 0.005s
I20260709 13:07:18.562114 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:18.613497 24518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.009s	sys 0.005s
I20260709 13:07:18.614136 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:18.555580 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:18.555631 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 378) took 1137 ms. Trace:
I20260709 13:07:18.615433 22898 rpcz_store.cc:276] 0709 13:07:17.404208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.416840 (+ 12632us) service_pool.cc:224] Handling call
0709 13:07:17.473998 (+ 57158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.493431 (+ 19433us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:17.923771 (+430340us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.388410 (+464639us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.440980 (+ 52570us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:18.463910 (+ 22930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.489755 (+ 25845us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.541489 (+ 51734us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":463351,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":74139,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":857148,"threads_started":2}
I20260709 13:07:18.617581 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:18.618247 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:18.618670 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:18.618798 22755 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:18.619236 22898 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:18.620062 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:18.566545 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.621384 22754 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:18.568099 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:18.575178 24661 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:18.561568 24499 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:18.578701 24530 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.579480 22752 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
W20260709 13:07:18.526350 24620 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.005s	sys 0.010s
I20260709 13:07:18.632872 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:18.588886 22749 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:18.633415 22829 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:18.541149 24602 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.004s	sys 0.010s
I20260709 13:07:18.373865 24630 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.622627 22751 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
W20260709 13:07:18.621557 24489 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.809s	user 0.010s	sys 0.000s
I20260709 13:07:18.624439 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:18.645610 22755 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:18.626147 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:18.649729 22902 raft_consensus.cc:2368] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:18.627787 22750 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:18.633786 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.655856 22826 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 2: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:18.634217 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.656817 22831 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:18.574541 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:18.638634 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.638900 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.677338 22828 raft_consensus.cc:2368] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:18.677227 22755 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:18.640064 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.640889 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.678440 22824 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 2: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:18.679293 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:18.680102 22824 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 2: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:18.687803 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:18.621096 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:18.688288 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.688768 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:18.637487 24620 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.005s	sys 0.010s
I20260709 13:07:18.689451 22826 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:18.688809 22828 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:18.762512 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:18.763109 22828 raft_consensus.cc:2368] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:18.763792 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.764279 22828 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:18.689894 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:18.690279 22830 raft_consensus.cc:2368] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:18.689069 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:18.688473 24651 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.705688 24643 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:18.755602 22754 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:18.740711 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.630995 24662 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:18.621680 24464 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:18.636389 24602 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.004s	sys 0.010s
I20260709 13:07:18.629197 24499 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:18.739751 23452 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.014s	sys 0.009s
W20260709 13:07:18.803684 23452 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.015s	sys 0.009s
I20260709 13:07:18.758713 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:18.556331 24660 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.761134 22901 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:18.822453 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:18.688035 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.632280 24635 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:18.765692 22824 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:18.693727 22899 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:18.694556 24622 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.003s
I20260709 13:07:18.771960 22831 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:18.776443 22825 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:18.631474 24661 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.781898 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.730717 24613 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:18.685951 24489 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:18.727033 24656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.004s	sys 0.021s
I20260709 13:07:18.893821 22901 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:18.775911 24620 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:18.905583 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:18.895829 22830 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:18.894659 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:18.909235 22826 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:18.896239 22898 raft_consensus.cc:2368] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:18.807026 24499 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:18.914494 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.808111 24655 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:18.915433 22826 raft_consensus.cc:2368] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:18.913400 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.894065 24648 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:18.778921 24651 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:18.917802 22830 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:18.918347 22756 raft_consensus.cc:2368] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
W20260709 13:07:18.906816 24656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.007s	sys 0.021s
I20260709 13:07:18.919907 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:18.932354 22826 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:18.932732 22901 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
W20260709 13:07:18.924678 22719 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.016s	sys 0.107s
I20260709 13:07:18.772557 24507 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:18.948685 22904 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.021s	sys 0.013s
W20260709 13:07:18.944110 22719 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.016s	sys 0.108s
I20260709 13:07:18.913678 24562 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:18.952658 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:18.953418 24671 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:07:18.943531 24665 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:18.954355 24665 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:18.949162 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 799164us
I20260709 13:07:18.946102 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:18.803582 24662 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.918928 24663 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:18.962994 22748 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.614s	user 0.005s	sys 0.005s
I20260709 13:07:18.953601 22901 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:18.972036 22748 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
W20260709 13:07:18.972530 24549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.004s	sys 0.023s
W20260709 13:07:18.949750 24668 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
I20260709 13:07:18.944881 24643 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:18.944950 24634 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:18.974157 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.970265 22753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.008s	sys 0.019s
I20260709 13:07:18.901017 24622 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.975342 22719 leader_election.cc:400] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3.
I20260709 13:07:18.998106 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:18.998812 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:18.999764 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:19.001078 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:18.973011 24549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.004s	sys 0.023s
I20260709 13:07:19.021121 24549 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:19.021715 24549 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.023625 24549 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:19.023833 24560 tablet_bootstrap.cc:492] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:07:19.024101 24639 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:19.024262 24530 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:19.024618 24639 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:18.979539 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:19.025581 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1595) took 1601 ms. Trace:
I20260709 13:07:19.022622 24622 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.025959 22905 rpcz_store.cc:276] 0709 13:07:17.423702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.424322 (+   620us) service_pool.cc:224] Handling call
0709 13:07:17.448802 (+ 24480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:17.469432 (+ 20630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:17.469463 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:17.476460 (+  6997us) log.cc:844] Serialized 26 byte log entry
0709 13:07:18.104740 (+628280us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:07:18.104776 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.104836 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.025387 (+920551us) raft_consensus.cc:1685] finished
0709 13:07:19.025408 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.025529 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20256,"raft.queue_time_us":20427,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":12762,"thread_start_us":569061,"threads_started":1,"wal-append.queue_time_us":998144}
I20260709 13:07:19.027693 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:19.028343 22905 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:19.026409 23446 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:19.029043 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:19.030231 22905 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:19.031445 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:19.032058 22901 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:18.905575 24620 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.033380 22719 leader_election.cc:403] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:19.033879 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 58717.6us
I20260709 13:07:19.032008 22905 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:18.975811 22904 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.021s	sys 0.014s
I20260709 13:07:19.040608 24549 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3.
I20260709 13:07:19.034973 22829 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:19.032500 24562 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:18.987246 24634 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:19.059779 24634 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.041796 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:19.062192 22719 leader_election.cc:400] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4.
I20260709 13:07:19.040119 22650 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33010 (request call id 221) took 3443 ms. Trace:
I20260709 13:07:18.974045 24648 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:19.039913 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:19.063747 22719 leader_election.cc:403] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:19.064455 22822 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:18.977891 24544 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:19.065423 24545 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4.
I20260709 13:07:18.975191 24628 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:19.066341 24628 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:19.063673 22650 rpcz_store.cc:276] 0709 13:07:15.588768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.588887 (+   119us) service_pool.cc:224] Handling call
0709 13:07:19.032430 (+3443543us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:16.100027 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:16.100721 (+   694us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:16.100764 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:07:16.100790 (+    26us) tablet.cc:662] Decoding operations
0709 13:07:16.101184 (+   394us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:16.101250 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:16.101293 (+    43us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:07:16.101518 (+   225us) tablet.cc:701] Row locks acquired
0709 13:07:16.101542 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:07:16.101706 (+   164us) write_op.cc:270] Start()
0709 13:07:16.101831 (+   125us) write_op.cc:276] Timestamp: P: 1783602436101656 usec, L: 0
0709 13:07:16.101859 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:07:16.102644 (+   785us) log.cc:844] Serialized 533 byte log entry
0709 13:07:17.235559 (+1132915us) op_driver.cc:464] REPLICATION: finished
0709 13:07:18.129669 (+894110us) write_op.cc:301] APPLY: starting
0709 13:07:18.146205 (+ 16536us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:18.146324 (+   119us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:18.146471 (+   147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:18.148416 (+  1945us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:18.148592 (+   176us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:07:18.148636 (+    44us) write_op.cc:312] APPLY: finished
0709 13:07:18.151566 (+  2930us) log.cc:844] Serialized 23 byte log entry
0709 13:07:18.943261 (+791695us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:18.943352 (+    91us) write_op.cc:454] Released schema lock
0709 13:07:18.943392 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:18.951795 (+  8403us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:18.951832 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":259126,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":847064,"apply.run_cpu_time_us":30353,"apply.run_wall_time_us":841672,"num_ops":1,"prepare.queue_time_us":176378,"prepare.run_cpu_time_us":3039,"prepare.run_wall_time_us":5494,"replication_time_us":1123611,"thread_start_us":1612866,"threads_started":3,"wal-append.queue_time_us":761131}]]}
I20260709 13:07:19.068184 24628 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:19.046169 22897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.005s	sys 0.022s
W20260709 13:07:19.068712 22897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.006s	sys 0.022s
I20260709 13:07:18.986853 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:19.034552 22753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.009s	sys 0.019s
I20260709 13:07:19.071723 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1604) took 1580 ms. Trace:
I20260709 13:07:19.062705 24549 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:19.072428 24549 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.072052 22897 rpcz_store.cc:276] 0709 13:07:17.491581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.686131 (+194550us) service_pool.cc:224] Handling call
0709 13:07:17.686653 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.687197 (+   544us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:18.463822 (+776625us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.071443 (+607621us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.071488 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:19.071525 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.071579 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.071664 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2343,"prepare.queue_time_us":298114,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"thread_start_us":1295211,"threads_started":2}
I20260709 13:07:19.073371 24549 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:19.063863 24648 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.075999 24648 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:19.060786 24562 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.078194 24562 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:19.052225 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 489766us
I20260709 13:07:19.065068 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:19.053412 24616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.020s	sys 0.043s
W20260709 13:07:19.051998 24669 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:07:19.111824 23419 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:19.120500 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:19.121237 22750 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 1.
I20260709 13:07:19.122805 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:19.122915 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:19.123412 22750 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:19.112190 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:19.127403 22824 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:19.129231 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:19.114583 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:19.115000 22905 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:18.975319 24663 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:19.118515 22871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.009s	sys 0.012s
I20260709 13:07:19.129827 22751 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 1.
W20260709 13:07:19.132251 22871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.010s	sys 0.012s
W20260709 13:07:19.132579 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 640872us
I20260709 13:07:19.135793 22755 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:19.136152 22899 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:19.129652 24489 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:19.097599 24648 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:19.137535 24530 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:19.137897 24648 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.138346 24648 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:19.138312 24530 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.140801 24530 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:19.137455 24464 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.142822 24530 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:19.143317 24544 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:19.144043 24544 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:19.144634 22826 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:19.144851 24639 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:19.145076 24544 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:19.146512 24631 consensus_meta.cc:287] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.889s	user 0.010s	sys 0.000s
I20260709 13:07:19.146888 24631 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.148098 23444 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:19.148834 24631 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.149535 24530 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:19.152295 24499 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:19.115990 24616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.021s	sys 0.043s
I20260709 13:07:19.150794 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1602) took 1682 ms. Trace:
I20260709 13:07:19.148716 24648 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.045910 24643 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.155471 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:19.156774 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:19.157059 24545 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:19.157471 24545 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.157784 24545 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:19.160337 24541 consensus_meta.cc:287] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.001s
I20260709 13:07:19.115633 24562 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:19.160799 24629 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:19.161092 24562 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.163537 24562 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.164340 24651 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:19.157858 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:19.165748 24464 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:19.166113 24464 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:19.166486 24464 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:19.167083 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:19.166781 24654 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.000s
I20260709 13:07:19.174237 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:19.174916 22755 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 1.
I20260709 13:07:19.175081 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:19.155547 24628 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:19.175642 22751 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:19.181064 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:19.181634 22750 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 1.
I20260709 13:07:19.155598 22896 rpcz_store.cc:276] 0709 13:07:17.468701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.676925 (+208224us) service_pool.cc:224] Handling call
0709 13:07:17.677255 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:17.677525 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:17.677561 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:17.678037 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:07:18.603499 (+925462us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:18.603545 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.603633 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.150601 (+546968us) raft_consensus.cc:1685] finished
0709 13:07:19.150618 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.150740 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":923397,"threads_started":1,"wal-append.queue_time_us":1300956}
I20260709 13:07:19.172602 24654 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:19.171587 24489 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.166955 24560 tablet_bootstrap.cc:492] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:07:19.187036 22748 raft_consensus.cc:2368] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:19.175426 24616 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:07:19.194634 22868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.006s	sys 0.008s
W20260709 13:07:19.195062 22868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.007s	sys 0.008s
W20260709 13:07:19.186440 22903 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.005s	sys 0.010s
W20260709 13:07:19.197131 22903 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.005s	sys 0.010s
I20260709 13:07:19.133776 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:19.184064 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:19.161257 24629 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:19.230522 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 96919us
I20260709 13:07:19.160799 24541 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.229573 24632 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:19.229681 24634 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:19.229815 24629 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:19.229725 24553 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.011s	sys 0.023s
I20260709 13:07:19.186625 24499 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:19.267552 24545 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.288196 24420 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:19.200276 24675 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:19.339486 24545 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:19.342820 24420 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.344089 24420 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.345623 24634 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.306526 22751 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
W20260709 13:07:19.312119 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 129718us
I20260709 13:07:19.311100 24639 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:19.337482 24553 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.011s	sys 0.023s
I20260709 13:07:19.351171 24654 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:19.186465 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:19.347470 24634 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:19.187590 24518 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.013s	sys 0.002s
I20260709 13:07:19.430769 24545 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.438822 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 281853us
I20260709 13:07:19.441745 22823 raft_consensus.cc:2368] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:19.441857 24639 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.442977 24553 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:19.187376 24530 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.392278 24560 ts_tablet_manager.cc:1403] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 2.579s	user 0.033s	sys 0.018s
I20260709 13:07:19.272181 24661 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:19.186889 24489 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.311986 24616 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:19.488505 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 380) took 1034 ms. Trace:
I20260709 13:07:19.577952 24698 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:19.608776 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.719372 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:19.607769 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.000s	sys 0.019s
W20260709 13:07:19.837620 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.001s	sys 0.023s
I20260709 13:07:19.837677 22751 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
W20260709 13:07:19.485020 22902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.022s	sys 0.031s
W20260709 13:07:19.838446 22902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.027s	sys 0.035s
W20260709 13:07:19.489948 23452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.008s	sys 0.007s
W20260709 13:07:19.838874 23452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.016s	sys 0.009s
W20260709 13:07:19.772176 24507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.009s	sys 0.007s
W20260709 13:07:19.839290 22901 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.009s	sys 0.015s
W20260709 13:07:19.578668 23417 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.988s	user 0.042s	sys 0.060s
W20260709 13:07:19.839964 23417 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.249s	user 0.044s	sys 0.064s
W20260709 13:07:19.655843 22797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.001s	sys 0.021s
W20260709 13:07:19.841442 22797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.001s	sys 0.024s
I20260709 13:07:19.841197 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:19.841743 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 930849us
W20260709 13:07:19.806362 24614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.221s	user 0.009s	sys 0.046s
I20260709 13:07:19.716977 23419 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:19.727111 22871 leader_election.cc:400] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
W20260709 13:07:19.844190 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 127274us
I20260709 13:07:19.844265 22871 leader_election.cc:403] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:07:19.444443 24654 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:19.699857 24685 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.499s	user 0.000s	sys 0.001s
W20260709 13:07:19.699539 24687 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
W20260709 13:07:19.706270 24464 consensus_meta.cc:287] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.000s
W20260709 13:07:19.772933 22825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.000s	sys 0.014s
I20260709 13:07:19.671736 24541 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:19.869736 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1605) took 1105 ms. Trace:
I20260709 13:07:19.762276 24545 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:19.867272 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:19.867623 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:19.831915 24630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.012s	sys 0.018s
W20260709 13:07:19.866469 22825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.000s	sys 0.018s
I20260709 13:07:19.870076 22902 rpcz_store.cc:276] 0709 13:07:18.764250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.765231 (+   981us) service_pool.cc:224] Handling call
0709 13:07:18.766143 (+   912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.766323 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:18.781237 (+ 14914us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.869443 (+1088206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.869495 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:19.869533 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.869589 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.869688 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28190,"prepare.queue_time_us":178414,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"thread_start_us":1033302,"threads_started":1}
I20260709 13:07:19.870002 24464 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.883456 24464 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.868723 24420 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:19.884658 23444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 357) took 1509 ms (client timeout 1814 ms). Trace:
W20260709 13:07:19.885222 23444 rpcz_store.cc:269] 0709 13:07:18.375123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.376887 (+  1764us) service_pool.cc:224] Handling call
0709 13:07:19.884381 (+1507494us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b540024ca54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:19.884564 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1277852544}
I20260709 13:07:19.886677 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:19.887176 24660 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:19.887679 23444 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:07:19.715679 24518 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.025s	sys 0.010s
I20260709 13:07:19.733446 24634 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.840692 24503 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:19.728832 24643 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:19.831314 24544 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.003s	sys 0.003s
I20260709 13:07:19.833730 22900 rpcz_store.cc:276] 0709 13:07:18.453905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.454660 (+   755us) service_pool.cc:224] Handling call
0709 13:07:18.454910 (+   250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.455260 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.455298 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.554775 (+ 99477us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.049074 (+494299us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:19.049113 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.049171 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.488249 (+439078us) raft_consensus.cc:1685] finished
0709 13:07:19.488274 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.488437 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14004,"raft.queue_time_us":182569,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":51611,"thread_start_us":483748,"threads_started":1,"wal-append.queue_time_us":877487}
I20260709 13:07:19.351943 24632 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:19.918936 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:19.919633 23451 raft_consensus.cc:2368] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
I20260709 13:07:19.919437 24643 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:19.920261 24632 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.920375 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:19.704759 23445 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
W20260709 13:07:19.803213 24549 consensus_meta.cc:287] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.000s
I20260709 13:07:19.920766 24639 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:19.920897 23451 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:19.846957 24648 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:19.873977 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:19.921166 24549 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:19.920387 24643 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.921785 24620 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.922293 24620 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:19.922530 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:19.922734 24620 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.923200 24549 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:19.923333 24643 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.842141 23449 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:07:19.923679 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:19.924256 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:19.924631 24620 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:19.924543 23447 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:19.925973 24663 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:19.891793 24630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.012s	sys 0.019s
W20260709 13:07:19.839625 23416 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.017s	sys 0.002s
W20260709 13:07:19.926538 23416 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.017s	sys 0.002s
W20260709 13:07:19.926857 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 718588us
I20260709 13:07:19.927697 24695 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:19.839576 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 656) took 1806 ms. Trace:
I20260709 13:07:19.735148 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 1626) took 1444 ms. Trace:
I20260709 13:07:19.929565 22753 rpcz_store.cc:276] 0709 13:07:18.274020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.274124 (+   104us) service_pool.cc:224] Handling call
0709 13:07:18.274361 (+   237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.274789 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.274815 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.275264 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.072063 (+796799us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:07:19.072115 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.072197 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.717758 (+645561us) raft_consensus.cc:1685] finished
0709 13:07:19.717776 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.718048 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":132,"raft.queue_time_us":7100,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":146,"thread_start_us":695806,"threads_started":1,"wal-append.queue_time_us":869213}
I20260709 13:07:19.929476 23452 rpcz_store.cc:276] 0709 13:07:18.033414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.033534 (+   120us) service_pool.cc:224] Handling call
0709 13:07:18.038067 (+  4533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.038286 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:18.891928 (+853642us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.839265 (+947337us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.839318 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:19.839363 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.839424 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.839530 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":961178,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":31264,"raft.queue_time_us":77,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6069,"thread_start_us":1712532,"threads_started":2,"wal-append.queue_time_us":976185}
I20260709 13:07:19.708942 24629 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.931643 24690 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:19.932310 23452 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:19.933516 23453 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
W20260709 13:07:19.841909 24507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.014s	sys 0.012s
I20260709 13:07:19.883354 24675 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:19.872671 23450 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:19.888863 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:19.839555 22901 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.009s	sys 0.015s
W20260709 13:07:19.810073 22754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.000s	sys 0.020s
W20260709 13:07:19.845518 24614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.282s	user 0.009s	sys 0.046s
W20260709 13:07:19.910310 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 194325us
W20260709 13:07:19.715427 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 1.15981e+06us
I20260709 13:07:19.846350 24654 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:19.937800 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 379) took 1484 ms. Trace:
I20260709 13:07:19.945457 22903 rpcz_store.cc:276] 0709 13:07:18.453624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.453749 (+   125us) service_pool.cc:224] Handling call
0709 13:07:18.454028 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.454303 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.454329 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.589821 (+135492us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.197600 (+607779us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:19.197653 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.197739 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.937573 (+739834us) raft_consensus.cc:1685] finished
0709 13:07:19.937601 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.937739 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14842,"raft.queue_time_us":136115,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":33602,"thread_start_us":519308,"threads_started":1,"wal-append.queue_time_us":1260934}
W20260709 13:07:19.948704 22754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.004s	sys 0.026s
W20260709 13:07:19.933969 24544 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.003s	sys 0.003s
W20260709 13:07:19.953025 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 110762us
I20260709 13:07:19.954239 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:19.925175 24620 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:19.955415 22797 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:19.968200 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:19.933240 24629 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:19.974527 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1606) took 1210 ms. Trace:
I20260709 13:07:19.979722 22901 rpcz_store.cc:276] 0709 13:07:18.764580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.046233 (+281653us) service_pool.cc:224] Handling call
0709 13:07:19.111207 (+ 64974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.111394 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:07:19.129233 (+ 17839us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.163193 (+ 33960us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b18002c8fc0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b23a 00007fcb74e05a2a
0709 13:07:19.974212 (+811019us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.974264 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:19.974305 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.974372 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.974481 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32510,"prepare.queue_time_us":17615,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"spinlock_wait_cycles":11336576,"thread_start_us":778276,"threads_started":1}
I20260709 13:07:19.844770 24698 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.941681 24675 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:19.950299 24628 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:19.987165 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:19.977305 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:19.986929 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:19.950585 24614 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:19.938884 24503 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.952924 24648 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:19.999272 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44962.7us
I20260709 13:07:19.965260 24549 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:20.005764 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:19.980520 24507 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:20.003821 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:19.974202 24624 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:19.938944 24634 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.012213 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:20.013083 22871 leader_election.cc:400] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:20.013310 22871 leader_election.cc:403] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:07:20.013861 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 58532.9us
I20260709 13:07:19.991884 24695 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:20.014875 22797 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:20.014979 22868 leader_election.cc:400] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:19.936650 24553 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.020511 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:20.017614 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:20.021181 22797 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:20.022404 22753 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:20.018095 22868 leader_election.cc:400] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:20.024109 22868 leader_election.cc:403] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:20.026466 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:20.011798 24518 raft_consensus.cc:2764] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:07:20.030526 23416 leader_election.cc:400] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:19.976982 24620 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.034186 24690 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:20.034659 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 244) took 1351 ms. Trace:
I20260709 13:07:20.034868 24690 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:20.035377 23416 leader_election.cc:403] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:20.035058 22825 rpcz_store.cc:276] 0709 13:07:18.683000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.978408 (+295408us) service_pool.cc:224] Handling call
0709 13:07:18.978704 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.978982 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.979009 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.981248 (+  2239us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.173157 (+191909us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b18001f7ec0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:07:19.880749 (+707592us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:19.880786 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.880844 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.034360 (+153516us) raft_consensus.cc:1685] finished
0709 13:07:20.034381 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.034577 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41873,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":11736,"spinlock_wait_cycles":6249344,"thread_start_us":707440,"threads_started":1,"wal-append.queue_time_us":979933}
I20260709 13:07:20.037243 24482 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:20.014086 24420 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:20.040258 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:20.040868 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:20.045928 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 77832.1us
I20260709 13:07:20.046406 22750 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:20.042398 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:20.043041 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:20.048210 22749 raft_consensus.cc:2410] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:07:20.043689 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:19.974619 24616 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.050299 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:20.051695 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:20.052354 23452 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.054273 22752 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:07:20.054705 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.056088 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.056277 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:20.056819 22751 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 1.
I20260709 13:07:20.059271 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:20.059805 22750 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:20.035565 24719 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:20.060883 24719 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.062119 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:20.062674 22748 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:20.046689 23451 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:20.061791 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:20.072468 22749 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 1.
I20260709 13:07:20.016443 24704 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:20.066481 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:20.019753 24708 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:20.061717 23419 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:20.076279 24708 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.071213 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.017026 24530 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:20.016259 24624 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.011600 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:20.114975 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 61251.2us
I20260709 13:07:20.129076 23419 leader_election.cc:400] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:20.129485 23419 leader_election.cc:403] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:07:20.020056 24507 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:20.009204 24648 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.141172 22652 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 334) took 4534 ms. Trace:
I20260709 13:07:20.141304 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:20.142894 22753 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
W20260709 13:07:20.114902 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 84518.7us
I20260709 13:07:20.127012 22653 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 3 to 4, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:20.135665 24662 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.137748 24702 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:20.139806 24518 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:20.011073 24549 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:19.991448 24696 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:20.030977 24632 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:20.022948 24689 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:20.154090 23452 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:20.045249 24694 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:20.170828 22750 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:20.113103 24521 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:07:20.174180 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 158858us
I20260709 13:07:20.158670 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:20.153048 22748 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:20.183578 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.141954 24633 raft_consensus.cc:2764] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:07:20.112885 24639 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:20.141049 24544 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:20.131110 24697 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:20.023947 24695 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.146548 24702 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:20.177066 24651 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:20.197530 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:20.199293 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 70345.9us
I20260709 13:07:20.194183 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:20.172873 24715 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:07:20.186884 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 120875us
I20260709 13:07:20.141719 22652 rpcz_store.cc:276] 0709 13:07:15.592478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.593415 (+   937us) service_pool.cc:224] Handling call
0709 13:07:20.126515 (+4533100us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:16.103049 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:16.103497 (+   448us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:16.103520 (+    23us) write_op.cc:435] Acquired schema lock
0709 13:07:16.103537 (+    17us) tablet.cc:662] Decoding operations
0709 13:07:16.103821 (+   284us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:16.103860 (+    39us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:16.103887 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:07:16.109923 (+  6036us) tablet.cc:701] Row locks acquired
0709 13:07:16.109937 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:07:16.110054 (+   117us) write_op.cc:270] Start()
0709 13:07:16.110153 (+    99us) write_op.cc:276] Timestamp: P: 1783602436110015 usec, L: 0
0709 13:07:16.110169 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:07:16.110662 (+   493us) log.cc:844] Serialized 538 byte log entry
0709 13:07:17.987526 (+1876864us) op_driver.cc:464] REPLICATION: finished
0709 13:07:18.969114 (+981588us) write_op.cc:301] APPLY: starting
0709 13:07:18.971346 (+  2232us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:18.981531 (+ 10185us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:19.014515 (+ 32984us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:19.045178 (+ 30663us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:19.079681 (+ 34503us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:07:19.093444 (+ 13763us) write_op.cc:312] APPLY: finished
0709 13:07:19.117921 (+ 24477us) log.cc:844] Serialized 23 byte log entry
0709 13:07:19.118147 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:19.118302 (+   155us) write_op.cc:454] Released schema lock
0709 13:07:19.118351 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:19.118745 (+   394us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:19.118783 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":259616,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":958118,"apply.run_cpu_time_us":2195,"apply.run_wall_time_us":169905,"num_ops":1,"prepare.queue_time_us":179520,"prepare.run_cpu_time_us":1982,"prepare.run_wall_time_us":7833,"replication_time_us":1877238,"thread_start_us":793592,"threads_started":1}]]}
W20260709 13:07:20.212478 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 53882.8us
I20260709 13:07:20.213491 23416 leader_election.cc:400] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:20.188773 24544 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:20.190145 24639 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:20.191092 24521 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:20.132361 24614 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.170879 24518 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:20.234759 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 213818us
I20260709 13:07:20.175285 24675 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.207806 24715 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.194756 24702 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.206566 24661 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:20.243820 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:20.232084 24544 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:20.174168 24696 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.233165 24694 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:20.232700 24639 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:20.249607 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:20.249393 22797 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:20.237130 24715 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.238436 24651 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:20.257627 22747 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:20.234668 24704 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:20.251781 24723 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:20.259765 24624 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.259734 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:20.260438 24707 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
I20260709 13:07:20.261058 22752 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:20.302978 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:20.264186 24722 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:20.265630 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:20.265983 24549 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.315330 22825 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.266686 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:20.267115 24545 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.267709 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.269378 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 237) took 1637 ms. Trace:
I20260709 13:07:20.336911 22823 rpcz_store.cc:276] 0709 13:07:18.631951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.632667 (+   716us) service_pool.cc:224] Handling call
0709 13:07:20.269173 (+1636506us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54001efad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:20.269282 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245263744}
I20260709 13:07:20.269263 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:20.265998 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:20.255664 24541 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:20.270447 24717 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:20.270802 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:20.271020 24634 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.271936 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:20.273341 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:20.273648 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:20.343410 22830 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:20.300604 24489 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:20.343947 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 334275us
I20260709 13:07:20.273886 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 362) took 1289 ms. Trace:
I20260709 13:07:20.274597 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:20.275200 24665 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.277201 24728 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:20.277604 24712 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.263s	user 0.002s	sys 0.000s
I20260709 13:07:20.279371 24696 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.216917 24507 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.279798 24733 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:20.280270 24651 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.269910 24698 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.284106 23448 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:20.284874 24633 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:20.286008 24521 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.275215 24628 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:20.287969 24503 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.288183 24648 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.253014 24718 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:20.379483 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:20.380012 22901 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:20.289927 24620 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.290675 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:20.290944 24708 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.291515 24629 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:20.297410 24560 raft_consensus.cc:359] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 3 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.298355 24694 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:20.299389 24727 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.262624 22750 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:20.314790 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:20.315227 24722 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.317346 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:20.317802 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 123691us
I20260709 13:07:20.321111 24731 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:20.322566 24730 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:20.323354 24695 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:20.300276 24705 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.271s	user 0.000s	sys 0.002s
I20260709 13:07:20.327977 24518 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.333971 22825 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:20.334771 24675 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.341547 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:20.341765 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1603) took 2851 ms. Trace:
I20260709 13:07:20.390617 24629 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:20.345253 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 135926us
I20260709 13:07:20.351266 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:20.420882 22823 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:20.316804 24676 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:20.358222 24420 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:20.425747 24420 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.361896 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:20.424867 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 77100.9us
I20260709 13:07:20.346206 24530 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:20.365721 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:20.349573 23445 rpcz_store.cc:276] 0709 13:07:18.984808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.986221 (+  1413us) service_pool.cc:224] Handling call
0709 13:07:20.273704 (+1287483us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400729a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:20.273813 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355737984}
I20260709 13:07:20.366555 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:20.367302 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:20.367448 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.368891 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.298797 24715 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:20.388092 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 97486.1us
W20260709 13:07:20.388736 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 70248.6us
I20260709 13:07:20.364531 24634 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:20.380463 22822 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:20.374620 24714 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.393402 22829 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.345762 24717 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.376543 24665 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:07:20.387805 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.386543 24733 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:20.391541 24622 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:20.416249 22904 rpcz_store.cc:276] 0709 13:07:17.486664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.685442 (+198778us) service_pool.cc:224] Handling call
0709 13:07:17.685782 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:17.813505 (+127723us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:18.317595 (+504090us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.070638 (+753043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.070709 (+    71us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:19.070767 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.337648 (+1266881us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54003e9254. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:20.337693 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.337798 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":172383,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":12904,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":96,"spinlock_wait_cycles":293123328,"thread_start_us":1200560,"threads_started":2,"wal-append.queue_time_us":862890}
I20260709 13:07:20.313375 24723 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:20.402612 24727 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.385653 24718 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.429821 22903 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:20.497306 22900 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:20.497519 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.497012 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:20.498221 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68252.1us
I20260709 13:07:20.499142 22901 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:20.500789 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:20.500808 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.366441 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:20.502766 22896 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:20.428263 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.386111 24628 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.375268 24728 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.501771 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.513818 22830 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:20.503727 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.503099 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.525110 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:20.525835 22897 raft_consensus.cc:2368] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:20.506721 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.526674 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.527292 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.528752 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.528977 22897 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:20.527953 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.528110 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:20.525537 22904 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:20.525385 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.525753 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.531646 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:20.532591 22904 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:20.533701 22900 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:20.533878 22904 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:20.535326 24420 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.535739 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:20.536715 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.417196 24736 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:20.425442 24629 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:20.434782 24695 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.488561 24689 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:20.405205 24541 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:20.404762 24731 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.496232 24722 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.396062 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 349) took 1220 ms. Trace:
I20260709 13:07:20.427670 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:20.523712 22899 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:20.546398 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.970496 23444 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:20.524226 24464 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:20.972582 24464 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:20.405094 24544 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.483801 24560 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:07:20.985337 24648 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.986212 24648 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.986814 24718 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.986435 24560 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:20.953765 24734 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:20.988361 24734 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.988965 24648 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.500762 24622 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:20.989827 24622 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:20.990247 24464 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.563220 24714 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.638151 24733 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.588191 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:20.544641 24482 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:20.470221 24690 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:20.994053 22823 consensus_meta.cc:287] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.568s	user 0.017s	sys 0.002s
I20260709 13:07:20.592344 24628 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.523918 24553 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:21.000406 24690 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:21.001058 24553 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:21.002177 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 361) took 2017 ms. Trace:
I20260709 13:07:20.399729 24704 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.404094 24730 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.541687 24736 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.041684 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.042063 24748 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:21.019920 22823 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:20.563304 24530 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:20.496551 24694 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.050360 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:20.431756 24518 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:20.461794 24623 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:21.061906 24639 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.756s	user 0.011s	sys 0.006s
I20260709 13:07:20.563088 24702 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:20.930811 22905 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:20.932154 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:20.915325 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 377959us
I20260709 13:07:20.938304 24715 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.938179 24727 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:20.940872 22747 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:07:20.681327 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 268303us
W20260709 13:07:20.616024 24742 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:07:20.963272 24728 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:20.430830 24732 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.082252 24732 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.084507 24622 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.964480 22755 rpcz_store.cc:276] 0709 13:07:19.176066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.176184 (+   118us) service_pool.cc:224] Handling call
0709 13:07:19.176444 (+   260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.176617 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.176640 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.183117 (+  6477us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.868152 (+685035us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:19.868192 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.868249 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.277090 (+408841us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:20.395776 (+118686us) raft_consensus.cc:1685] finished
0709 13:07:20.395796 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.396001 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8790912,"thread_start_us":151661,"threads_started":1,"wal-append.queue_time_us":782326}
I20260709 13:07:20.968043 24703 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:20.968628 22829 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
W20260709 13:07:20.965853 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 538158us
I20260709 13:07:20.995090 24482 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:20.432758 24676 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:20.617878 24723 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.989176 24560 ts_tablet_manager.cc:1434] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 1.155s	user 0.010s	sys 0.000s
I20260709 13:07:21.002749 24632 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.002687 23448 rpcz_store.cc:276] 0709 13:07:18.984471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.988566 (+  4095us) service_pool.cc:224] Handling call
0709 13:07:21.001678 (+2013112us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54006dea54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:21.001945 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2875646720}
I20260709 13:07:21.038432 24745 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:07:21.003115 24654 consensus_meta.cc:287] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.029s	user 0.007s	sys 0.015s
I20260709 13:07:21.016913 24734 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.003334 24489 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.002645 24553 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.129020 24654 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.129643 24622 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:21.060760 24695 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:20.535207 24634 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:20.592453 24752 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:07:21.042910 24748 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.065197 24623 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.067166 22902 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:21.076077 22826 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:21.042071 24736 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:21.107942 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.085096 24732 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:21.126627 24560 ts_tablet_manager.cc:1467] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Tablet startup took 1290ms
I20260709 13:07:21.135244 24745 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:21.157914 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:21.158270 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45968.9us
I20260709 13:07:21.138163 24703 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:21.050416 24541 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.163635 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.121841 24482 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.166714 23446 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 2: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:21.167294 24503 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.167953 24503 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.172206 24518 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.174320 24632 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.175097 24731 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:21.165148 23447 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 2: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:21.146863 24639 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.181506 24730 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.181825 22756 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:21.183285 22903 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:21.185352 24623 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.162447 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.175391 24518 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.192725 24544 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:21.178205 24489 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.196959 24489 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:21.176592 24632 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.173302 24719 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.173404 24717 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.178032 24503 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:21.202756 24634 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.203023 24702 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:21.204062 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:21.199141 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:21.181502 24639 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:21.247356 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 93125us
I20260709 13:07:21.253294 24695 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.254112 24518 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:21.254508 24518 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.254743 24518 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:21.258744 22748 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:21.200739 24703 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.278240 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:21.278942 22751 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:21.279835 22755 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:21.257558 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:21.283131 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:21.283849 22749 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.257658 24748 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:21.290419 24702 raft_consensus.cc:2764] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:07:21.295307 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 110087us
W20260709 13:07:21.295109 24629 consensus_meta.cc:287] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.752s	user 0.020s	sys 0.021s
W20260709 13:07:21.305699 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 78611.7us
W20260709 13:07:21.307024 23446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 363) took 1469 ms (client timeout 1723 ms). Trace:
W20260709 13:07:21.307474 23446 rpcz_store.cc:269] 0709 13:07:19.837952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.840876 (+  2924us) service_pool.cc:224] Handling call
0709 13:07:20.267560 (+426684us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:07:21.306735 (+1039175us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54007d0fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:21.306958 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835584512}
I20260709 13:07:21.312674 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.322494 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.305721 24676 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:21.318794 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.312958 24544 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:07:21.318717 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:21.144608 24560 ts_tablet_manager.cc:1469] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Trace:
0709 13:07:15.149097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.149239 (+   142us) service_pool.cc:224] Handling call
0709 13:07:15.316896 (+167657us) ts_tablet_manager.cc:934] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
0709 13:07:15.910420 (+593524us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:07:15.953734 (+ 43314us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:15.960886 (+  7152us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:07:16.037070 (+ 76184us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:07:16.041100 (+  4030us) tablet_replica.cc:206] Creating consensus instance
0709 13:07:16.080803 (+ 39703us) ts_tablet_manager.cc:989] Replica registered.
0709 13:07:16.081634 (+   831us) inbound_call.cc:177] Queueing success response
0709 13:07:16.813039 (+731405us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:07:19.836046 (+3023007us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:07:20.037496 (+201450us) tablet_replica.cc:261] Starting instrumentation
0709 13:07:20.199353 (+161857us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":34469,"mutex_wait_us":20601,"raft.queue_time_us":7115,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":160,"tablet-copy.queue_time_us":43520,"tablet-download-pool.queue_time_us":147275,"tablet-download-pool.run_cpu_time_us":1310,"tablet-download-pool.run_wall_time_us":32839,"thread_start_us":118943,"threads_started":2}
I20260709 13:07:21.333564 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.334443 23446 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:21.304522 24629 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.331111 22749 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:21.340021 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:21.321228 24482 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.348886 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.350203 22751 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:21.328099 24745 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.352617 24499 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.357937 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 1648) took 1069 ms. Trace:
I20260709 13:07:21.360174 23419 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.358625 24745 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.361191 24464 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:21.361307 24499 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.359741 22756 rpcz_store.cc:276] 0709 13:07:20.288912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.302013 (+ 13101us) service_pool.cc:224] Handling call
0709 13:07:20.312478 (+ 10465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.182317 (+869839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.352875 (+170558us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b5400045454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:21.357761 (+  4886us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998538112}
I20260709 13:07:21.363907 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.365904 24482 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.366904 24695 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.367799 24695 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.370003 24622 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:21.361835 24464 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:21.374229 24420 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:21.351989 24676 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.374270 24622 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:21.390069 24676 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.377797 24482 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.391563 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 390) took 1013 ms. Trace:
I20260709 13:07:21.428071 22871 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:21.426143 22897 rpcz_store.cc:276] 0709 13:07:20.377715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.531605 (+153890us) service_pool.cc:224] Handling call
0709 13:07:20.531874 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.167196 (+635322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:21.167267 (+    71us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:21.167320 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:21.167359 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.391193 (+223834us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b54001b7454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:21.391259 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.391397 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1442692352}
I20260709 13:07:21.431200 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.373529 24703 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:21.436009 24772 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:07:21.438144 24622 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.442363 24499 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:21.370021 24518 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.442389 24676 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.448215 24518 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.446658 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.446399 24530 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:21.431984 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.453168 23448 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:21.452497 24530 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.446012 24464 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.454483 24482 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:21.446593 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 397) took 1043 ms. Trace:
I20260709 13:07:21.454509 24530 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:21.456172 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.451850 24695 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:21.455950 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:21.455664 24753 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.416s	user 0.001s	sys 0.001s
I20260709 13:07:21.455718 22903 rpcz_store.cc:276] 0709 13:07:20.403290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.525545 (+122255us) service_pool.cc:224] Handling call
0709 13:07:21.446404 (+920859us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400059bd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:21.446542 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1468850048}
I20260709 13:07:21.461849 24634 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:21.472136 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 143737us
I20260709 13:07:21.471788 24420 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:21.473088 24420 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.473603 24420 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:21.473636 24777 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:07:21.475188 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:21.476831 24662 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.008s	sys 0.007s
I20260709 13:07:21.476974 24518 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:07:21.478967 24662 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.008s	sys 0.007s
I20260709 13:07:21.485261 22750 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:21.485857 22753 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:21.486910 22765 tablet_copy_service.cc:342] P 6a5191aaff76469bbf6c4735dc9e6509: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-4d8a8f3cf129492eb679da24703f5b84 received from {username='slave'} at 127.0.0.1:56780
I20260709 13:07:21.493031 22765 tablet_copy_service.cc:434] P 6a5191aaff76469bbf6c4735dc9e6509: ending tablet copy session c45e1805f049467697a7358fb566de2f-4d8a8f3cf129492eb679da24703f5b84 on tablet 4d8a8f3cf129492eb679da24703f5b84 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.491981 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:21.495276 22756 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.492216 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.498257 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.499365 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:21.500751 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:21.485882 24420 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.504763 24507 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:21.505184 24507 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.505409 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.506213 23446 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:21.506726 24693 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:21.505625 24507 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:21.501729 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.479470 24629 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:21.478453 24545 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.515602 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:21.515748 22756 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:21.518204 22871 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:21.526749 24420 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.527791 24695 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:21.528239 24695 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.527904 24545 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.527798 24518 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:21.529409 24518 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.529749 24518 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:21.530828 24545 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.528676 24695 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:21.540340 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.536352 24690 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:21.544004 24779 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:21.543166 24420 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:21.536810 24634 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.546182 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.547089 24420 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.548744 24420 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:21.550127 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49452.5us
I20260709 13:07:21.549899 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.542112 24544 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:07:21.558916 24629 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:21.557153 24634 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.556599 24676 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:21.562463 24634 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:21.576647 24754 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.646s	user 0.002s	sys 0.002s
I20260709 13:07:21.577967 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:21.578816 23448 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:21.579322 22903 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:07:21.580687 24746 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.634s	user 0.000s	sys 0.002s
I20260709 13:07:21.587766 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.556792 24622 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:21.598851 24482 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:21.602308 23453 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:21.605593 23453 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:21.609638 24616 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:21.610541 24616 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:21.610648 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1297) took 1336 ms. Trace:
I20260709 13:07:21.610953 24616 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:21.611678 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:21.591282 24764 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.267s	user 0.003s	sys 0.000s
I20260709 13:07:21.612135 24616 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.615056 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:21.616686 22748 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:21.617995 22753 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
W20260709 13:07:21.618170 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.619031 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 452550us
W20260709 13:07:21.621806 22747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 1640) took 1362 ms (client timeout 1642 ms). Trace:
W20260709 13:07:21.622212 22747 rpcz_store.cc:269] 0709 13:07:20.259421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.259529 (+   108us) service_pool.cc:224] Handling call
0709 13:07:21.621597 (+1362068us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54003f11d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:21.621740 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861058944}
I20260709 13:07:21.623638 24639 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.624272 24639 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.624975 22749 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:21.626288 24639 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.626657 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.628391 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.630098 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:21.630551 24518 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.632623 24518 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.629724 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.633347 22756 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:21.633999 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:21.611276 22826 rpcz_store.cc:276] 0709 13:07:20.274017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.317136 (+ 43119us) service_pool.cc:224] Handling call
0709 13:07:21.610406 (+1293270us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54002c9054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:21.610570 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709986560}
I20260709 13:07:21.623811 22750 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:21.638140 24616 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:21.639003 23446 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:21.639627 24654 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:21.642727 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:21.646651 22749 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:21.649804 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:21.650688 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 63090.7us
I20260709 13:07:21.650318 24420 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.624277 24694 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:21.654312 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:21.645970 24622 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:21.660521 24420 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.657938 24694 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:21.662446 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:21.663722 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.664266 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.665112 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.665951 24518 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:21.666632 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.667462 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.652691 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:21.664330 24714 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:21.670290 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.675143 22829 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:21.677533 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.677753 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.658538 24719 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:21.682250 24719 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:21.685006 22902 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:21.683229 24719 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.687686 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:21.688504 22830 raft_consensus.cc:2368] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4.
I20260709 13:07:21.689019 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:21.687182 24654 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:21.649290 24507 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.697886 22897 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:21.689966 23447 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:21.707924 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.708439 22826 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:21.697252 24654 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.711427 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:21.711558 23444 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 3: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:21.711972 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:21.712100 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.713012 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.713282 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.714519 22829 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:21.689087 24622 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.721186 22750 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:21.726310 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.726872 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.727231 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.727718 22901 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:21.728389 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:21.729069 22900 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:21.729817 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.730237 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.730932 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.731499 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.731823 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.731536 22904 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:21.733119 22751 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:21.733594 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.726401 22899 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:21.734179 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:21.719272 22794 connection.cc:256] RPC call timeout handler was delayed by -2.6734335890000693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 13:07:21.739418 22794 connection.cc:256] RPC call timeout handler was delayed by -2.4934335890000057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
I20260709 13:07:21.715706 24654 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:21.746753 23449 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 2: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:21.699178 24695 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.754339 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.754282 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.748148 22903 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:21.734629 23446 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:21.762291 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.761878 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.769761 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.772185 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.774070 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:21.775501 23452 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:21.782397 22756 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:21.784422 22794 connection.cc:256] RPC call timeout handler was delayed by -2.2900335890000179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
I20260709 13:07:21.785370 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.786238 24722 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:21.786446 22828 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:21.786865 24722 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:07:21.788116 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.788278 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.788892 24722 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:21.790247 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.790552 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.790571 22751 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:21.791376 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:21.790752 23445 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:21.786885 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:21.794101 22896 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:21.783514 24420 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:21.791447 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.795135 24420 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.797201 24420 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:21.798269 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.800799 22898 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:21.803180 22903 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:21.803540 23450 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:21.806425 22753 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:21.806684 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.807639 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.807806 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:21.784729 22794 connection.cc:256] RPC call timeout handler was delayed by -2.2531335890000719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
I20260709 13:07:21.804738 24530 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.810413 22747 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:21.810619 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.811676 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.810987 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.810472 22829 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
W20260709 13:07:21.814829 22794 connection.cc:256] RPC call timeout handler was delayed by -1.3370976759999849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 13:07:21.815337 22794 connection.cc:256] RPC call timeout handler was delayed by -1.323597675999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:07:21.815704 22794 connection.cc:256] RPC call timeout handler was delayed by -1.2506976759999588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 13:07:21.812458 22830 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:21.816553 22905 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:21.819192 22828 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:21.819799 24530 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:21.817129 24722 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.820796 24722 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.818110 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.829676 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.830533 22828 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:21.809808 23448 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:21.836375 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.837277 22902 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:21.840793 22903 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:21.761828 24694 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.828426 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.845727 24733 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:21.850538 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.852207 24697 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:07:21.852568 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.854925 22899 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 1.
I20260709 13:07:21.855159 24530 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:21.857439 24714 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:21.868561 23416 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:21.872290 24620 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:21.872730 24620 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:21.873288 24620 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:21.876374 23447 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:21.877405 23453 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:21.878248 24722 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:21.849742 24780 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
W20260709 13:07:21.882328 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69546us
I20260709 13:07:21.881945 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:21.879825 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:21.883478 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.883683 22904 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:21.884126 22896 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:21.834306 24634 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:07:21.898427 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.879738 24530 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:21.904145 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:21.906076 23452 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:21.908182 22830 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:21.909184 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:21.909737 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.910904 22824 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:07:21.912202 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 63917.4us
I20260709 13:07:21.912674 24730 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:21.913879 24695 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:21.914776 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.915215 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:21.915692 22794 leader_election.cc:400] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
I20260709 13:07:21.915823 22902 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:21.916013 22794 leader_election.cc:403] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:21.916289 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:21.916606 24730 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:21.917572 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:21.918206 24639 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:21.918391 22896 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:21.883831 23446 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:21.918965 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.920343 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:21.909740 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:21.917690 24545 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:21.922191 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.922713 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:21.917606 24730 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.879663 24716 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:07:21.925374 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.917263 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:21.926604 24530 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.927287 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.882519 24507 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:21.912267 22749 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:21.929422 23449 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:21.931516 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.948062 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:21.950577 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:21.966621 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.968004 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.968848 22753 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:21.969578 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:21.970665 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.970749 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.971369 22825 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:21.971485 22822 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:21.904805 24694 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:21.913305 24733 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:21.979831 24420 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:21.966233 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.978435 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:21.980217 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 64734.1us
I20260709 13:07:21.967175 22830 raft_consensus.cc:2376] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3. Re-sending same reply.
I20260709 13:07:21.979974 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.977420 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:21.952822 24545 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:21.981978 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:21.953442 24629 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.973052 24620 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.992574 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:21.953714 24695 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:22.001889 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:22.003855 24420 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:22.003957 22826 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:07:22.004681 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.967450 24654 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:21.997591 24622 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:21.991079 24544 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:07:22.008899 24622 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:22.009063 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:22.009395 24622 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.009680 22830 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:22.010223 24420 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:21.980162 24482 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:07:22.011930 23416 leader_election.cc:336] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.557s (SENT)
I20260709 13:07:22.012076 24420 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:22.012607 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.013505 24609 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:07:22.020443 22871 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:22.002303 24733 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:22.000720 24676 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:22.000186 24695 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:22.029201 24482 consensus_queue.cc:1243] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:07:22.002748 24545 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:22.031850 24420 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:22.032687 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:22.000468 24616 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:07:21.998380 24629 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.056038 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:22.062048 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:22.035348 24695 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.034416 24731 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:22.078023 24420 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:22.065030 24545 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.067028 24704 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:07:22.067602 24482 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:22.070600 24731 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.072028 24676 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:22.089751 24420 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:22.086510 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:22.093211 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:22.080618 24733 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:22.083567 24482 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:22.095793 24482 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:22.109019 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 180673us
I20260709 13:07:22.089047 24731 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:22.082504 24518 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:07:22.138978 24420 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.140542 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:22.088263 24676 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:22.087231 24622 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.155457 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:22.142899 24544 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:22.171027 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:22.170949 24420 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:22.159538 24544 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:22.180125 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48070.3us
I20260709 13:07:22.178917 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:22.188040 24723 raft_consensus.cc:2764] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:07:22.189397 24795 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:07:22.190392 24797 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:07:22.192180 24482 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.181584 24544 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:22.213337 22826 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:22.180488 24676 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.244956 22653 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 330) took 6327 ms. Trace:
I20260709 13:07:22.246610 24609 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
W20260709 13:07:22.247689 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 77469.1us
I20260709 13:07:22.250226 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:22.209393 24733 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:22.274374 24794 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.368s	user 0.003s	sys 0.000s
W20260709 13:07:22.274699 24802 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:07:22.275341 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:22.295255 22754 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:22.295917 22828 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:22.283995 24654 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:22.284389 24808 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.263757 24811 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.308143 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:22.306993 24639 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:22.332931 24811 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.325301 24482 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:22.331436 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:22.378683 24799 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:22.379047 24799 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:22.379386 24799 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:22.382704 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 74949.3us
I20260709 13:07:22.348532 24741 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:07:22.349691 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 76946.7us
I20260709 13:07:22.359531 24622 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:22.362627 24731 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.363111 24632 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:22.364248 24620 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:22.364068 24722 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:22.330721 24808 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.346354 24723 consensus_queue.cc:1059] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:07:22.353097 24544 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.370762 24676 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:22.373220 24811 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:22.333400 24609 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:22.385429 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:22.439687 22747 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:07:22.439821 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 54483.6us
I20260709 13:07:22.440546 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:22.441728 24817 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:22.442512 24817 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:22.435417 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:22.437817 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:22.447141 22752 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:22.248813 22653 rpcz_store.cc:276] 0709 13:07:15.917815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.924019 (+  6204us) service_pool.cc:224] Handling call
0709 13:07:22.244743 (+6320724us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:20.299300 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:20.299858 (+   558us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:20.299907 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:07:20.299924 (+    17us) tablet.cc:662] Decoding operations
0709 13:07:20.300339 (+   415us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:20.300393 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:20.300422 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:07:20.300606 (+   184us) tablet.cc:701] Row locks acquired
0709 13:07:20.300626 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:07:20.300758 (+   132us) write_op.cc:270] Start()
0709 13:07:20.300881 (+   123us) write_op.cc:276] Timestamp: P: 1783602440300712 usec, L: 0
0709 13:07:20.300899 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:07:20.311222 (+ 10323us) log.cc:844] Serialized 538 byte log entry
0709 13:07:21.579580 (+1268358us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:21.780141 (+200561us) op_driver.cc:464] REPLICATION: finished
0709 13:07:22.012561 (+232420us) write_op.cc:301] APPLY: starting
0709 13:07:22.012909 (+   348us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:22.013176 (+   267us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:22.013212 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:22.013552 (+   340us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:22.013786 (+   234us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:07:22.013834 (+    48us) write_op.cc:312] APPLY: finished
0709 13:07:22.014150 (+   316us) log.cc:844] Serialized 23 byte log entry
0709 13:07:22.014985 (+   835us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:22.015080 (+    95us) write_op.cc:454] Released schema lock
0709 13:07:22.015137 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:22.064775 (+ 49638us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:22.064813 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":257468,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":198095,"apply.run_cpu_time_us":2223,"apply.run_wall_time_us":50885,"num_ops":1,"prepare.queue_time_us":126992,"prepare.run_cpu_time_us":31082,"prepare.run_wall_time_us":101285,"raft.queue_time_us":120061,"raft.run_cpu_time_us":13815,"raft.run_wall_time_us":192312,"replication_time_us":1419097,"spinlock_wait_cycles":367814912,"thread_start_us":259420,"threads_started":4,"wal-append.queue_time_us":187992}]]}
I20260709 13:07:22.381206 22654 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 3 to 4, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 4 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:22.442529 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:22.387094 24632 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:07:22.462942 23416 leader_election.cc:336] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.087s (SENT)
I20260709 13:07:22.452637 22754 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:22.465408 24609 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.465747 22751 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:22.465811 24808 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:22.465909 24609 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.466660 22748 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:22.467873 22748 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:22.468016 24609 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:22.472003 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:22.472851 24811 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:22.473358 24811 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:22.474022 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:22.474267 24811 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.475078 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:22.475662 22749 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:22.476764 22749 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:07:22.480412 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:22.480926 22903 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:22.481225 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:22.481769 22900 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:22.483162 24639 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.483652 22654 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:22.485263 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:22.485347 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:22.486769 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:22.490309 24629 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:22.490377 24731 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:22.485294 22654 catalog_manager.cc:5697] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:22.492028 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:22.491485 24632 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.495682 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:22.495997 22900 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:22.496409 22749 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:22.500141 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:22.465451 24817 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:07:22.503805 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 119825us
I20260709 13:07:22.488544 24545 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:22.508416 22654 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 3, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:22.499235 24731 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:22.517616 22749 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
W20260709 13:07:22.511437 23416 leader_election.cc:336] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.197s (SENT)
I20260709 13:07:22.519897 22871 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:07:22.522497 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.518711 24731 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.523265 24731 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:22.509178 24808 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:22.504732 24733 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:22.525534 24808 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:22.526033 24808 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:22.525732 22654 catalog_manager.cc:5697] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 3 to 4, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:22.527364 22654 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:22.509061 24545 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:22.530316 24545 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:22.529924 24530 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:22.534510 24530 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:22.535429 24530 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.507328 24817 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:22.540165 22903 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:22.537519 23416 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:22.543574 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 51603.9us
I20260709 13:07:22.546005 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:07:22.548100 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 299966us
I20260709 13:07:22.498489 24695 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:22.538030 23447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.015s	sys 0.015s
I20260709 13:07:22.537956 24632 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:22.554538 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.559887 22752 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
W20260709 13:07:22.555080 23447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.015s	sys 0.015s
I20260709 13:07:22.565827 22897 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 2: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:22.551753 24733 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:22.553092 24714 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:22.549135 24817 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:22.569337 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.576340 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:22.565089 24693 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:07:22.581830 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 162060us
I20260709 13:07:22.590310 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:22.562604 24544 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:22.602067 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:22.602762 22903 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:22.577137 24733 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:22.562768 24716 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:07:22.566977 24662 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:07:22.593452 24714 raft_consensus.cc:697] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:22.577404 24654 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:07:22.596061 24697 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:22.626681 24798 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:22.628374 24545 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.630103 22896 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:22.630645 23449 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:22.607784 24733 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:22.643319 24819 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:07:22.653499 22903 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
W20260709 13:07:22.659286 22868 leader_election.cc:336] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.780s (SENT)
I20260709 13:07:22.655989 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:22.714488 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:22.657385 24798 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.729413 24629 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:22.752789 24798 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:22.753038 24629 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:22.754305 24629 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.733981 24722 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:22.734699 22897 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:22.751817 24609 raft_consensus.cc:3055] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:22.758895 24609 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:22.759351 24545 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:22.752703 24821 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:22.759925 24821 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:22.759972 24609 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.760844 24821 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.761582 24714 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.762724 24609 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.762076 24821 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.702847 24697 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:22.771414 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 112151us
W20260709 13:07:22.770915 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 242897us
W20260709 13:07:22.784178 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.733141 24817 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.801240 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:22.789469 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 213121us
I20260709 13:07:22.703552 24799 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.804387 24828 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:22.804644 22753 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:22.804981 24828 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:22.804421 24799 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:22.806092 24828 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:22.807746 24828 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.808621 24798 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:07:22.815488 22868 leader_election.cc:336] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.889s (SENT)
I20260709 13:07:22.816004 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:22.818030 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 235603us
I20260709 13:07:22.786561 24695 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:22.825626 22750 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:22.783576 24723 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:07:22.840941 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 216046us
I20260709 13:07:22.819733 24530 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.859107 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:22.864027 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:22.890225 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 104581us
I20260709 13:07:22.892758 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:22.839488 24815 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:22.861585 24831 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:07:22.867183 24834 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:22.889590 24545 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:22.924417 24831 consensus_queue.cc:1243] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:07:22.925231 23447 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:22.925726 23452 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:22.920428 24722 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:07:22.931742 22753 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:07:22.932600 24808 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.897157 24639 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:07:22.934108 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.868351 24530 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:22.829043 24697 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:22.935144 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 50639.2us
I20260709 13:07:22.943593 24695 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:07:22.944233 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 47478.8us
I20260709 13:07:22.949753 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:22.950691 23449 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:22.968688 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:22.900681 24810 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:07:22.970041 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.965034 22797 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:22.945744 24639 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.970631 22900 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:22.941671 24464 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.943264 24741 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:22.976313 22896 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 13:07:22.976554 22719 leader_election.cc:336] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.814s (SENT)
I20260709 13:07:22.976704 22896 pending_rounds.cc:85] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 1
I20260709 13:07:22.976984 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:22.978526 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:22.979329 23446 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 2: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:22.981904 23453 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:07:22.983484 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:22.983985 24827 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:22.984534 24827 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:22.942133 24799 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:07:22.986356 24837 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:22.986792 24837 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:22.977351 22896 pending_rounds.cc:107] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:07:22.987558 22896 raft_consensus.cc:2889] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:22.987879 24798 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:22.986394 24817 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:22.992939 22868 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.064s (SENT)
I20260709 13:07:22.997009 22900 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:22.976253 24639 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:23.017617 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 153662us
I20260709 13:07:23.018965 24833 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:23.019716 24830 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:23.020147 24741 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:22.969198 24714 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:07:23.020663 24741 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:23.020200 24830 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:23.022158 24830 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:23.021135 24835 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:23.024001 24835 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:23.024361 22902 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:23.025349 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:23.025027 24835 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.025763 24808 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:22.969635 24694 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:07:23.026474 24835 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:23.026818 24694 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:23.026592 22828 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:23.027210 24694 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:23.027563 22828 raft_consensus.cc:606] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:23.027973 22828 raft_consensus.cc:614] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:23.026114 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54395.3us
W20260709 13:07:23.034209 22719 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.988s (SENT)
I20260709 13:07:22.973008 24609 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:07:23.046164 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.047735 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.048134 22824 raft_consensus.cc:2368] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4.
I20260709 13:07:22.991617 24731 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:23.050408 22830 raft_consensus.cc:2368] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
W20260709 13:07:23.051882 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 77200.6us
I20260709 13:07:23.054193 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.055548 24530 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:23.055425 24741 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.056375 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:23.056730 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.057292 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:23.057956 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.061434 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:22.998939 24464 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:23.062520 22719 leader_election.cc:336] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.150s (SENT)
I20260709 13:07:23.032225 24723 raft_consensus.cc:2764] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:07:23.063374 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.065224 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.066002 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:23.061877 22868 leader_election.cc:336] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.086s (SENT)
I20260709 13:07:23.069460 24694 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.064527 22830 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:23.069378 24837 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.071309 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.072088 24837 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:23.072156 24731 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:23.056915 24741 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:23.073598 24464 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:23.066028 22822 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:23.037753 24722 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:07:23.076447 24722 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:23.076784 24722 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:23.077584 24722 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.079545 24530 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:23.078831 24722 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:23.081419 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:23.082543 22719 leader_election.cc:336] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.229s (SENT)
I20260709 13:07:23.083020 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:23.083737 24808 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.084645 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:23.085170 22751 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:23.085454 24716 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:23.086236 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:23.087201 22747 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:23.089404 24799 consensus_queue.cc:1059] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:23.090639 24799 raft_consensus.cc:3055] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:23.091007 24799 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:23.039430 24733 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.091748 24799 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.092612 22756 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:23.098269 22823 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:23.099056 22751 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:23.102523 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:23.103505 24810 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:23.103885 24810 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.106626 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.107549 22823 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:23.110055 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:23.080926 24530 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:23.115291 22752 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:07:23.117604 22752 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:23.092969 24799 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:23.123442 22752 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:23.123991 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:23.121222 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:23.092085 24694 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:23.123056 22748 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:23.108548 24810 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:23.134409 24545 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:23.135430 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.135561 24827 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.137630 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.138600 22822 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:23.140250 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.140914 22827 raft_consensus.cc:2376] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6. Re-sending same reply.
I20260709 13:07:23.141574 24837 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.142477 24622 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.142598 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.143286 22830 raft_consensus.cc:2368] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4.
I20260709 13:07:23.142992 24622 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.144009 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.145032 24622 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:23.142016 24827 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.141788 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.145404 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.162151 22829 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:23.124788 24733 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:23.152293 24809 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:23.173295 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.177165 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:23.180451 24848 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:07:23.168785 22827 raft_consensus.cc:2376] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4. Re-sending same reply.
I20260709 13:07:23.142647 24837 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:23.146184 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.145901 22822 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:23.146667 23416 leader_election.cc:336] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.523s (SENT)
I20260709 13:07:23.150383 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:23.145447 24693 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:23.157704 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.145011 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.146982 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.160926 22824 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:23.214633 22823 raft_consensus.cc:2368] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:23.197897 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.169126 24808 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:07:23.208418 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.177764 24809 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:23.188735 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:23.215546 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 112251us
I20260709 13:07:23.236887 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:23.238626 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 91976.3us
I20260709 13:07:23.242250 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:23.240430 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.562s (SENT) [suppressed 47 similar messages]
I20260709 13:07:23.296312 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.297504 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:23.322922 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 106715us
W20260709 13:07:23.325269 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.325829 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 89984.1us
I20260709 13:07:23.326086 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.327491 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.330515 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.330793 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.331547 22824 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:23.332149 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:23.330544 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 73637.1us
I20260709 13:07:23.236683 24662 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:23.343113 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:23.344460 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:23.340101 22829 raft_consensus.cc:2376] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4. Re-sending same reply.
I20260709 13:07:23.242007 24821 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:07:23.350958 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.352501 22900 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:23.358668 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:23.237717 24831 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 13:07:23.370972 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:23.374891 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 268798us
I20260709 13:07:23.377386 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:23.380332 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 139924us
W20260709 13:07:23.381111 23416 leader_election.cc:336] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.600s (SENT)
I20260709 13:07:23.377931 22829 raft_consensus.cc:2376] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2. Re-sending same reply.
I20260709 13:07:23.319067 24833 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:07:23.391079 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:23.393020 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:23.396041 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.377174 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.379509 24811 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:23.398429 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.379397 24632 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:23.396865 22825 raft_consensus.cc:2376] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3. Re-sending same reply.
W20260709 13:07:23.493086 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 94313.2us
W20260709 13:07:23.493243 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 112155us
I20260709 13:07:23.494143 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:23.494400 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:23.377508 24862 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.352123 24662 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.390707 24530 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:23.370463 24639 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:23.579330 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 85266.3us
I20260709 13:07:23.609812 24859 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:23.575228 24616 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:23.611929 24697 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:23.609954 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.610603 24859 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:23.611572 24853 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
I20260709 13:07:23.619194 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:23.581780 22829 raft_consensus.cc:2368] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:23.495242 24811 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:23.396181 24482 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.186s
I20260709 13:07:23.645157 22754 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
W20260709 13:07:23.621106 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.579430 24835 consensus_meta.cc:287] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.001s
W20260709 13:07:23.574647 24855 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.532s	user 0.004s	sys 0.000s
I20260709 13:07:23.575506 24730 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:07:23.492869 24860 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:23.654771 22825 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:23.578405 24815 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:23.645733 24741 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.573s	user 0.004s	sys 0.004s
I20260709 13:07:23.527889 24530 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:23.669842 22752 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:23.610620 24851 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:23.621968 24859 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:23.676568 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.674006 22826 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
W20260709 13:07:23.673755 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 175455us
I20260709 13:07:23.645228 24811 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:23.682327 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:23.683635 23416 leader_election.cc:336] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.652s (SENT)
I20260709 13:07:23.694440 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.695675 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:23.697273 23416 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.721s (SENT)
I20260709 13:07:23.673352 24530 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:23.474093 24809 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:23.686769 24856 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.465s	user 0.001s	sys 0.000s
W20260709 13:07:23.731263 23416 leader_election.cc:336] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.767s (SENT)
I20260709 13:07:23.696851 24815 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:07:23.736727 24865 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:07:23.737790 24863 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:07:23.741168 22831 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:23.743901 22831 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:23.679965 24851 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:23.758261 22825 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:07:23.745746 24810 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.577s	user 0.006s	sys 0.003s
I20260709 13:07:23.786556 22831 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:23.787137 22825 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:07:23.788223 22825 pending_rounds.cc:85] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 4
I20260709 13:07:23.788458 22825 pending_rounds.cc:107] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:07:23.788681 22825 raft_consensus.cc:2889] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:23.789820 24859 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.790558 22831 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:07:23.791574 22831 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:23.791843 22831 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:23.791884 24859 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:23.792764 22831 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.796654 22831 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:23.804428 24827 consensus_meta.cc:287] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.635s	user 0.005s	sys 0.000s
I20260709 13:07:23.804986 22828 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
W20260709 13:07:23.808200 24799 consensus_meta.cc:287] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.681s	user 0.000s	sys 0.007s
I20260709 13:07:23.808995 22747 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:23.808646 22831 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:23.811002 22831 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:23.812079 22831 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:07:23.812297 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.818086 22830 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:23.830945 24616 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:23.831379 24639 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:23.795277 24810 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:23.826422 24530 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:07:23.834438 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 100644us
W20260709 13:07:23.838570 23416 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.551s (SENT)
I20260709 13:07:23.839658 22898 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:23.841018 24815 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:23.841728 24815 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:23.844581 22829 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:23.831899 24616 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:23.855113 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 477275us
I20260709 13:07:23.855683 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:23.833158 24859 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.852417 24858 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.513s	user 0.000s	sys 0.002s
I20260709 13:07:23.864599 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:23.862079 24810 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:23.869511 22721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.027s	sys 0.012s
I20260709 13:07:23.864717 24662 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:23.870455 22898 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:23.871876 24883 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:23.872418 24883 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:23.873469 24883 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:23.876271 24616 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:23.869850 22721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.027s	sys 0.012s
W20260709 13:07:23.878131 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.869874 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:23.866670 24859 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:23.884778 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.885046 24859 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:23.886040 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.887236 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.887889 22831 raft_consensus.cc:2368] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:23.887655 24811 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.889751 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:23.890625 22755 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:23.890710 22902 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:23.866570 24873 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:07:23.891534 24714 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:23.891911 24714 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:23.894008 24851 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:23.895457 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.896526 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.897164 22830 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:23.898082 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.898835 22830 raft_consensus.cc:2368] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3.
I20260709 13:07:23.899585 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.900172 22830 raft_consensus.cc:2376] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2. Re-sending same reply.
I20260709 13:07:23.900597 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:23.896288 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:23.892700 24714 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:23.906920 23453 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:23.894660 24815 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:23.910255 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.911895 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:23.912032 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:23.913669 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:23.901057 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.922293 22902 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:23.976295 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 357131us
I20260709 13:07:23.977257 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:23.977720 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.978708 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:23.980729 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.908914 24530 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:23.920724 24714 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:23.983083 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 143563us
I20260709 13:07:23.983703 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:23.975598 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:23.984663 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 148627us
I20260709 13:07:23.990589 22902 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:23.999524 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:24.001036 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.002028 22830 raft_consensus.cc:2368] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 4.
I20260709 13:07:24.008335 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:23.991930 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:24.012720 23453 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:23.990891 23416 leader_election.cc:336] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.649s (SENT)
I20260709 13:07:24.029806 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:23.982798 24815 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:07:24.038779 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.217628 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:24.034540 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.234985 22897 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:24.032680 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.025245 23416 leader_election.cc:336] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.834s (SENT)
I20260709 13:07:24.224361 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 256) took 1167 ms. Trace:
I20260709 13:07:24.235918 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.236016 22826 rpcz_store.cc:276] 0709 13:07:23.056549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.057737 (+  1188us) service_pool.cc:224] Handling call
0709 13:07:24.224194 (+1166457us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54001f90d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:24.224296 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1406892928}
I20260709 13:07:24.194214 24834 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:24.237238 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:24.237758 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 253974us
I20260709 13:07:24.238902 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.239615 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:24.033275 24873 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.208s
I20260709 13:07:24.237268 24834 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:24.055717 23453 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:24.242399 24834 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:24.243405 24834 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:24.241216 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.245750 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.209465 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.211311 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 1663) took 1116 ms. Trace:
I20260709 13:07:24.124159 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 255) took 1068 ms. Trace:
I20260709 13:07:24.124855 22825 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:24.258018 24811 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:24.259613 22831 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:24.261121 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.263635 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.265723 22749 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:07:24.268167 24851 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:24.269815 24830 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:24.270323 24830 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:24.271302 24830 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:24.248216 22828 rpcz_store.cc:276] 0709 13:07:23.056158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.056446 (+   288us) service_pool.cc:224] Handling call
0709 13:07:24.123991 (+1067545us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b54001f9fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:24.124097 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1719251840}
I20260709 13:07:24.273845 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.274478 22828 raft_consensus.cc:2376] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4. Re-sending same reply.
I20260709 13:07:24.275480 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.275362 24830 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.276764 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.189440 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.322868 22752 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:24.323292 22824 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:24.323567 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.323832 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1898) took 1125 ms. Trace:
I20260709 13:07:24.280031 22755 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:24.292905 24722 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:24.327638 24722 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:24.328644 24862 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:24.329033 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 90199.8us
I20260709 13:07:24.329463 22756 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:24.329190 24862 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:24.311636 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.314651 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.248096 22754 rpcz_store.cc:276] 0709 13:07:23.095348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.095483 (+   135us) service_pool.cc:224] Handling call
0709 13:07:23.095735 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.649838 (+554103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.211101 (+561263us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b5400045e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:24.211255 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263125888}
I20260709 13:07:24.159910 22830 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:07:24.320658 22826 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
W20260709 13:07:24.321290 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 296043us
I20260709 13:07:24.314128 24714 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.030435 22827 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:24.329825 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.000084 22898 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:24.324445 22824 rpcz_store.cc:276] 0709 13:07:23.198650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.146122 (+947472us) service_pool.cc:224] Handling call
0709 13:07:24.323776 (+177654us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:24.322889 22828 raft_consensus.cc:2368] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4.
I20260709 13:07:24.333889 22747 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:07:23.986418 24884 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:24.213107 24837 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:24.337733 23416 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.837s (SENT)
I20260709 13:07:24.307715 24877 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:24.341058 22748 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:24.137778 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:24.077946 24863 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:24.334681 22897 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:24.347851 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.349596 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.358608 22831 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:24.102231 24827 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.360226 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:07:24.353609 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.361940 22822 raft_consensus.cc:2376] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2. Re-sending same reply.
I20260709 13:07:24.340118 24874 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:24.367869 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 283) took 1164 ms. Trace:
I20260709 13:07:24.370364 22825 rpcz_store.cc:276] 0709 13:07:23.195250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.789724 (+594474us) service_pool.cc:224] Handling call
0709 13:07:24.124455 (+334731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:24.250194 (+125739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.359193 (+108999us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54001f9fd0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:24.359312 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":769190784}
I20260709 13:07:24.368614 22828 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:07:24.370394 23449 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:24.343858 24676 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:24.371572 24827 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:24.380268 24827 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:24.380731 24827 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.381459 24544 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:24.383891 24722 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:24.385020 24722 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:24.385625 24837 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:07:24.385957 22827 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:24.360603 24482 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:24.402026 22829 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
W20260709 13:07:24.403653 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 65940.6us
W20260709 13:07:24.404417 23416 leader_election.cc:336] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.851s (SENT)
I20260709 13:07:24.372152 24884 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:24.375479 24876 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:24.375028 24874 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.415838 24874 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:24.416121 22748 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:07:24.417375 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57211us
I20260709 13:07:24.418247 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:24.375216 24877 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
W20260709 13:07:24.424278 23416 leader_election.cc:336] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.858s (SENT)
W20260709 13:07:24.426055 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.430106 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.395500 24676 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:24.457490 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 46565.6us
W20260709 13:07:24.460557 23416 leader_election.cc:336] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.904s (SENT)
W20260709 13:07:24.457870 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 88681.9us
I20260709 13:07:24.421898 24731 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.467281 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.468320 22754 raft_consensus.cc:2410] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:07:24.469527 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.459586 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 225476us
W20260709 13:07:24.471963 24899 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.058s	user 0.003s	sys 0.000s
I20260709 13:07:24.471824 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:24.473423 24799 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.473516 24883 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:24.473574 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:24.474030 24883 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:24.473924 24799 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.474525 24883 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:24.475070 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:24.476513 24799 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:24.477432 24817 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.478609 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.481087 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:24.425309 24851 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:24.461648 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.461017 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:24.483353 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 65207.9us
I20260709 13:07:24.481729 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.484686 22825 raft_consensus.cc:2376] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4. Re-sending same reply.
I20260709 13:07:24.457947 24815 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:24.496873 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.480459 24817 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:24.480890 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.482517 24851 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:24.502970 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.501289 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:07:24.517853 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57316.8us
I20260709 13:07:24.517516 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.518879 24798 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:24.486199 24882 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:24.482111 24799 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:24.520802 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.523654 22825 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:24.495957 24695 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:24.532074 24695 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.532757 24808 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:24.499943 24545 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:07:24.528812 23416 leader_election.cc:336] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.987s (SENT)
I20260709 13:07:24.534240 24877 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:24.534631 24877 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:07:24.534873 23416 leader_election.cc:336] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.015s (SENT)
W20260709 13:07:24.535750 23416 leader_election.cc:336] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.108s (SENT)
I20260709 13:07:24.535490 24877 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:24.536595 23416 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.120s (SENT)
I20260709 13:07:24.526691 22827 raft_consensus.cc:2376] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2. Re-sending same reply.
I20260709 13:07:24.536614 24877 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:24.540642 22754 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:07:24.528978 24879 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:24.549265 24877 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:07:24.518446 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.550114 24877 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.522138 24881 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:24.553237 24877 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.555475 24851 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:24.556517 24851 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:24.557515 24882 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.559605 24882 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:24.560079 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.520843 24874 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:07:24.562074 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.520320 24891 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.562769 22755 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:24.559782 24877 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:24.562912 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 61708.7us
I20260709 13:07:24.563642 24871 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:24.564585 24871 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:24.565099 24871 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.564627 24908 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:24.568492 24622 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:24.568847 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:24.569468 24697 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:24.523021 24730 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:07:24.574728 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.574999 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.575346 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.575548 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.583088 22824 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:24.569963 24889 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:24.583295 22826 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:24.563468 24695 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:24.518702 24811 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:07:24.582577 22831 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:24.517093 24837 raft_consensus.cc:3055] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:24.583899 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.582669 22830 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.531350 24799 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:24.589061 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.571683 24908 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:24.525141 24860 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:07:24.592373 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:24.597431 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:24.598807 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.599399 22825 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:24.599947 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.600410 22825 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:24.601034 22825 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:24.601249 22825 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:24.602444 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:24.582144 24862 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:24.603660 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.604311 24616 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:24.604708 24810 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.589572 24798 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:07:24.604990 24891 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:24.587967 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.587707 24837 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:24.609018 22830 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:24.607296 24482 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:24.610419 24482 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:24.610477 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45430.5us
I20260709 13:07:24.610913 24482 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:24.603892 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:24.613032 23416 leader_election.cc:336] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.940s (SENT)
I20260709 13:07:24.586755 24859 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:24.589869 24799 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:24.602222 22825 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:24.627731 22825 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:24.623143 22897 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:24.626205 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:24.628938 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:24.604940 24878 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:24.596982 24887 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.634016 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:24.639917 23449 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:24.634876 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:24.644460 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:24.638582 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:24.589464 24808 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:07:24.663614 22831 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:24.651669 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.665268 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:24.666390 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:24.624452 24862 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:24.671269 23449 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:07:24.640039 24909 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 13:07:24.674064 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.630889 24798 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:24.674767 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:24.681289 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:24.684705 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 71680.8us
I20260709 13:07:24.685770 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.689473 24545 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:24.666332 24882 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:24.690999 24882 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:24.691865 22823 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:24.655680 24874 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:24.665441 24817 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:07:24.737219 23416 leader_election.cc:336] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.180s (SENT)
W20260709 13:07:24.738274 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.744575 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:24.745419 24545 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:24.747248 24545 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.674391 24730 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:24.641762 24881 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:07:24.631744 24859 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.753176 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:24.758935 24482 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:24.692510 24850 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:24.761055 24482 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:24.736406 24815 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:24.771613 22754 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:07:24.734956 24874 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:24.774797 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 89966.7us
I20260709 13:07:24.742617 24716 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.781522 23449 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:07:24.781870 23449 pending_rounds.cc:85] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 5
I20260709 13:07:24.666950 24887 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.782074 23449 pending_rounds.cc:107] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:07:24.786316 23449 raft_consensus.cc:2889] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:24.754673 24860 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:24.790792 22830 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:24.791754 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.757511 24817 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:24.799022 22897 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:07:24.800048 24914 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.800345 22901 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:24.684067 24862 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:24.808902 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.765563 24730 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:24.803632 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:24.784492 24902 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:07:24.780591 24834 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:24.825378 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.786670 24874 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:24.836544 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:24.837289 23453 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:24.800716 24914 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:24.843655 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 90549.2us
I20260709 13:07:24.844578 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:24.683365 24908 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:24.847630 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44108.2us
I20260709 13:07:24.848492 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:24.846010 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:24.815614 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:24.860425 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.844046 24834 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.796218 24716 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.753878 24837 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:07:24.844041 24901 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
W20260709 13:07:24.868013 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.870765 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:24.837913 24918 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:24.863448 24482 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:24.896822 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:24.852664 24884 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:24.902585 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.905439 22747 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:24.868857 24798 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:24.868718 24920 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:07:24.896579 24912 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:07:24.915455 22825 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:07:24.874636 24908 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:24.916514 22825 pending_rounds.cc:85] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 2
I20260709 13:07:24.917366 22825 pending_rounds.cc:107] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:07:24.917641 22825 raft_consensus.cc:2889] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:24.919588 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.920253 22825 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:24.921594 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:24.923306 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.924544 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.925189 22825 raft_consensus.cc:2376] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2. Re-sending same reply.
W20260709 13:07:24.928642 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.951563 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:24.952414 22829 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:24.897346 24918 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:07:24.921882 22898 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.000s
I20260709 13:07:24.844882 24882 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:24.913431 24877 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.916872 24890 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.901548 24907 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.077s
W20260709 13:07:24.974682 24915 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:07:24.975343 24850 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:24.994041 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:24.995584 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.975124 24920 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:24.980182 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 125616us
I20260709 13:07:24.977115 24918 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:24.894932 24817 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.901815 24815 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.982406 22898 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:24.991235 24676 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:25.060457 22829 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:25.083715 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:25.084275 22829 raft_consensus.cc:2376] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4. Re-sending same reply.
I20260709 13:07:24.988827 24877 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:24.972496 24730 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.090385 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:25.091014 23452 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:24.994786 24850 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.057394 24859 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:24.989916 24914 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:25.083132 24917 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:07:25.101892 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:25.104872 22824 raft_consensus.cc:2376] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2. Re-sending same reply.
I20260709 13:07:25.105934 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:25.092998 24927 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:07:25.073673 24695 raft_consensus.cc:2764] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:07:25.103438 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.992468 24890 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.073053 24798 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:25.106871 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:25.117712 24920 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.113216 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:25.122591 22829 raft_consensus.cc:2376] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3. Re-sending same reply.
I20260709 13:07:25.123706 22829 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:07:25.123863 24943 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:25.124745 24878 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:25.125758 24883 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.000s
I20260709 13:07:25.103641 24862 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:25.126165 24883 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.127601 22752 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:25.115393 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:25.114084 22902 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:25.129244 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 225833us
I20260709 13:07:25.106557 22824 raft_consensus.cc:2376] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2. Re-sending same reply.
I20260709 13:07:25.086261 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1208) took 1166 ms. Trace:
I20260709 13:07:25.135550 22824 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:25.136036 22824 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:25.137923 22901 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:25.138780 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:25.136941 22824 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:25.144837 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.134177 24507 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:25.115895 24810 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:07:25.119680 24798 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:25.121717 24879 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:07:25.154013 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:25.154222 24862 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:07:25.158589 24799 consensus_meta.cc:287] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.001s
I20260709 13:07:25.159747 24676 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:25.153385 22897 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:25.190029 24722 consensus_meta.cc:287] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.805s	user 0.010s	sys 0.000s
I20260709 13:07:25.150076 23447 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:25.197726 22901 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
W20260709 13:07:25.230341 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.202232 22825 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:25.197408 23453 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:25.233778 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:25.234616 23450 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
W20260709 13:07:25.235049 24851 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.002s
I20260709 13:07:25.198251 24810 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:07:25.268239 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.273794 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:25.274711 22830 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:25.200924 24883 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:25.235814 23452 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:25.185572 24716 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:25.236739 24890 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:25.201660 24877 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:25.241832 24922 log.cc:927] Time spent T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:07:25.246979 24884 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:07:25.249459 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.249655 24934 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:07:25.267369 24810 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:25.259387 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.161777 24507 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:25.215830 22824 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:25.299374 22794 connection.cc:256] RPC call timeout handler was delayed by -2.2423457749998761s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 13:07:25.261876 22901 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:07:25.165468 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50144.9us
W20260709 13:07:25.299997 22794 connection.cc:256] RPC call timeout handler was delayed by -2.1694457749998719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
I20260709 13:07:25.134434 22898 rpcz_store.cc:276] 0709 13:07:23.912649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.921115 (+  8466us) service_pool.cc:224] Handling call
0709 13:07:25.074388 (+1153273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:25.191948 24947 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:07:25.160504 24871 consensus_meta.cc:287] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.001s
W20260709 13:07:25.279053 22756 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.010s	sys 0.007s
I20260709 13:07:25.197587 24676 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:25.253756 22897 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:25.297145 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 158460us
W20260709 13:07:25.378952 22756 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.010s	sys 0.016s
I20260709 13:07:25.383433 24676 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.352916 24884 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.387153 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:25.300511 22794 connection.cc:256] RPC call timeout handler was delayed by -2.1316457749999245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1642s for a response
I20260709 13:07:25.350661 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:25.387919 22794 connection.cc:256] RPC call timeout handler was delayed by -2.0812457749999567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
I20260709 13:07:25.304380 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:25.388877 22902 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:25.389701 22901 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:07:25.390451 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.391084 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:25.341977 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:25.348666 24918 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.193742 24799 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.394289 22755 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:25.305424 24507 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.345471 24722 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.395256 24507 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:25.395385 24722 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.370810 24885 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:25.396687 24885 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:25.307799 24947 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.366746 24810 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.342000 24883 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:07:25.373176 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:25.398573 24885 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.399978 24885 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.401505 24799 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:25.402889 24918 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:25.344316 24919 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.501s	user 0.001s	sys 0.000s
I20260709 13:07:25.335847 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:25.347040 24837 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:25.227599 24874 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.415503 22905 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:25.312654 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.387379 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.416644 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.417062 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.369168 23416 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.380s (SENT)
I20260709 13:07:25.333925 24871 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.368247 22824 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:25.397624 24947 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:25.388222 22794 connection.cc:256] RPC call timeout handler was delayed by -2.0569457749999174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
W20260709 13:07:25.422353 22794 connection.cc:256] RPC call timeout handler was delayed by -1.9399457749999556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
W20260709 13:07:25.422667 22794 connection.cc:256] RPC call timeout handler was delayed by -1.7203457749999416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 13:07:25.423062 22794 connection.cc:256] RPC call timeout handler was delayed by -1.6510457749999432s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
W20260709 13:07:25.423408 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.423892 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:25.418388 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.429715 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:25.446230 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.452895 22904 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:25.455183 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 119386us
I20260709 13:07:25.381518 24908 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:07:25.428076 22899 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:25.459949 24943 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:25.460963 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:25.456387 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:07:25.466492 24957 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:07:25.462917 24808 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:25.426920 24827 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:25.473439 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 104279us
I20260709 13:07:25.475100 22752 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:25.420887 24884 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:25.492268 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.455964 24851 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.499969 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:25.463063 24918 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:25.509622 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48752us
I20260709 13:07:25.558583 23449 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:25.486138 22822 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
W20260709 13:07:25.506981 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.552177 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 95894.1us
I20260709 13:07:25.566011 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:25.471464 24799 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 13:07:25.568295 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:25.571550 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:25.572309 22751 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:25.573444 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:25.546810 24878 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.575634 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:25.547137 24916 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:25.578531 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:25.565337 22822 pending_rounds.cc:85] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 2
I20260709 13:07:25.588037 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:25.491087 24936 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 13:07:25.589448 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:25.583465 24916 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:25.580005 23449 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:25.548064 24851 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:25.600332 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.579210 24890 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:25.584800 22822 pending_rounds.cc:107] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:07:25.590265 24859 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:25.570626 24918 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:25.606081 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 129998us
I20260709 13:07:25.598495 24883 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:07:25.612279 23416 leader_election.cc:336] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.796s (SENT)
W20260709 13:07:25.614475 24895 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:25.595862 24916 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:25.616062 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:25.604885 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:25.620008 24958 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:07:25.620730 24884 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:25.621038 24950 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:25.621584 24950 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.622872 24907 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:25.623260 24907 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:25.623993 24950 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:25.624266 24907 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:25.625074 24952 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:07:25.608052 22822 raft_consensus.cc:2889] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:25.640533 24622 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:25.628451 24834 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:25.649263 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:25.649463 23416 leader_election.cc:336] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.982s (SENT)
W20260709 13:07:25.660531 23416 leader_election.cc:336] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.247s (SENT)
W20260709 13:07:25.665642 24953 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:07:25.634194 24874 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:25.617364 24871 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:25.634055 24883 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:25.607466 24890 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.606618 24928 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:07:25.633496 24859 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.673226 24907 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:25.673769 23416 leader_election.cc:336] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.119s (SENT)
I20260709 13:07:25.693188 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:25.674638 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:25.674037 24877 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.657743 24947 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:25.704437 23416 leader_election.cc:336] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.850s (SENT)
I20260709 13:07:25.682550 24878 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:25.712890 24908 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:25.723870 22756 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:25.725463 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 50885.7us
I20260709 13:07:25.723774 24827 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:25.729357 24622 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:25.730567 24622 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.731014 24622 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:25.716265 24507 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:25.733016 24961 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:07:25.716010 24908 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:25.711229 24877 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.733361 24883 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.734565 24914 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.770854 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:25.771623 22831 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:25.744756 24908 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:25.774302 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.773720 24954 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:07:25.784125 24947 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:25.780033 24851 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:25.779232 24918 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:25.774204 24616 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 13:07:25.825028 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.781687 24716 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:07:25.774567 24914 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.835111 24809 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.874215 24965 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:25.884768 24965 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:25.886873 24965 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:25.855821 24890 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:25.918048 24809 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:25.940632 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.874256 24859 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:25.917080 24946 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:07:25.957682 24918 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:25.963855 24918 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:25.974499 24877 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:25.979841 24622 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:25.987620 24963 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:07:25.975222 24965 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:25.995263 24811 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.989668 24914 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:25.856091 24620 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:07:26.035948 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1217) took 1235 ms. Trace:
I20260709 13:07:26.036397 22905 rpcz_store.cc:276] 0709 13:07:24.798864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.799441 (+   577us) service_pool.cc:224] Handling call
0709 13:07:24.799758 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.415940 (+616182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.033965 (+618025us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b5400275e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:26.034149 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415506304}
I20260709 13:07:25.994313 24887 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:25.973508 24955 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.068689 24955 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:26.065867 24622 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:26.066747 24879 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:26.067391 24916 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:26.038954 24965 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:26.074177 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 296601us
I20260709 13:07:26.036369 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1214) took 1249 ms. Trace:
W20260709 13:07:26.078727 22871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.006s	sys 0.026s
W20260709 13:07:26.090631 22871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.006s	sys 0.026s
W20260709 13:07:26.091085 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 581268us
I20260709 13:07:26.080127 24914 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:26.080680 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.091955 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.091852 24914 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:26.092419 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.083357 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 308627us
I20260709 13:07:26.093986 24914 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:26.078994 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.096159 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.097182 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.097553 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.100311 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.102240 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:26.074156 24879 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:26.120787 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:26.036052 24811 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:26.106160 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.112416 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:26.108295 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:26.110148 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:26.083609 24815 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.011s	sys 0.022s
I20260709 13:07:26.129582 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:26.142045 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.170783 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:26.171072 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:26.168255 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:26.171422 22754 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
W20260709 13:07:26.176905 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.177389 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.178001 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:26.179984 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:26.180671 22905 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:26.182608 22905 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:26.121704 24975 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.184567 22905 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:26.187595 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 599600us
I20260709 13:07:26.185441 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:26.133846 24815 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.011s	sys 0.023s
I20260709 13:07:26.185284 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:26.189845 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:26.190480 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:26.137517 24918 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:26.121276 24887 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:07:26.206395 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:26.207376 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:26.209540 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:26.173650 24914 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.211794 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:26.218279 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:26.137740 24622 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:07:26.183172 24979 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:26.184196 24885 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:07:26.227213 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:26.231055 22905 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:26.214771 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:26.090365 22899 rpcz_store.cc:276] 0709 13:07:24.784473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.787710 (+  3237us) service_pool.cc:224] Handling call
0709 13:07:24.858732 (+ 71022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.474174 (+615442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.999261 (+525087us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540034cbd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:26.005417 (+  6156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286659456}
I20260709 13:07:26.287668 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:26.291069 22903 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:26.212555 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:26.281567 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.295923 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 197762us
I20260709 13:07:26.299989 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:26.212832 24981 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:26.220314 24877 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.229604 24969 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:07:26.302263 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 87484us
I20260709 13:07:26.227509 24851 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:26.219033 24616 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:07:26.306625 24907 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:07:26.315325 24973 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:07:26.310094 24969 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:26.317023 24955 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:26.219175 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 120420us
I20260709 13:07:26.190891 24975 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:26.314957 24907 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.314944 24966 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.317200 24969 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:26.313830 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:26.304600 24809 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:26.278985 24983 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
W20260709 13:07:26.456015 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 131953us
I20260709 13:07:26.454968 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:26.316233 24978 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:07:26.227897 24914 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:07:26.481091 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.482343 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:26.508332 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.484370 24984 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:26.489748 24851 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:26.491374 24981 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.254526 24879 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:26.498602 24885 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:26.287158 24979 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:26.325685 24907 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:26.508257 22754 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:26.482520 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:26.508805 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.309597 24837 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.153s
W20260709 13:07:26.456602 24883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.086s	sys 0.134s
I20260709 13:07:26.480530 24965 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:26.456287 22827 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:26.312911 24877 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:26.461241 24935 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:26.551617 22826 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:07:26.563066 22826 pending_rounds.cc:85] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 3
I20260709 13:07:26.563414 22826 pending_rounds.cc:107] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:07:26.563700 22826 raft_consensus.cc:2889] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:26.551816 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.550348 22871 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:26.517619 24969 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:26.452802 24966 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:26.511875 24916 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:26.521813 24991 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:07:26.551092 24883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.087s	sys 0.135s
W20260709 13:07:26.551919 24908 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.766s	user 0.010s	sys 0.000s
I20260709 13:07:26.596472 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1343) took 1040 ms. Trace:
I20260709 13:07:26.596943 22829 rpcz_store.cc:276] 0709 13:07:25.556557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.557100 (+   543us) service_pool.cc:224] Handling call
0709 13:07:25.569186 (+ 12086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:25.713359 (+144173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.713394 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:25.715717 (+  2323us) log.cc:844] Serialized 26 byte log entry
0709 13:07:25.934834 (+219117us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:07:25.934884 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.934947 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:26.596218 (+661271us) raft_consensus.cc:1685] finished
0709 13:07:26.596259 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.596405 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140695,"raft.queue_time_us":142002,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":3786,"thread_start_us":218834,"threads_started":1,"wal-append.queue_time_us":417129}
I20260709 13:07:26.581441 24879 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:26.593510 24991 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:26.594331 24811 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:26.601240 24955 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:07:26.654316 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.631764 24991 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:26.655583 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2077) took 1790 ms. Trace:
I20260709 13:07:26.551978 24984 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:26.557670 24981 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:26.603238 24908 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:26.580623 24935 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:26.659446 22755 rpcz_store.cc:276] 0709 13:07:24.865312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.866839 (+  1527us) service_pool.cc:224] Handling call
0709 13:07:24.867143 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.394760 (+527617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.655349 (+1260589us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b5400077bd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:26.655521 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211824896}
W20260709 13:07:27.303462 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 820962us
W20260709 13:07:26.985703 22795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.014s	sys 0.019s
W20260709 13:07:27.304149 22795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.015s	sys 0.019s
W20260709 13:07:26.668113 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:27.462355 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:27.318463 22830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.000s	sys 0.013s
W20260709 13:07:27.462966 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.797462 24988 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:27.463110 22830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.000s	sys 0.016s
W20260709 13:07:27.146509 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.463728 24988 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:26.813730 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 491741us
I20260709 13:07:27.464653 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:27.464933 24883 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:26.979766 24985 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.324s	user 0.001s	sys 0.001s
I20260709 13:07:27.465482 22748 raft_consensus.cc:2410] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:07:27.460566 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 156284us
I20260709 13:07:27.466231 24817 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:26.670112 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.466687 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:27.147392 24851 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.001s	sys 0.027s
I20260709 13:07:27.466769 24817 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:27.318609 24837 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.012s	sys 0.001s
W20260709 13:07:27.467456 24851 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.004s	sys 0.029s
I20260709 13:07:27.465456 24883 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.469110 24851 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:27.464046 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:27.463387 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.469846 24935 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:27.470366 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.977263 24936 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.470476 24883 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:26.677052 24916 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:27.471760 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.914611 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 600938us
W20260709 13:07:27.145743 22905 consensus_meta.cc:287] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.847s	user 0.007s	sys 0.004s
I20260709 13:07:27.471272 24916 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.474402 22905 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:27.475023 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1224) took 1248 ms. Trace:
I20260709 13:07:27.471959 24817 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.475378 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:26.663491 24881 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.472205 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:27.472482 22829 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:27.475378 22905 rpcz_store.cc:276] 0709 13:07:26.226925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.230151 (+  3226us) service_pool.cc:224] Handling call
0709 13:07:26.230805 (+   654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.474811 (+1244006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.474975 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:27.476146 24936 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:26.656385 24986 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:26.979267 24977 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:07:27.477293 24986 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.478457 24936 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:27.478684 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 396) took 1922 ms. Trace:
I20260709 13:07:27.476214 24881 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.479313 23450 rpcz_store.cc:276] 0709 13:07:25.556355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.556529 (+   174us) service_pool.cc:224] Handling call
0709 13:07:25.563391 (+  6862us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:25.563764 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.563794 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:25.564230 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:07:26.078265 (+514035us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:26.078316 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.078414 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:27.478485 (+1400071us) raft_consensus.cc:1685] finished
0709 13:07:27.478506 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.478612 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11242,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":4635,"thread_start_us":490369,"threads_started":1,"wal-append.queue_time_us":726779}
I20260709 13:07:27.480922 24881 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:27.305652 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1348) took 1092 ms. Trace:
W20260709 13:07:27.306855 23417 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.008s	sys 0.013s
I20260709 13:07:26.805966 24914 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:27.254833 24989 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.589s	user 0.001s	sys 0.000s
W20260709 13:07:27.304926 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:27.467615 24837 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.015s	sys 0.001s
W20260709 13:07:27.482638 23417 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.015s	sys 0.015s
W20260709 13:07:27.499451 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.482156 24986 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:26.633486 24955 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.815146 24877 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:27.514711 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.823180 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.255868 24622 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.239634 24966 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:26.864864 24860 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.486387 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:27.535851 22829 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.488546 22905 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:07:27.489801 22824 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:07:27.537245 22824 pending_rounds.cc:85] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 5
I20260709 13:07:27.537576 22824 pending_rounds.cc:107] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:07:27.537848 22824 raft_consensus.cc:2889] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:27.538213 22824 pending_rounds.cc:107] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:07:27.538506 22824 raft_consensus.cc:2889] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:27.492700 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:27.478320 24969 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.898s	user 0.007s	sys 0.003s
I20260709 13:07:27.481632 22823 rpcz_store.cc:276] 0709 13:07:26.213584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.289473 (+ 75889us) service_pool.cc:224] Handling call
0709 13:07:26.307628 (+ 18155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.452729 (+145101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.452760 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.452800 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:26.452842 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.305361 (+852519us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b5400432454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:27.305425 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.305582 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43438976}
I20260709 13:07:27.071365 24975 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:27.146912 24993 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.452s	user 0.001s	sys 0.000s
I20260709 13:07:27.134773 24811 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.525296 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:27.255565 24994 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.454s	user 0.000s	sys 0.001s
I20260709 13:07:27.470475 24935 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.439770 24982 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.539893 22904 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:27.463212 24984 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:27.475907 24998 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:27.496886 24979 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:27.545050 24991 consensus_meta.cc:287] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.868s	user 0.008s	sys 0.002s
W20260709 13:07:27.545436 24879 consensus_meta.cc:287] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.921s	user 0.010s	sys 0.000s
I20260709 13:07:27.505513 24955 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:27.524840 24986 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:27.587814 22751 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:27.536127 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:27.589267 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 122746us
I20260709 13:07:27.528717 24716 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:27.532330 24936 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:07:27.561415 22822 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:07:27.564141 24998 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:27.602077 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:27.556684 24811 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.589650 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:27.665529 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 63540.7us
I20260709 13:07:27.586870 24914 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.564842 24935 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.588697 24883 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.656435 22822 pending_rounds.cc:85] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 1
W20260709 13:07:27.659998 24993 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:07:27.668668 25003 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:27.568858 24982 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.652395 25002 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.667884 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:27.568287 24837 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:07:27.662249 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.616781 24986 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:27.547225 24860 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.657394 25000 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.616500 22827 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:27.678786 22829 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:27.682425 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:27.685001 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1926) took 1495 ms. Trace:
I20260709 13:07:27.589037 24991 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.674413 22822 pending_rounds.cc:107] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:07:27.690599 22831 rpcz_store.cc:276] 0709 13:07:26.186002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.186138 (+   136us) service_pool.cc:224] Handling call
0709 13:07:26.191027 (+  4889us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:26.206141 (+ 15114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.206177 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:26.206676 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:07:26.453179 (+246503us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:26.484500 (+ 31321us) log.cc:844] Serialized 14 byte log entry
0709 13:07:26.978481 (+493981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.978584 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:27.680648 (+702064us) raft_consensus.cc:1685] finished
0709 13:07:27.680673 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.680831 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19662,"prepare.queue_time_us":31162,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":118467,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":24620,"thread_start_us":734742,"threads_started":2,"wal-append.queue_time_us":427909}
I20260709 13:07:27.650339 24716 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.706559 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.708961 25005 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:27.694113 24975 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.666756 25015 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:27.718281 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:27.719019 22905 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:27.697530 24879 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.675020 24914 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.682154 24883 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.693127 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:27.722710 22829 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:27.693974 24969 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.709117 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.698553 22822 raft_consensus.cc:2889] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:27.710021 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.535390 24622 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.725302 25011 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.566803 24995 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.162s
I20260709 13:07:27.673632 25003 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.728305 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.735078 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.728488 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.715406 25006 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.731261 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.732650 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:27.716485 24966 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:27.725411 22794 connection.cc:256] RPC call timeout handler was delayed by -1.241993381000043s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
I20260709 13:07:27.703959 25001 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:27.710960 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 395) took 2156 ms. Trace:
I20260709 13:07:27.718998 25007 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.735663 22905 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:27.735486 22896 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:27.724082 24979 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.689848 25000 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.740820 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:27.683213 25002 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.762945 25002 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.745308 25006 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.765301 25006 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:27.766636 22896 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:27.768852 24883 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.769361 24883 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.769834 24883 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.772286 24914 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:27.773088 23453 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:27.773022 24883 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.774699 25003 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.776065 24935 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.776490 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.777016 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:27.743826 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:27.736442 22900 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:27.740051 22904 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:27.778257 24860 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:27.741753 24966 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.779151 25003 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:27.747879 24979 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.728971 24879 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.780534 24966 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:27.780529 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:27.781920 24979 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.782191 24879 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.741528 24986 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.783358 25003 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:27.784719 25003 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:27.786211 24908 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.741252 23446 rpcz_store.cc:276] 0709 13:07:25.554561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.560039 (+  5478us) service_pool.cc:224] Handling call
0709 13:07:25.601300 (+ 41261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.712628 (+111328us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:25.712929 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 13:07:26.176086 (+463157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:26.176164 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:26.176232 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.710622 (+1534390us) spinlock_profiling.cc:243] Waited 93.6 ms on lock 0x7b5400729a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:27.710688 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.710860 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":860,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":2441,"raft.queue_time_us":141609,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":215339264,"thread_start_us":462756,"threads_started":1,"wal-append.queue_time_us":594824,"wal-append.run_cpu_time_us":1331,"wal-append.run_wall_time_us":1213435}
I20260709 13:07:27.727015 24969 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.789170 24969 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:27.790400 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:27.790594 23447 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:07:27.791270 24979 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:27.749946 24991 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:27.750670 22831 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
W20260709 13:07:27.751295 25008 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:07:27.720861 24975 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.759346 24811 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.734321 25011 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:27.761157 24907 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.203s	user 0.011s	sys 0.000s
I20260709 13:07:27.733357 24722 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.791451 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:27.749845 24982 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:27.735544 25005 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.813609 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.815922 22903 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:27.814581 22904 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
W20260709 13:07:27.817356 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 103476us
I20260709 13:07:27.818169 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:27.818397 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:27.819397 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:27.820533 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:27.822546 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.822300 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:27.815292 24979 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:27.824636 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.817431 24811 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:27.823187 24969 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:27.834025 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:27.834424 24914 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:27.834808 24914 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.835022 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.835281 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.835554 25001 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:27.835884 23452 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:27.836737 23449 raft_consensus.cc:2410] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:07:27.822793 25005 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.837052 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:27.838554 25005 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.838759 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.822701 25002 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:27.822781 24722 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:27.840077 22831 raft_consensus.cc:606] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:27.840476 22831 raft_consensus.cc:614] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:27.835161 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:27.835323 24837 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:27.842615 24837 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.842967 24837 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:27.845646 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.846257 25002 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:27.846633 25002 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.746491 25007 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.835124 24914 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:27.846971 25002 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:27.847922 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:27.849715 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:27.824250 24966 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:27.815112 24907 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.833427 24908 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.830276 22827 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:27.859338 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:27.856221 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.849129 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1223) took 1621 ms. Trace:
I20260709 13:07:27.861428 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2097) took 1675 ms. Trace:
I20260709 13:07:27.871255 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:27.858070 24979 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:27.874245 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.858848 24860 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:27.849534 24811 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.874504 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:27.858316 24966 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.897616 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:27.869462 22756 rpcz_store.cc:276] 0709 13:07:26.186466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.186587 (+   121us) service_pool.cc:224] Handling call
0709 13:07:26.186857 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.187048 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:27.729852 (+1542804us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.860981 (+131129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.861082 (+   101us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:27.861140 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.861220 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.861364 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1472866,"prepare.queue_time_us":1485865,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":15080,"thread_start_us":191422,"threads_started":2}
I20260709 13:07:27.870028 25009 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.884817 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2092) took 1773 ms. Trace:
W20260709 13:07:27.893916 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 388461us
I20260709 13:07:27.852512 24955 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.861562 24947 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.895397 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:27.895984 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.896126 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:27.876401 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 460) took 2325 ms. Trace:
I20260709 13:07:27.898293 22749 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:27.901882 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:27.894920 22755 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:27.860280 25005 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:07:27.879365 24979 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:27.905359 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.873855 24935 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.874128 25010 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.913208 22756 raft_consensus.cc:2410] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:07:27.914433 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:27.921727 22828 raft_consensus.cc:2410] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:07:27.895780 24969 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:27.922232 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.895941 24879 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:27.915076 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:27.915477 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:27.870203 24908 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.913846 22752 rpcz_store.cc:276] 0709 13:07:25.547836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.578129 (+ 30293us) service_pool.cc:224] Handling call
0709 13:07:25.599413 (+ 21284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.599856 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:25.636251 (+ 36395us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.586087 (+1949836us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b18003fac60. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb787f33c7 00007fcb74e059a7 00007fcb74e150bf 00007fcb74e15038
0709 13:07:27.586317 (+   230us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.872625 (+286308us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.872669 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:27.872709 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.872764 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.872841 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1917048,"prepare.queue_time_us":2013643,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":72,"raft.queue_time_us":40857,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":10138,"spinlock_wait_cycles":7611136,"thread_start_us":285929,"threads_started":2,"wal-append.queue_time_us":229456}
I20260709 13:07:27.917038 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:27.903797 22747 rpcz_store.cc:276] 0709 13:07:26.111157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.111328 (+   171us) service_pool.cc:224] Handling call
0709 13:07:26.130542 (+ 19214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.130831 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:27.660174 (+1529343us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.884420 (+224246us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.884484 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:27.884541 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.884619 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.884754 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1483560,"prepare.queue_time_us":1531662,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":33433,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":57,"thread_start_us":223764,"threads_started":2,"wal-append.queue_time_us":207384}
I20260709 13:07:27.893797 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.894222 25001 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.913945 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:27.986012 23447 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:27.986092 25001 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.924612 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.902576 24966 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:27.931977 24975 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.933519 25019 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.933888 25010 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.934767 24879 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.935729 22749 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:27.936863 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.910899 24947 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.937731 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:28.123663 23453 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
W20260709 13:07:27.938529 25022 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:07:27.906396 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.940563 25011 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.940652 24908 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.941556 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.941830 25026 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:28.135670 22896 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 2: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:27.942207 24907 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:27.942917 25007 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:27.910295 25009 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:27.928277 24979 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:27.945572 24986 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.946113 25028 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.946970 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:27.952280 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:27.952831 25017 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.953508 25006 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:27.954303 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:27.954546 25005 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:27.954926 25015 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:27.931384 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.936545 24837 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.957249 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:27.958585 22900 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:28.151449 22756 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:27.960083 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.971690 24817 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:27.973067 24811 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:27.978348 23416 connection.cc:256] RPC call timeout handler was delayed by -1.6648466029998872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 13:07:28.154536 23416 connection.cc:256] RPC call timeout handler was delayed by -1.5694466029999603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
I20260709 13:07:28.138018 23446 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
W20260709 13:07:28.154928 23416 connection.cc:256] RPC call timeout handler was delayed by -1.5361466029999065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
W20260709 13:07:28.155329 23416 connection.cc:256] RPC call timeout handler was delayed by -1.2958466029999727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
I20260709 13:07:28.156199 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:28.156950 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:27.984959 24622 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:27.986718 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.111847 24991 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.115902 22654 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 336) took 11071 ms. Trace:
I20260709 13:07:28.113106 22752 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:28.079182 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 91472.1us
I20260709 13:07:28.113567 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:28.124305 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.125100 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.128188 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:28.129637 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 250385us
I20260709 13:07:28.167182 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.168099 22750 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:28.115535 25019 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:27.917423 24982 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.217s
I20260709 13:07:28.141826 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:28.142664 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:28.126008 25000 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.123203 25020 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.145897 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:28.175302 22824 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.150022 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:28.150264 22904 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:27.945574 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:27.987239 24935 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.141580 24722 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:28.166662 22749 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 1.
I20260709 13:07:28.164376 22654 rpcz_store.cc:276] 0709 13:07:17.044593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.047092 (+  2499us) service_pool.cc:224] Handling call
0709 13:07:28.115070 (+11067978us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:22.990563 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:22.991163 (+   600us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:22.991214 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:07:22.991232 (+    18us) tablet.cc:662] Decoding operations
0709 13:07:23.089548 (+ 98316us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:23.089673 (+   125us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:23.089702 (+    29us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:07:23.090327 (+   625us) tablet.cc:701] Row locks acquired
0709 13:07:23.090389 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:07:23.090535 (+   146us) write_op.cc:270] Start()
0709 13:07:23.090686 (+   151us) write_op.cc:276] Timestamp: P: 1783602443090478 usec, L: 0
0709 13:07:23.090709 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:07:23.091342 (+   633us) log.cc:844] Serialized 2449 byte log entry
0709 13:07:24.782188 (+1690846us) op_driver.cc:464] REPLICATION: finished
0709 13:07:25.269713 (+487525us) write_op.cc:301] APPLY: starting
0709 13:07:25.269794 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:25.269977 (+   183us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:25.302676 (+ 32699us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:25.333764 (+ 31088us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:25.385034 (+ 51270us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:07:25.418411 (+ 33377us) write_op.cc:312] APPLY: finished
0709 13:07:25.458358 (+ 39947us) log.cc:844] Serialized 53 byte log entry
0709 13:07:25.854957 (+396599us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:25.871774 (+ 16817us) write_op.cc:454] Released schema lock
0709 13:07:25.875541 (+  3767us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:26.040374 (+164833us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:26.047753 (+  7379us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":480919,"apply.run_cpu_time_us":18363,"apply.run_wall_time_us":670176,"num_ops":6,"prepare.queue_time_us":374215,"prepare.run_cpu_time_us":14336,"prepare.run_wall_time_us":246420,"raft.queue_time_us":826984,"raft.run_cpu_time_us":13684,"raft.run_wall_time_us":254565,"replication_time_us":1672347,"spinlock_wait_cycles":1549056,"thread_start_us":1243337,"threads_started":5,"wal-append.queue_time_us":756774,"wal-append.run_cpu_time_us":55078,"wal-append.run_wall_time_us":3332255}]]}
I20260709 13:07:28.168345 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:28.113965 25029 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:28.169224 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:28.136065 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 266980us
I20260709 13:07:28.150362 25005 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.169911 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:28.173144 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:28.173874 22901 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:28.192345 22756 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:28.151721 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:28.140306 25026 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:28.162791 25007 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.182222 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.206738 22749 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:28.173902 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.187474 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:28.150473 25028 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.209641 25028 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.150465 24986 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.165861 25033 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:28.173596 24622 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:28.204938 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.163539 24817 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.215615 22756 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:28.178638 25003 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:28.175401 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:28.151276 25006 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.205962 25005 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:28.252641 25018 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:28.261808 25018 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.262238 25018 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:28.262549 25001 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.170485 24991 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.254967 24507 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:28.211018 25028 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.264601 23451 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:28.151381 25017 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.214041 23447 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:28.273455 24507 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:28.274840 25027 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:07:28.154050 25015 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:28.173094 25002 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:28.316159 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:27.871084 22898 rpcz_store.cc:276] 0709 13:07:26.226355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.247836 (+ 21481us) service_pool.cc:224] Handling call
0709 13:07:26.326659 (+ 78823us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:26.584010 (+257351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:26.587748 (+  3738us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:26.654930 (+ 67182us) log.cc:844] Serialized 52 byte log entry
0709 13:07:27.684870 (+1029940us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:07:27.693074 (+  8204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.698154 (+  5080us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:27.824534 (+126380us) raft_consensus.cc:1685] finished
0709 13:07:27.824554 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:27.847849 (+ 23295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4594,"raft.queue_time_us":20159,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":10692,"thread_start_us":201130,"threads_started":1,"wal-append.queue_time_us":1075698}
I20260709 13:07:28.191136 24722 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:28.283279 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:28.299409 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 126325us
I20260709 13:07:28.333777 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:28.251187 24982 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:28.302886 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1347) took 2089 ms. Trace:
W20260709 13:07:28.301792 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 133477us
I20260709 13:07:28.334681 22826 rpcz_store.cc:276] 0709 13:07:26.213249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.278532 (+ 65283us) service_pool.cc:224] Handling call
0709 13:07:26.300431 (+ 21899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.564230 (+263799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.302687 (+1738457us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54002f03d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:28.302835 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576772992}
I20260709 13:07:28.304450 25041 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:28.306303 24975 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.263995 24622 raft_consensus.cc:697] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:28.316752 25026 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.280777 22824 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:28.214957 25007 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.281814 24881 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:28.326614 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:28.329138 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.331300 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:28.282084 25021 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:28.344554 22750 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:28.339309 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 138752us
I20260709 13:07:28.339581 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:28.297981 24955 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:28.185853 25020 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:28.347502 24507 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.354357 24507 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.353955 24622 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.354782 25020 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:28.355068 24837 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.345923 24982 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:28.336477 24969 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:28.357307 25031 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:07:28.356966 24982 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.347764 24881 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:28.287703 25028 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.307171 24809 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:07:28.338227 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:28.343663 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.346625 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:28.205834 25029 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.350274 24482 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:28.350667 24817 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.340577 24879 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.358505 24881 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:28.363715 22871 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:28.250357 25033 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.365530 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.340883 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:28.371390 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:28.376345 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.505s (SENT) [suppressed 11 similar messages]
I20260709 13:07:28.411132 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.414176 22824 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
W20260709 13:07:28.418352 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 54687us
W20260709 13:07:28.418988 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 78197.8us
I20260709 13:07:28.419623 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:28.353363 25036 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:28.354667 25006 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.363077 24620 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:28.424346 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.425047 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:28.341701 24616 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:07:28.428969 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:28.426045 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54760.5us
I20260709 13:07:28.362442 25021 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:28.372469 24955 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:28.427318 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.430450 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.344210 24975 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.410984 25009 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.432054 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:28.364292 25019 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.503403 24998 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.508070 24947 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.506590 25028 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.508271 24731 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.509485 25017 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.422369 24997 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:28.519160 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 87159.6us
I20260709 13:07:28.524042 23452 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:28.369149 25041 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:07:28.495613 24916 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.527931 22750 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:28.565218 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:28.517560 24986 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.423105 25001 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:28.559445 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.437683 24955 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:28.594893 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 218564us
W20260709 13:07:28.596011 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.574182 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:28.598520 23416 leader_election.cc:336] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.648s (SENT)
W20260709 13:07:28.596035 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 166178us
W20260709 13:07:28.588759 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 159558us
I20260709 13:07:28.495012 24991 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.598971 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:28.601218 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.433043 24482 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.155s
I20260709 13:07:28.559216 25009 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:28.603592 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:28.587944 25039 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:07:28.431071 24837 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:28.430948 24860 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:07:28.570094 24914 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.710s	user 0.006s	sys 0.002s
I20260709 13:07:28.434141 25036 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:28.437932 25011 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:28.567281 24998 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:28.621726 25037 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:07:28.433043 25012 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:28.622788 25012 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:28.623179 25012 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:28.568885 24975 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.433369 24935 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.610242 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.624101 22824 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:28.567341 25003 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:28.604984 25009 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:28.598132 24955 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:28.607635 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:28.585037 24716 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.602650 24916 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:28.604839 24991 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:28.629629 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:28.628074 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 440613us
I20260709 13:07:28.634090 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:28.614192 24997 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:28.636727 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:28.638964 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:28.640759 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 212989us
I20260709 13:07:28.641709 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.642617 22871 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:28.627122 25009 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.626099 25003 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:28.645474 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.641726 24716 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:28.647274 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.647837 24716 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:28.648298 24969 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:28.648883 24969 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:28.650098 24969 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:28.651453 24962 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 2.853s	user 0.014s	sys 0.011s
I20260709 13:07:28.652977 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1922) took 3270 ms. Trace:
I20260709 13:07:28.646512 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.634313 24916 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.654465 22825 rpcz_store.cc:276] 0709 13:07:25.382804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.382908 (+   104us) service_pool.cc:224] Handling call
0709 13:07:25.385004 (+  2096us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:25.385305 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.385354 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:25.385695 (+   341us) log.cc:844] Serialized 26 byte log entry
0709 13:07:25.548227 (+162532us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:25.568980 (+ 20753us) log.cc:844] Serialized 14 byte log entry
0709 13:07:25.710803 (+141823us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b180032c140. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb74df3060 00007fcb74e05716 00007fcb74e16287 00007fcb74e16208
0709 13:07:25.712550 (+  1747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.712615 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.587572 (+2874957us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:28.652766 (+ 65194us) raft_consensus.cc:1685] finished
0709 13:07:28.652786 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.652902 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28755,"prepare.queue_time_us":24882,"prepare.run_cpu_time_us":532,"prepare.run_wall_time_us":7858,"raft.queue_time_us":149531,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":10424,"spinlock_wait_cycles":10707456,"thread_start_us":162077,"threads_started":1,"wal-append.queue_time_us":311103}
I20260709 13:07:28.610260 24914 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:28.654003 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:28.638266 24507 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:28.641937 24616 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:28.640527 25002 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.638840 24997 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:28.667222 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:28.674860 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:28.675848 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:28.634999 25005 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:28.655480 25003 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:28.679546 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 77048.4us
I20260709 13:07:28.681037 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.682415 22826 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:28.665601 24916 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.684887 22825 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
W20260709 13:07:28.685599 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.680533 24947 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:28.686508 24482 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:28.648900 24716 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.673542 25028 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:28.673694 24616 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:28.689218 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:28.717211 22897 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:28.717983 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:28.718535 22824 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:28.692540 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:28.696123 24482 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:28.698118 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.664759 24935 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:07:28.699620 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:28.703459 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.703816 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:28.705863 25026 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:28.706569 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.722697 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:28.709486 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:28.712153 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.713431 24981 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.718029 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:28.727989 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:28.678792 24997 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:28.692035 22902 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:28.567826 25021 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.698724 25047 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:28.687801 25019 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:28.708810 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.731781 22905 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:28.720363 22825 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:28.692546 24947 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:28.720494 24616 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:28.745596 24810 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.745904 25047 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:28.746369 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.746935 25029 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.747123 22825 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:28.747460 25029 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:28.747896 25029 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:28.748337 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.695645 25041 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:07:28.726522 24879 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:28.751503 22747 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:07:28.752837 24966 consensus_meta.cc:287] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.626s	user 0.006s	sys 0.000s
I20260709 13:07:28.722251 25028 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:28.754029 25028 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.730898 25033 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.748636 25029 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.770565 22755 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:07:28.736934 25002 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:28.772730 24507 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:28.775100 25002 raft_consensus.cc:697] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:28.775965 25002 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.776631 25040 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.777114 25040 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:28.784534 25005 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.785211 22756 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:07:28.785571 22756 pending_rounds.cc:85] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 1
I20260709 13:07:28.785764 22756 pending_rounds.cc:107] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:07:28.785986 22756 raft_consensus.cc:2889] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:28.788546 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1924) took 2610 ms. Trace:
I20260709 13:07:28.788841 22797 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:28.789847 24979 consensus_meta.cc:287] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.000s
I20260709 13:07:28.789959 24981 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:28.772958 24969 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:28.790282 24979 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.791226 24722 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.791738 22828 raft_consensus.cc:2368] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4.
I20260709 13:07:28.792325 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:28.794081 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:28.795919 24914 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.797693 22755 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:28.742319 25043 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:28.742640 24947 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:28.804212 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.740434 25019 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:28.812136 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:28.788904 22830 rpcz_store.cc:276] 0709 13:07:26.178031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.179097 (+  1066us) service_pool.cc:224] Handling call
0709 13:07:26.179409 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.581974 (+402565us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:26.598798 (+ 16824us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.715793 (+1116995us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.715879 (+    86us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:27.715946 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.788355 (+1072409us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400432e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:28.788395 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.788499 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17237,"prepare.queue_time_us":18986,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":91013,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":22155,"spinlock_wait_cycles":925109504,"thread_start_us":1116562,"threads_started":2,"wal-append.queue_time_us":981771,"wal-append.run_cpu_time_us":1413,"wal-append.run_wall_time_us":1120981}
I20260709 13:07:28.735335 25026 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:28.875523 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 134907us
I20260709 13:07:28.753252 24966 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.817691 24988 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:28.884428 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:28.820245 25019 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:28.886755 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.835654 25046 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:28.924353 22903 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:28.924854 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 221098us
I20260709 13:07:28.925860 23452 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 3 LEARNER]: Advancing to term 4
I20260709 13:07:28.925712 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:28.927229 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:28.932525 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:28.934227 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:28.938071 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:28.875258 24916 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.889672 24935 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:07:28.940054 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.941219 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.941802 22830 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:28.942246 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 336) took 1022 ms. Trace:
I20260709 13:07:28.942564 22830 rpcz_store.cc:276] 0709 13:07:27.919438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.940983 (+1021545us) service_pool.cc:224] Handling call
0709 13:07:28.942201 (+  1218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:28.943502 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.944031 22830 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
W20260709 13:07:28.944226 25018 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.000s
I20260709 13:07:28.928815 24810 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:28.961557 24810 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:28.923965 25046 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:28.962019 24810 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:28.962083 24947 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.944880 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.959789 22903 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:28.958678 23452 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 4 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:28.964664 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:28.964908 22719 connection.cc:256] RPC call timeout handler was delayed by -1.0313523650000889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23900000000000002s for a response
I20260709 13:07:28.961015 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:28.936537 24969 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:28.950100 24990 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 2.401s	user 0.003s	sys 0.008s
I20260709 13:07:28.939056 24979 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.950570 24966 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:28.944774 25018 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.974802 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:28.945181 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:28.944880 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:28.938910 24860 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:28.953653 25052 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:28.978927 23452 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:28.886029 25030 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:07:28.981278 22830 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:29.011029 22829 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:28.972795 24997 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.952311 25009 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:28.962675 25046 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:29.011564 25052 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.074626 22825 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:29.116636 25046 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:29.145255 22827 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 326) took 1402 ms (client timeout 1505 ms). Trace:
W20260709 13:07:29.146494 22827 rpcz_store.cc:269] 0709 13:07:27.742920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.745692 (+  2772us) service_pool.cc:224] Handling call
0709 13:07:29.145056 (+1399364us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b54002f0dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.145187 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94667648}
I20260709 13:07:29.147296 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1349) took 2933 ms. Trace:
I20260709 13:07:29.147599 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.139810 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.149555 22823 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:29.140956 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.111202 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:29.091691 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.013013 23446 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 1.
I20260709 13:07:29.150961 25052 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.006381 24616 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.152738 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 726) took 1273 ms. Trace:
I20260709 13:07:29.144412 25040 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:29.168067 23449 rpcz_store.cc:276] 0709 13:07:27.879831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.880075 (+   244us) service_pool.cc:224] Handling call
0709 13:07:29.140845 (+1260770us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6e9c 00007fcb71cd6a5b 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:07:29.152665 (+ 11820us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8612608}
W20260709 13:07:29.170881 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.109722 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.182554 23445 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:29.138820 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.183264 25019 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:29.174234 25064 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 13:07:29.174625 25065 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:07:29.177754 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 467) took 1263 ms. Trace:
I20260709 13:07:29.188324 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.178265 24997 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:29.178789 24916 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.179275 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 403) took 1674 ms. Trace:
I20260709 13:07:29.179395 25018 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.180545 25009 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.183629 23447 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
W20260709 13:07:29.177206 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.141949 25026 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:29.178794 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.147713 22822 rpcz_store.cc:276] 0709 13:07:26.213862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.544481 (+330619us) service_pool.cc:224] Handling call
0709 13:07:26.587457 (+ 42976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.741115 (+1153658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.147040 (+1405925us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b54001fa9d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:29.147236 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2158998912}
I20260709 13:07:29.187294 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:29.181046 24482 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:29.174234 25062 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:07:29.188679 22903 rpcz_store.cc:276] 0709 13:07:27.914741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.914865 (+   124us) service_pool.cc:224] Handling call
0709 13:07:29.177615 (+1262750us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b5400206554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.177701 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2291341696}
I20260709 13:07:29.202595 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:29.190264 24955 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.001s
I20260709 13:07:29.211493 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:29.210367 25003 consensus_meta.cc:287] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.000s
W20260709 13:07:29.219534 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.209427 25063 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:29.205706 25068 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:29.218672 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.237772 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.209887 24955 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:29.246121 25019 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.222199 25003 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.253587 23445 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:29.253777 22827 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:29.192276 23448 rpcz_store.cc:276] 0709 13:07:27.505102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.506940 (+  1838us) service_pool.cc:224] Handling call
0709 13:07:27.549836 (+ 42896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.724430 (+174594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.727093 (+  2663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:27.730602 (+  3509us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:27.733580 (+  2978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.179054 (+1445474us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400750154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:29.179100 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.179219 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378044288}
W20260709 13:07:29.256172 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.257994 25012 consensus_meta.cc:287] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.001s
I20260709 13:07:29.258587 25012 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.259865 25007 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:29.221855 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.261484 22896 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
W20260709 13:07:29.261166 22829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 325) took 1523 ms (client timeout 1648 ms). Trace:
W20260709 13:07:29.263820 22829 rpcz_store.cc:269] 0709 13:07:27.737981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.740603 (+  2622us) service_pool.cc:224] Handling call
0709 13:07:29.260973 (+1520370us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400081e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.261097 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2841329536}
I20260709 13:07:29.265130 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.265873 22829 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:29.254719 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.267445 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.270268 22829 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:29.271723 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.280071 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.261099 25012 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.283478 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.284049 22829 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:29.284785 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.268239 23446 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:29.286233 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.287166 22829 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:29.288311 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.289273 22829 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:29.290442 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.290949 22829 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:29.281996 25003 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.291908 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.281584 22822 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:29.293502 22829 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:29.286675 24810 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.300963 25063 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:29.307132 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.313239 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.305284 24616 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:29.308585 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.312204 24955 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:29.327800 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.328457 23449 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:29.317080 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.332901 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.336597 24877 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:29.339480 25029 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:29.346608 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 486) took 1214 ms. Trace:
I20260709 13:07:29.347091 22753 rpcz_store.cc:276] 0709 13:07:28.132395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.132525 (+   130us) service_pool.cc:224] Handling call
0709 13:07:28.132892 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.211227 (+ 78335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:28.211256 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.211307 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:28.211343 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.346297 (+1134954us) spinlock_profiling.cc:243] Waited 78.1 ms on lock 0x7b54002aa154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:29.346360 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.346539 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179733888}
W20260709 13:07:29.357753 25071 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:07:29.363426 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 331) took 1448 ms. Trace:
I20260709 13:07:29.365172 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 330) took 1450 ms. Trace:
I20260709 13:07:29.365017 22823 rpcz_store.cc:276] 0709 13:07:27.915341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.153455 (+1238114us) service_pool.cc:224] Handling call
0709 13:07:29.163878 (+ 10423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.164413 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.164454 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.166985 (+  2531us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.285864 (+118879us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:29.328858 (+ 42994us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.332615 (+  3757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.332680 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.363123 (+ 30443us) raft_consensus.cc:1685] finished
0709 13:07:29.363154 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.363342 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6355,"prepare.queue_time_us":42292,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":29071,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4358,"thread_start_us":112085,"threads_started":2,"wal-append.queue_time_us":133168}
I20260709 13:07:29.370528 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.337092 24877 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:29.402086 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57774 (request call id 12) took 1658 ms. Trace:
I20260709 13:07:29.412873 22831 rpcz_store.cc:276] 0709 13:07:27.743353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.743475 (+   122us) service_pool.cc:224] Handling call
0709 13:07:29.401841 (+1658366us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b5400091ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74df5853 00007fcb79dc360a 00007fcb733497d4 00007fcb73349762 00007fcb73349691 00007fcb7334960c 00007fcb733488c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.401986 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107728768}
I20260709 13:07:29.413439 22823 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:07:29.414127 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 335) took 1495 ms. Trace:
I20260709 13:07:29.388399 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2120) took 1043 ms. Trace:
I20260709 13:07:29.415099 22756 rpcz_store.cc:276] 0709 13:07:28.344968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.345064 (+    96us) service_pool.cc:224] Handling call
0709 13:07:28.784921 (+439857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:28.786517 (+  1596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.388162 (+601645us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b5400336dd0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.388322 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010949504}
I20260709 13:07:29.401548 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:29.414413 22823 rpcz_store.cc:276] 0709 13:07:27.918763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.412888 (+1494125us) service_pool.cc:224] Handling call
0709 13:07:29.413202 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.413896 (+   694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.414077 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.418041 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:29.418676 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1935) took 1229 ms. Trace:
I20260709 13:07:29.418601 23419 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:29.398195 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1948) took 1092 ms. Trace:
I20260709 13:07:29.419188 22823 rpcz_store.cc:276] 0709 13:07:28.189040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.417941 (+1228901us) service_pool.cc:224] Handling call
0709 13:07:29.418184 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.418345 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.418370 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:29.418424 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:29.418492 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.418543 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.418631 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.420603 22826 rpcz_store.cc:276] 0709 13:07:28.308851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.414111 (+105260us) service_pool.cc:224] Handling call
0709 13:07:29.397992 (+983881us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54000ac3d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.398144 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572021632}
W20260709 13:07:29.419620 23453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 732) took 1285 ms (client timeout 1632 ms). Trace:
I20260709 13:07:29.383381 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2099) took 3176 ms. Trace:
W20260709 13:07:29.419137 23450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 401) took 1921 ms (client timeout 2390 ms). Trace:
W20260709 13:07:29.423934 23450 rpcz_store.cc:269] 0709 13:07:27.497456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.498553 (+  1097us) service_pool.cc:224] Handling call
0709 13:07:29.387727 (+1889174us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54007d0fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.419080 (+ 31353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848984448}
W20260709 13:07:29.422048 23453 rpcz_store.cc:269] 0709 13:07:28.133793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.141574 (+  7781us) service_pool.cc:224] Handling call
0709 13:07:29.419462 (+1277888us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54007d0354. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.419569 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735032064}
I20260709 13:07:29.424693 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.422797 22751 rpcz_store.cc:276] 0709 13:07:26.206936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.207036 (+   100us) service_pool.cc:224] Handling call
0709 13:07:26.228555 (+ 21519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.588270 (+1359715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.383082 (+1794812us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540016a154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:29.383306 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3125060352}
I20260709 13:07:29.425376 22903 raft_consensus.cc:2368] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:29.365702 22830 rpcz_store.cc:276] 0709 13:07:27.914530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.151044 (+1236514us) service_pool.cc:224] Handling call
0709 13:07:29.151369 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.151577 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.151605 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.152043 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.282501 (+130458us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:29.327852 (+ 45351us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.328587 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.328652 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.364997 (+ 36345us) raft_consensus.cc:1685] finished
0709 13:07:29.365014 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.365125 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45414,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":365,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":127644,"threads_started":2,"wal-append.queue_time_us":146842}
I20260709 13:07:29.420106 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.426532 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.427146 22903 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:29.428203 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.421736 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1936) took 1232 ms. Trace:
I20260709 13:07:29.430015 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.385165 25020 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:29.430171 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.430655 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.431200 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:29.431797 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.429575 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:29.471908 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 53931.3us
I20260709 13:07:29.472733 22823 rpcz_store.cc:276] 0709 13:07:28.189350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.421020 (+1231670us) service_pool.cc:224] Handling call
0709 13:07:29.421244 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.421417 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.421441 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:29.421503 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:29.421541 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.421593 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.421687 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:29.473393 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46976us
I20260709 13:07:29.473847 22903 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:29.428694 23444 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:29.418892 24999 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:29.474506 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.398648 24851 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:29.496748 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:29.497251 22823 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:07:29.474531 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51370.2us
I20260709 13:07:29.471967 22896 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:29.503199 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.503809 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.505577 23452 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:29.505712 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.506279 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.506914 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:29.507997 22823 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:07:29.508293 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.507057 25025 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:29.499229 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.499933 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:29.528198 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:29.506289 22900 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:29.530072 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.530267 22904 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:29.532004 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.534179 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.425179 25003 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:29.475147 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1937) took 1285 ms. Trace:
I20260709 13:07:29.500552 22899 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.502123 24947 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.517766 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:29.509385 25021 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:29.587944 22825 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:29.588670 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1938) took 1398 ms. Trace:
I20260709 13:07:29.589035 22825 rpcz_store.cc:276] 0709 13:07:28.190106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.554603 (+1364497us) service_pool.cc:224] Handling call
0709 13:07:29.576016 (+ 21413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.588424 (+ 12408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.588622 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.518607 25007 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:29.519164 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.540714 25012 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:29.500012 24997 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:07:29.594133 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.594362 24997 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.594841 24997 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:29.502497 24851 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:29.596722 24851 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:29.519799 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.592872 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.520236 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.521138 22794 leader_election.cc:400] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:29.598728 22794 leader_election.cc:403] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:29.599229 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 78270.5us
I20260709 13:07:29.521576 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:29.563827 24995 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:29.523383 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.606508 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.607539 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.370621 23448 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:29.395661 25052 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.476142 24616 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:29.535912 22756 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:29.536955 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.537688 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.626726 22825 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:07:29.627019 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 89416.7us
I20260709 13:07:29.627404 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1939) took 1328 ms. Trace:
I20260709 13:07:29.627810 22825 rpcz_store.cc:276] 0709 13:07:28.298658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.589909 (+1291251us) service_pool.cc:224] Handling call
0709 13:07:29.626475 (+ 36566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.627084 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.627349 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.628700 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.499068 24999 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:29.536213 24731 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:29.501003 25066 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:29.472262 25020 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:29.544350 23451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 731) took 1412 ms (client timeout 1506 ms). Trace:
I20260709 13:07:29.548293 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.549966 23444 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:29.552445 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:29.553051 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:29.638239 22754 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:29.553802 22828 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 329) took 1639 ms (client timeout 1704 ms). Trace:
I20260709 13:07:29.543135 25003 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.554749 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1943) took 1249 ms. Trace:
W20260709 13:07:29.642903 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.643494 22825 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:29.555547 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.558763 22751 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:29.644328 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 345) took 1034 ms. Trace:
I20260709 13:07:29.505220 24986 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:07:29.655544 25052 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.656050 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 334) took 1739 ms. Trace:
I20260709 13:07:29.557948 24877 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.657425 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 332) took 1741 ms. Trace:
I20260709 13:07:29.656446 22827 rpcz_store.cc:276] 0709 13:07:27.916946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.386212 (+1469266us) service_pool.cc:224] Handling call
0709 13:07:29.386598 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.386966 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.387000 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.387536 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.514130 (+126594us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:29.560876 (+ 46746us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.592829 (+ 31953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.605091 (+ 12262us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.655851 (+ 50760us) raft_consensus.cc:1685] finished
0709 13:07:29.655874 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.655995 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34748,"prepare.queue_time_us":46778,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":51337,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":8202,"thread_start_us":126208,"threads_started":1,"wal-append.queue_time_us":213316}
I20260709 13:07:29.657800 22829 rpcz_store.cc:276] 0709 13:07:27.915882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.294291 (+1378409us) service_pool.cc:224] Handling call
0709 13:07:29.294559 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.305115 (+ 10556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.305150 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.305593 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.380107 (+ 74514us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:29.381671 (+  1564us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.381857 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.381948 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.653110 (+271162us) raft_consensus.cc:1685] finished
0709 13:07:29.653137 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.657369 (+  4232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":675,"prepare.queue_time_us":2371,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":1443,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":3935,"spinlock_wait_cycles":22400,"thread_start_us":71329,"threads_started":1,"wal-append.queue_time_us":234092}
I20260709 13:07:29.562943 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.659996 22829 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:07:29.660548 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 347) took 1020 ms. Trace:
I20260709 13:07:29.661814 24999 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.662892 24999 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:29.561714 22826 rpcz_store.cc:276] 0709 13:07:28.189619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.421407 (+1231788us) service_pool.cc:224] Handling call
0709 13:07:29.432011 (+ 10604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.474870 (+ 42859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.474892 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:29.474931 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:29.474966 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.475016 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.475100 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.665558 24916 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:29.667084 22826 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:07:29.668756 22826 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:29.564460 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.670076 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.669840 25020 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.670804 25020 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.671134 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.539490 24936 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:29.672740 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.674937 25003 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.629814 24955 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:29.675704 25032 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:29.676476 24881 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.568313 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.677860 22904 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:29.537456 25015 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:29.573350 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.678928 22898 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:29.576040 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.576948 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.522437 25010 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.600245 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.573084 25068 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:29.640115 25066 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:07:29.681491 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 81312.4us
I20260709 13:07:29.681864 25066 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:29.682199 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.642661 22824 rpcz_store.cc:276] 0709 13:07:28.304998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.624724 (+319726us) service_pool.cc:224] Handling call
0709 13:07:29.552547 (+927823us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b54001ee6d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.554703 (+  2156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189316480}
I20260709 13:07:29.683259 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.684530 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.602504 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.620918 22830 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:07:29.687482 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1942) took 1382 ms. Trace:
I20260709 13:07:29.687788 22830 rpcz_store.cc:276] 0709 13:07:28.304751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.620237 (+1315486us) service_pool.cc:224] Handling call
0709 13:07:29.620560 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.687260 (+ 66700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.687433 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.622612 22901 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.691123 24877 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.692859 22824 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:07:29.621246 22903 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:29.693977 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.694972 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.695681 22903 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:29.598026 24851 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:29.696902 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 128887us
I20260709 13:07:29.660851 22829 rpcz_store.cc:276] 0709 13:07:28.640321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.659516 (+1019195us) service_pool.cc:224] Handling call
0709 13:07:29.659797 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.660333 (+   536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.660501 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.698783 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.699312 22829 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:29.699743 22902 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:07:29.700145 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:29.702107 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:29.639060 22828 rpcz_store.cc:269] 0709 13:07:27.914203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.946705 (+ 32502us) service_pool.cc:224] Handling call
0709 13:07:29.553646 (+1606941us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54001f9ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.553754 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484603776}
I20260709 13:07:29.705858 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:29.553932 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:29.635977 23451 rpcz_store.cc:269] 0709 13:07:28.132312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.132478 (+   166us) service_pool.cc:224] Handling call
0709 13:07:29.544203 (+1411725us) spinlock_profiling.cc:243] Waited 92 ms on lock 0x7b5400620f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:29.544302 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211621760}
I20260709 13:07:29.637295 23444 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:29.575659 24482 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:07:29.605023 24995 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.543460 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.711169 22753 raft_consensus.cc:2368] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6.
W20260709 13:07:29.646271 25079 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:07:29.647240 22652 catalog_manager.cc:5697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:29.650787 24810 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.629599 24616 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:29.718102 25009 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:29.718207 24616 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.718575 25009 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:29.718953 25009 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:29.720135 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.707442 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.705053 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:29.708158 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.708153 22897 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:07:29.713150 22754 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:29.613909 22823 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:29.724157 22823 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:29.725261 22823 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:29.733923 22754 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:29.693977 22824 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:07:29.644678 22825 rpcz_store.cc:276] 0709 13:07:28.609844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.628622 (+1018778us) service_pool.cc:224] Handling call
0709 13:07:29.643340 (+ 14718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.643968 (+   628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.644281 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:29.701115 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:29.713871 24980 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:29.716107 25015 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:29.716777 22899 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:29.714109 23444 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.763784 23444 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:29.717625 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:29.764660 22754 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:29.735598 23450 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:29.735743 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.709824 22901 raft_consensus.cc:2410] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:29.738217 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.714097 24995 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.726456 22823 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:29.766821 22755 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:29.739383 22753 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:29.734887 22652 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.768239 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.740912 25003 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.721393 25002 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:29.735921 25018 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:29.774945 22754 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
W20260709 13:07:29.777194 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 217761us
I20260709 13:07:29.779474 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:29.785854 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.787503 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:29.789919 22871 leader_election.cc:400] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6.
I20260709 13:07:29.790385 22871 leader_election.cc:403] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:07:29.791369 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:07:29.768535 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 268680us
I20260709 13:07:29.777402 22652 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.796051 22652 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.797978 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.798988 24482 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:29.785166 25003 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.798091 22652 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.735184 24997 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.800374 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:29.780346 25015 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:29.804234 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:29.800915 22652 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.804736 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:29.774185 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.806433 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.806505 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:29.808169 24992 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:29.809922 24992 consensus_queue.cc:1059] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:07:29.815577 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 186917us
I20260709 13:07:29.817996 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.820312 24809 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:29.823055 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 84898.2us
I20260709 13:07:29.823616 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:29.827550 24809 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:29.827966 24809 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.828336 24809 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:29.829959 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.831987 25032 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:29.834504 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.836061 25032 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:07:29.844141 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.844350 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:29.844802 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.781145 25002 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.845861 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.837306 24809 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.799556 25019 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:29.797576 25051 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6.
I20260709 13:07:29.868220 23450 raft_consensus.cc:3055] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:29.868650 23450 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:29.868920 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62473us
I20260709 13:07:29.870436 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:29.854296 22652 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:29.858537 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:29.845165 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:29.847319 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.863762 25002 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:29.778919 25006 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:07:29.802172 24482 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:29.843937 25047 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:29.908576 24809 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.876984 23452 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:29.909020 25047 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:29.909508 25047 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:29.874680 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:29.870462 25015 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:29.912317 25020 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:29.849586 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:29.881016 22829 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:29.913892 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 338) took 1928 ms. Trace:
I20260709 13:07:29.876049 22652 catalog_manager.cc:5697] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 2 to 4, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.914247 22831 rpcz_store.cc:276] 0709 13:07:27.987590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.413719 (+1426129us) service_pool.cc:224] Handling call
0709 13:07:29.414034 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.414378 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.414403 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.674456 (+260053us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b2c00000e98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:07:29.674820 (+   364us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.770731 (+ 95911us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:29.792837 (+ 22106us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.814800 (+ 21963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.814868 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.913658 (+ 98790us) raft_consensus.cc:1685] finished
0709 13:07:29.913739 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.913843 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1303,"prepare.queue_time_us":22236,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":4032,"raft.queue_time_us":3125,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":9134,"spinlock_wait_cycles":28217600,"thread_start_us":117305,"threads_started":2,"wal-append.queue_time_us":236906}
I20260709 13:07:29.915748 22652 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 2 to 4, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.873308 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 333) took 1954 ms. Trace:
I20260709 13:07:29.860769 25010 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:29.917816 25010 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:29.918579 25010 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:29.860652 24947 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:29.864780 24890 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:29.921039 24947 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:29.921605 24947 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.878230 24975 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:29.892563 25007 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:29.892706 24482 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:29.912839 25020 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.925372 25020 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.795658 24810 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
W20260709 13:07:29.894364 25029 consensus_meta.cc:287] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
I20260709 13:07:29.805706 25021 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:07:29.876443 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:29.870167 25019 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:29.935879 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 59509.4us
I20260709 13:07:29.911653 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:29.921098 24890 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:29.881803 23450 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.927273 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:29.878463 25030 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:29.928722 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:29.792734 25016 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 13:07:29.930848 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60511us
I20260709 13:07:29.860522 24860 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:07:29.931993 22752 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:07:29.937098 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:29.929579 25079 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:29.800750 25085 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:07:29.878901 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:29.947398 23450 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:29.951107 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.884332 25087 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:29.938702 24885 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:29.946591 24890 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:29.970615 25098 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:07:29.973524 25083 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:07:29.973924 25093 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:29.941499 25019 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:29.943825 25063 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:29.977725 25051 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:07:29.987891 25098 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:30.008124 25098 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.009687 25098 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:30.019568 24995 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.967485 25087 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:30.025779 25087 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.951849 24916 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:29.946059 25030 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:29.912810 25015 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:29.995065 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 346) took 1384 ms. Trace:
I20260709 13:07:29.995266 24997 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:29.996403 25003 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:29.947008 25029 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:07:29.982419 24616 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:29.999814 24947 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:29.917538 22822 rpcz_store.cc:276] 0709 13:07:27.916391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.299132 (+1382741us) service_pool.cc:224] Handling call
0709 13:07:29.299454 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.306353 (+  6899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.308902 (+  2549us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.329664 (+ 20762us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.407134 (+ 77470us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b4800130158. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb79d24aa1 00007fcb79d1bb37 00007fcb79d1b9ca 00007fcb79e1326f 00007fcb79e12f37 00007fcb79e2755c 00007fcb79e27508 00007fcb79e27488 00007fcb79e27443 00007fcb79e266ec 00007fcb78d50a96
0709 13:07:29.474671 (+ 67537us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:29.543891 (+ 69220us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.598435 (+ 54544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.622775 (+ 24340us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.784652 (+161877us) raft_consensus.cc:1685] finished
0709 13:07:29.849868 (+ 65216us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.871077 (+ 21209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30790,"prepare.queue_time_us":30036,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":1531,"raft.queue_time_us":35221,"raft.run_cpu_time_us":2804,"raft.run_wall_time_us":100780,"spinlock_wait_cycles":3037056,"thread_start_us":32283,"threads_started":1,"wal-append.queue_time_us":154486}
I20260709 13:07:30.067567 24936 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:29.975426 25052 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:07:30.068225 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 81552.1us
I20260709 13:07:30.008730 25066 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:30.075616 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.028762 25009 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:30.077803 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 232014us
I20260709 13:07:30.081779 24810 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:07:30.096240 25090 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:07:30.077117 23450 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:07:30.082759 24995 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:29.993893 24860 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.988040 23444 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:07:30.118968 23444 pending_rounds.cc:85] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 4
I20260709 13:07:30.119323 23444 pending_rounds.cc:107] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:07:30.119576 23444 raft_consensus.cc:2889] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:30.119897 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:30.098697 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 151523us
W20260709 13:07:30.099160 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.069940 25030 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:30.068188 25025 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:29.975420 24885 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:30.113685 23450 pending_rounds.cc:85] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 4
I20260709 13:07:30.132736 23450 pending_rounds.cc:107] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:07:30.133069 23450 raft_consensus.cc:2889] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:30.083156 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 339047us
I20260709 13:07:30.114442 23449 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:30.115144 22751 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:30.083580 25063 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:07:30.125072 22830 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:30.125468 22897 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:30.136014 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:30.126169 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.146409 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.147123 22900 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:30.138314 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:30.148702 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.149638 23444 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:30.140221 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:30.141438 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 197471us
I20260709 13:07:30.155047 23453 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:30.155997 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.127404 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:30.108604 25052 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.158900 22903 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:30.139438 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.113224 24881 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:07:30.182224 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54902.1us
W20260709 13:07:30.155606 24996 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:07:30.140797 24885 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.162122 25052 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:30.190696 25103 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:07:30.189085 25095 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:07:30.191439 25112 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:30.191489 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:30.236676 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 45283.4us
I20260709 13:07:30.184065 25063 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:30.194813 25107 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:07:30.140830 25012 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:07:30.196750 24809 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:30.120729 24955 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:07:30.134262 25111 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:30.156262 25003 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:30.158296 25093 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:30.213213 22750 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:30.215510 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:30.181102 24966 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:07:30.217478 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62403.1us
W20260709 13:07:30.222743 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:30.262764 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.124722 24860 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.233790 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:30.148092 25032 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:07:30.235339 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.153856 25020 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.238261 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.186580 25114 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.077327 22827 rpcz_store.cc:276] 0709 13:07:28.610515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.658361 (+1047846us) service_pool.cc:224] Handling call
0709 13:07:29.658692 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.659016 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.659050 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.659496 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.843254 (+183758us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:29.966175 (+122921us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.994765 (+ 28590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.994844 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.994885 (+    41us) raft_consensus.cc:1685] finished
0709 13:07:29.994900 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.995012 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43062,"prepare.queue_time_us":88336,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":4077,"raft.queue_time_us":18113,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":207,"spinlock_wait_cycles":1408,"thread_start_us":183357,"threads_started":1,"wal-append.queue_time_us":216627}
I20260709 13:07:30.245206 25009 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.246272 25066 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:30.232745 25092 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:07:30.250383 24811 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.187268 25047 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:30.261458 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46013.7us
I20260709 13:07:30.263927 24966 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:30.250576 24809 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:30.255326 25123 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.336596 25123 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.336926 24809 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.338212 24809 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.338661 25123 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.253211 25029 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:30.339200 25029 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:30.288676 25110 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:07:30.292701 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 57420.7us
I20260709 13:07:30.292717 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:30.342476 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 54583.8us
I20260709 13:07:30.343369 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:30.345609 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.273613 25019 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.272384 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:30.358389 22748 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:30.316983 24881 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.252875 25087 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.285753 24890 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.250002 25063 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.292598 25010 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.252543 25096 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:07:30.341496 24616 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.264878 25106 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.294154 25111 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:30.331972 25128 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:30.340025 25029 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.332424 25020 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.334959 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:30.386543 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51682.9us
W20260709 13:07:30.293489 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 55869.5us
W20260709 13:07:30.347890 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.359520 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.334432 24947 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.400348 23416 leader_election.cc:400] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4.
I20260709 13:07:30.401548 23444 raft_consensus.cc:2368] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
W20260709 13:07:30.360677 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88393us
I20260709 13:07:30.350723 25015 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:30.342150 25123 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.409976 25020 raft_consensus.cc:3055] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:30.409850 25123 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.410400 25020 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:30.411001 25019 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.411761 25123 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.346925 25009 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.412037 25114 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.370819 23452 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:07:30.371708 24890 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.371709 25016 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.413606 25047 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.413707 25084 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:30.414213 25084 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:30.414332 25029 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.366729 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.372376 25010 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.362977 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:30.373970 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.375612 25095 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:30.376132 24860 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:30.450358 23453 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:30.380272 25063 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.369513 25116 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:30.381858 25087 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.367012 24986 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:30.466041 24986 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:30.466666 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:30.466921 24986 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.468129 24986 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.340837 24811 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.402500 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:30.470986 22756 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:30.402987 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:30.472247 22750 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:30.400617 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:30.411235 25020 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:30.373004 22747 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:30.470374 25016 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:30.424813 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 742) took 1026 ms. Trace:
I20260709 13:07:30.430284 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:30.421012 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:30.492589 23449 rpcz_store.cc:276] 0709 13:07:29.398746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.400247 (+  1501us) service_pool.cc:224] Handling call
0709 13:07:29.545364 (+145117us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:29.545516 (+   152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.134791 (+589275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.424591 (+289800us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b540056ac54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:30.424751 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1313459328}
W20260709 13:07:30.493251 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72311.3us
I20260709 13:07:30.383116 24966 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.381071 25002 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.388358 25098 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:30.391878 25111 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:07:30.464567 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63402.3us
I20260709 13:07:30.457310 25066 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:07:30.474504 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:30.516343 25131 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:30.516739 25131 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:30.517500 25131 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.423761 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.504910 22826 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:30.520301 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.523608 22825 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.522533 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.521785 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.505327 25002 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.505743 24810 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:30.505852 25010 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:30.509402 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:30.393976 25007 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 13:07:30.513886 24811 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.499634 25032 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:07:30.515285 24885 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.532105 24810 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.526814 22897 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:30.527577 22904 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:30.527710 22750 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:30.525926 24966 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.400318 25052 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.472352 25116 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:30.530699 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:30.565652 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:30.508767 25025 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:30.451743 23445 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:30.535181 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.535804 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.533486 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:30.535992 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.536525 22794 leader_election.cc:400] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
I20260709 13:07:30.449844 25047 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:07:30.517171 24482 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:30.539898 22901 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:30.518950 25131 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.519275 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.549389 22822 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:30.576388 22903 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:30.550423 23450 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:30.548578 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:30.577335 22751 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:30.575881 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.564544 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:30.548893 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.532433 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:30.568216 22831 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:30.539206 25020 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.531860 25019 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.548593 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.534677 25010 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:30.569665 22794 leader_election.cc:403] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:07:30.553575 24731 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:30.548978 25111 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.587711 24731 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:30.588596 24731 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.590240 24731 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.539352 25104 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:07:30.591411 25104 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:30.591732 25104 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:30.568902 25116 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.551219 24885 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:30.583523 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:30.509974 24860 consensus_queue.cc:1175] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c45e1805f049467697a7358fb566de2f" responder_term: 3 status { last_received { term: 3 index: 6 } last_committed_idx: 4 last_received_current_leader { term: 3 index: 6 } }
I20260709 13:07:30.547756 25127 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:07:30.603312 25025 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:30.603806 25025 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:30.592409 25010 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.605288 25104 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.607460 25104 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:30.574374 24482 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.587188 25029 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:30.588968 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.583098 22825 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:30.604671 25025 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:30.595957 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 59516.7us
I20260709 13:07:30.619468 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.608235 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:07:30.585388 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 53053.5us
I20260709 13:07:30.610805 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:30.611685 25114 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:30.612453 24947 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:30.540135 25128 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:07:30.616138 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.580132 25068 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.621753 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:30.548748 25030 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.586048 24817 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:30.586341 25087 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.620536 25127 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:30.592739 24809 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:07:30.617202 24482 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.616195 24851 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
I20260709 13:07:30.622262 25114 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:30.593703 25063 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:07:30.649612 22825 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:07:30.624193 23446 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:30.640234 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.643112 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.621536 24966 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:07:30.631618 24817 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:30.677501 24817 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.678475 25030 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.678792 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1372) took 1296 ms. Trace:
I20260709 13:07:30.679386 25063 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:30.679124 22830 rpcz_store.cc:276] 0709 13:07:29.381992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.688451 (+306459us) service_pool.cc:224] Handling call
0709 13:07:29.719604 (+ 31153us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b5800144f98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:29.719761 (+   157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.152187 (+432426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.678590 (+526403us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b540043b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:30.678742 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976309248}
I20260709 13:07:30.679876 25063 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:30.673794 24811 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.681749 24811 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.683635 24811 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:30.642867 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:30.686187 23453 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:30.687815 25052 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.658546 25009 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:30.688493 25009 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.688854 25009 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.636511 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.655893 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.623934 24731 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:30.624531 24947 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:30.628440 25007 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:07:30.624114 24916 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:30.699587 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:30.657969 25085 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:30.660631 25018 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:30.662531 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.713546 25018 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.715134 23446 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:07:30.699708 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.715555 25018 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.662827 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.663223 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.663383 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.663793 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:30.664932 22754 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:30.627408 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:30.639732 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.633018 25123 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:30.676095 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.675488 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:30.641209 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:30.723047 22753 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:07:30.723471 22753 pending_rounds.cc:85] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 3
I20260709 13:07:30.660513 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.696925 22828 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:30.697543 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 90278.9us
I20260709 13:07:30.680665 25063 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.728387 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.657907 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.733713 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:30.735128 22904 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:30.722882 25085 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.739445 25085 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:30.740373 25098 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:30.741068 23453 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:30.743189 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:30.743757 22825 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:30.745461 25128 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:30.746140 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.745934 25128 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.747171 22899 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:30.699188 25030 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:30.671761 25025 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:30.703320 25122 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.331s	user 0.002s	sys 0.000s
I20260709 13:07:30.698424 24731 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.704627 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:30.658967 25029 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:07:30.705757 24947 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.706614 25096 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.708123 24810 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:30.709023 25125 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:07:30.709769 25129 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.336s	user 0.000s	sys 0.001s
W20260709 13:07:30.710083 25135 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:07:30.718784 23449 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.721127 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 486) took 1159 ms. Trace:
W20260709 13:07:30.724301 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 96947.8us
I20260709 13:07:30.799621 22749 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:30.797665 22900 rpcz_store.cc:276] 0709 13:07:29.562207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.562687 (+   480us) service_pool.cc:224] Handling call
0709 13:07:30.720988 (+1158301us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b540034d0d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:30.721080 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1044203904}
I20260709 13:07:30.724027 22896 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:30.723703 22753 pending_rounds.cc:107] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:07:30.725502 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.726267 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:30.801244 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.630805 25082 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:07:30.726779 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.727877 25007 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.747946 25128 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.757895 25116 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.758837 24986 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.760109 24817 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.761381 22828 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:30.763880 25010 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.766278 25130 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:30.767132 24482 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:30.772380 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:30.700147 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44325.8us
I20260709 13:07:30.784686 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:30.791465 23419 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:30.801290 22753 raft_consensus.cc:2889] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:30.650144 25084 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:30.801726 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76304.8us
I20260709 13:07:30.822575 25087 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:30.816452 22829 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.804841 22900 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:30.792582 24731 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.802285 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:30.795672 25125 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:30.850538 25116 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.851619 24817 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.851819 25098 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:30.789601 25085 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:07:30.852739 25085 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:30.853116 25085 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:30.853979 25085 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.854811 25010 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.855172 25085 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.856868 25009 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.857472 22751 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:30.858079 22822 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:30.846504 25015 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.858906 25009 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.827246 24986 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.861569 24986 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:30.844189 25096 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.862434 25096 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.863353 23448 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:30.864800 25096 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.844144 25087 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:30.866273 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1248) took 1176 ms. Trace:
I20260709 13:07:30.866138 25087 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.866645 22896 rpcz_store.cc:276] 0709 13:07:29.709622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.714026 (+  4404us) service_pool.cc:224] Handling call
0709 13:07:30.721693 (+1007667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.815511 (+ 93818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.866090 (+ 50579us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540034d0d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:30.866219 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316728704}
I20260709 13:07:30.867303 25087 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.834877 22901 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:30.834481 25114 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.835294 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.833496 25130 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:30.871577 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:30.871320 25130 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.872645 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.872864 25130 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:30.839434 23449 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
W20260709 13:07:30.873804 25142 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:07:30.842633 24482 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:30.813908 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.819726 25032 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:07:30.844708 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53358us
I20260709 13:07:30.886874 25032 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:07:30.887001 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 73186.4us
I20260709 13:07:30.887754 25032 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.889777 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.890368 23445 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:30.891969 25098 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:30.893381 25098 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:30.893704 25098 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:30.893996 24731 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.895102 22903 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:30.895984 24731 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.897675 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:30.898259 23449 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:30.898859 22829 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
W20260709 13:07:30.844839 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 60232.6us
I20260709 13:07:30.844983 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:30.901671 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:30.901945 22753 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.823011 24947 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:30.848496 22826 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:30.851158 25125 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.791085 24966 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:07:30.829213 24810 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:30.909168 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:30.873507 24817 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:30.842036 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:30.873693 25029 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:30.874131 25010 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:30.874984 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72771.5us
I20260709 13:07:30.875445 24616 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:30.873854 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:30.904120 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:30.904448 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:30.904615 23449 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 4 LEARNER]: Advancing to term 5
I20260709 13:07:30.903887 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:30.908097 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:30.912556 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 70594.3us
I20260709 13:07:30.930631 22903 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:30.933281 22756 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.936385 22829 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:30.937356 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:30.933813 23450 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:30.937620 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:30.938398 22830 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.939245 25025 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:30.939625 25025 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.939977 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:30.940547 23448 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.940006 25025 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.934803 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:30.956943 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:30.908643 25096 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:30.894441 25098 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.909649 25009 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.324s
I20260709 13:07:31.234627 23445 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:30.939451 22822 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:31.302568 22753 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:31.377813 25095 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:31.388847 25025 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:31.389907 25131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.017s	sys 0.024s
W20260709 13:07:31.390657 25131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.017s	sys 0.024s
I20260709 13:07:31.393332 25025 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:31.406893 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:31.407831 23453 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.415880 23449 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:31.416105 23453 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:31.417894 25063 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:30.910604 25015 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:31.450155 25015 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.452589 25015 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:30.968703 25125 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:31.467536 22747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.015s	sys 0.000s
W20260709 13:07:31.467918 22747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.015s	sys 0.000s
W20260709 13:07:31.483163 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 545730us
W20260709 13:07:31.496578 23448 consensus_meta.cc:287] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.555s	user 0.007s	sys 0.007s
I20260709 13:07:31.497118 23448 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
W20260709 13:07:31.522127 25087 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.011s
W20260709 13:07:31.530360 25134 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.554s	user 0.000s	sys 0.004s
W20260709 13:07:31.538813 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 664861us
W20260709 13:07:31.546833 22756 consensus_meta.cc:287] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.001s
I20260709 13:07:31.547300 22756 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:31.547276 24947 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:31.549674 25029 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:31.554123 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:31.558260 25010 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:07:31.575709 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 245880us
I20260709 13:07:31.575858 24881 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:31.576664 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:31.577746 25012 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:31.578140 25012 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:31.579519 25012 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.582589 25140 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:31.584753 25140 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:31.584564 25088 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:31.574935 25029 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:31.585155 25140 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:31.586969 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.590839 25125 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:31.591287 24811 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.591418 25125 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.591998 24811 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.594384 24811 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.595364 24811 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:31.596009 25140 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.597270 25051 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:31.596083 24811 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.597967 24817 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:31.598052 25051 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.601868 25029 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:31.602401 25029 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:31.602542 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.603454 25029 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.604959 25029 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:31.606462 25012 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:31.614009 25125 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:31.614668 25125 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:31.615552 24966 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:31.616362 24966 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.622532 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:31.623529 25063 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:31.624073 25063 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:31.626912 25098 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.625174 25025 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
W20260709 13:07:31.631083 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50732us
I20260709 13:07:31.615583 25125 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.631220 25051 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:31.634908 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:31.638309 25088 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.641975 25088 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.647068 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:31.647931 25032 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.625275 25063 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.648343 24881 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:31.648470 25032 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.649004 24881 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:31.650784 25131 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:31.652230 25131 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:31.652568 25131 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:31.653671 25131 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:31.655843 25032 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.648417 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:31.665426 25131 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.666136 25125 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:31.666225 25012 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:31.666962 25012 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:31.668103 25012 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:30.911971 25111 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:31.672051 25111 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.676705 24811 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.679481 25111 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.679873 25114 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.650023 24881 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:31.682891 25144 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.008s	sys 0.028s
W20260709 13:07:31.683749 25144 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.008s	sys 0.028s
W20260709 13:07:31.685210 25130 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.000s
I20260709 13:07:31.689035 24966 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:31.691030 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.691316 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.692987 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:31.694111 22754 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:31.695888 22754 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:31.697432 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.700124 25002 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:31.701260 24482 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:31.703842 25002 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.704180 24482 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:31.704211 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:31.708688 24482 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.709698 24482 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:31.710065 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.712867 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:31.713829 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 756842us
I20260709 13:07:31.714671 25123 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:31.713034 24616 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:07:31.707775 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46286.1us
I20260709 13:07:31.706387 25002 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.729369 25052 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:31.729812 25052 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:31.731251 25095 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:31.731128 25052 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.731941 25095 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.732424 25052 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.734133 22903 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:31.734494 25015 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:31.735183 25015 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.724524 25106 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:31.715272 25123 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:31.725338 25111 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:31.726353 24851 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:31.728583 25084 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.737533 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.724773 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:31.747849 25106 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.748958 24851 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.750186 25106 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.750859 25140 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.751021 24851 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:31.737247 25015 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:31.747822 25111 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.753773 25095 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.757700 24916 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:07:31.763749 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:31.764559 22896 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:31.765810 25123 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.769783 22754 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:31.778918 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:31.779213 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:31.780007 22899 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:31.781432 25131 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.779891 22903 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:31.782048 25131 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.782720 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 352) took 1249 ms. Trace:
I20260709 13:07:31.783074 22826 rpcz_store.cc:276] 0709 13:07:30.533703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.533843 (+   140us) service_pool.cc:224] Handling call
0709 13:07:30.534134 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.907582 (+373448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:31.782273 (+874691us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54001f90d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:31.782551 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722475392}
I20260709 13:07:31.784309 25009 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.784953 25009 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.786731 25111 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:31.786911 25009 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.789798 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:30.955736 22830 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:31.795245 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:31.796644 22899 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:31.798785 25111 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:31.799398 25111 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.800050 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:31.800691 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:31.801338 22899 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:31.802023 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.802938 22754 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:31.803174 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:31.801494 25111 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.803714 22901 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:31.805256 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:31.806226 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:31.806522 25161 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:31.807075 25161 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.806828 22900 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:31.807600 25161 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.808314 25085 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:31.809017 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:31.806831 25111 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:31.809638 25111 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:31.809999 25111 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.813607 24482 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.813704 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:31.814462 24916 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:31.814890 24916 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:31.815235 24916 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:31.818917 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.814175 24482 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.814618 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:31.826608 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.828406 22754 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:31.826587 25015 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:31.780786 25002 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.831979 22899 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:31.833791 24890 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.844152 24890 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.849718 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:31.850368 23448 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:31.850632 24890 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:31.851358 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.852509 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:31.854430 25088 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:31.856035 25131 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.861922 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:31.831527 25015 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:31.871241 24482 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.872798 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:31.879599 25084 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:07:31.879776 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 155263us
I20260709 13:07:31.881695 22903 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:31.875684 24482 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:07:31.889130 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:31.833029 25161 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.894210 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.894385 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:31.870206 25015 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:31.896445 24482 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:31.900259 24482 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:31.897665 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:31.903525 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:31.904328 23444 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:31.904878 25002 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:31.872264 25106 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:31.910951 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:31.907069 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.918133 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:31.919984 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:31.895115 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:31.921986 23451 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:07:31.924062 23453 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:31.924422 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.923808 23451 pending_rounds.cc:85] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 5
I20260709 13:07:31.928718 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:31.921887 25052 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:31.932251 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:31.921343 25131 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:31.934131 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:31.901167 24916 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.936522 25162 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:07:31.930977 25140 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:31.945740 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:31.946751 23447 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:31.947877 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.948352 22750 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:31.948513 22825 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:31.901950 25084 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:07:31.938133 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:31.939317 22754 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:31.927244 23451 pending_rounds.cc:107] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:07:31.949640 23451 raft_consensus.cc:2889] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:31.945818 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:31.938058 23444 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:31.949996 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:31.950609 24482 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.926213 25106 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:31.949366 25002 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:31.961334 25002 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:31.962013 24482 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:31.962273 25002 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:31.957841 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:31.939870 25131 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.958698 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:31.986655 23453 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:31.934618 23445 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:31.992225 22830 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:31.992396 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080008bcc0 after lost signal to thread 21632
I20260709 13:07:31.992833 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:31.993949 22827 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:31.989900 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:31.996980 22749 raft_consensus.cc:2410] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
W20260709 13:07:31.997836 22685 debug-util.cc:398] Leaking SignalData structure 0x7b08001b57c0 after lost signal to thread 22688
W20260709 13:07:31.998298 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080008bc20 after lost signal to thread 22702
I20260709 13:07:31.965382 25140 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:07:32.000248 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.950066 25088 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:31.983640 25131 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:32.003791 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 133397us
I20260709 13:07:32.005270 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:32.006923 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800134100 after lost signal to thread 22780
I20260709 13:07:32.007265 22903 raft_consensus.cc:2368] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4.
I20260709 13:07:32.000119 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:32.008152 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:32.003791 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.008903 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:32.009528 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800134240 after lost signal to thread 22781
I20260709 13:07:32.009569 23453 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:32.012866 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.016111 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.005505 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.008203 24851 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.013486 22823 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:32.012959 23451 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 2: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:32.021142 22830 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:32.020087 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:32.023658 23444 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:07:32.024101 22685 debug-util.cc:398] Leaking SignalData structure 0x7b08002eb5a0 after lost signal to thread 22855
W20260709 13:07:32.024290 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080035a340 after lost signal to thread 22856
I20260709 13:07:32.026125 22831 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:32.001940 25140 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.031356 22901 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:07:31.979770 25111 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.004421 25088 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.042006 22899 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:32.037019 23449 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
W20260709 13:07:32.041564 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800383240 after lost signal to thread 22929
I20260709 13:07:32.026908 22822 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:32.046826 25052 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.047389 25052 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.044200 25088 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:31.894752 25095 raft_consensus.cc:2955] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:07:32.106971 22685 debug-util.cc:398] Leaking SignalData structure 0x7b08002e3360 after lost signal to thread 23478
W20260709 13:07:32.110232 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800353580 after lost signal to thread 23479
W20260709 13:07:32.110639 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800302080 after lost signal to thread 24877
W20260709 13:07:32.111019 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.045606 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:32.023464 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.045471 22829 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
W20260709 13:07:32.111042 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080033ae20 after lost signal to thread 24995
I20260709 13:07:32.050623 22903 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:32.021200 22900 raft_consensus.cc:2955] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:32.032689 25009 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:07:32.117868 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 72339.7us
I20260709 13:07:32.042519 25140 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:32.122892 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800336f80 after lost signal to thread 25089
W20260709 13:07:32.123378 22685 debug-util.cc:398] Leaking SignalData structure 0x7b0800395de0 after lost signal to thread 25097
W20260709 13:07:32.123670 22685 debug-util.cc:398] Leaking SignalData structure 0x7b08002f9440 after lost signal to thread 25099
I20260709 13:07:32.110795 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.129263 22825 raft_consensus.cc:2410] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
W20260709 13:07:32.132699 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 109648us
I20260709 13:07:32.134987 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:32.121048 24731 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:32.133268 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 90417.1us
I20260709 13:07:32.115207 25087 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.042387 25162 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 13:07:32.120286 25009 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:32.148953 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:32.055267 24851 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.151427 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:32.153038 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:32.165055 22903 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:32.143311 24996 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:32.152504 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:32.153461 25009 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:32.146124 25161 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.152755 25116 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.181840 23450 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:32.148303 25125 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.183761 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:32.187922 22903 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.167186 25123 raft_consensus.cc:2764] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:07:32.149910 23448 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:32.174888 24996 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.202852 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:32.203370 23449 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:32.205017 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 52082.3us
I20260709 13:07:32.186416 25063 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:32.166931 25087 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.200848 24996 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:32.220029 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:32.221743 25128 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:07:32.218068 25166 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:07:32.217923 22653 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.219367 24881 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.221702 25002 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:32.218403 25130 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:32.208302 22797 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:32.226967 22903 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:32.180251 24731 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:32.209336 23416 connection.cc:256] RPC call timeout handler was delayed by -1.0263113319999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 13:07:32.231765 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 47781.3us
I20260709 13:07:32.237686 25111 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.238149 25127 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:32.230406 23444 raft_consensus.cc:2955] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 LEARNER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:32.217269 25106 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:32.244581 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:32.202626 25125 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.228319 25063 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:32.244916 23449 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:32.239809 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.222903 25096 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.255441 22904 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:32.213696 25116 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.263170 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.273989 25173 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.274401 25125 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:32.274503 25173 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:32.275113 25173 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:32.243182 25002 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.282867 25002 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:32.283350 25002 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.283839 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:32.283790 25002 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.255637 25123 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:32.285377 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:32.285692 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.286301 25063 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.287149 25002 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.287011 25029 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:32.287963 25029 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.288558 25029 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:32.288884 25029 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:32.257288 25172 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:32.289608 25002 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:07:32.290145 25170 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:07:32.289810 25172 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.290207 25002 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.290887 22753 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:32.291975 25172 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.280736 25015 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.292474 25002 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.292991 22826 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:32.264180 25084 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:32.294139 25084 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.294690 24731 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.295737 22652 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 335) took 10430 ms. Trace:
I20260709 13:07:32.296113 25084 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:32.296866 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:32.260596 25111 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:32.297752 25111 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:32.298115 25111 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.300588 25052 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.300901 23450 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:32.285547 25123 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:32.302726 25116 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:32.256841 25098 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:32.303779 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.303484 25098 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.302985 22653 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.304654 25029 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.305482 25098 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:32.263758 25106 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:32.306185 25029 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.315755 24482 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:32.316437 24482 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.318599 24482 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.319126 24482 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:32.319631 24482 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:32.325462 25167 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
W20260709 13:07:32.330060 25169 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:07:32.302296 25123 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.305065 22653 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.246104 24947 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:07:32.355321 24482 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.361244 25098 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:07:32.361526 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 64797.5us
I20260709 13:07:32.362020 25098 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:32.362815 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:32.363253 25098 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.364066 25114 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.250579 24881 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.369784 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.324851 25111 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.375174 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.378597 22750 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:32.366309 25084 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:07:32.367992 25162 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.391247 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.394999 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.395262 22756 raft_consensus.cc:2410] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:07:32.403658 23450 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:32.403565 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:32.375006 25172 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:32.406864 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 121591us
I20260709 13:07:32.407106 23449 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:32.402428 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:32.408330 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:32.270341 25096 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.408829 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:32.373358 25052 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:32.411549 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.411613 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:32.359822 25140 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.384601 24731 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.380587 25128 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:32.404145 22653 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.419240 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.414513 22750 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:32.414140 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:32.413458 25184 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.421195 25172 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.422775 25087 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:32.423393 25172 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.423995 25172 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:32.424130 25178 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:32.400781 25015 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.417542 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:32.403793 25123 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.417181 22823 raft_consensus.cc:2368] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4.
I20260709 13:07:32.427191 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.427326 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.430246 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.434047 22756 raft_consensus.cc:2410] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:07:32.424563 25172 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:32.411700 24851 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:32.411492 24947 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:32.440961 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:32.445591 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:32.446866 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:32.448896 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:32.424801 25178 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:32.454638 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.456032 25084 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:32.459908 24955 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:07:32.460045 25084 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.461230 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:32.462694 25084 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:32.463426 25084 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:32.463896 25084 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.464293 25116 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:32.464738 25116 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:32.472043 25016 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:32.472563 25016 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:32.473414 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.474612 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.476714 25128 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.478677 25096 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.479101 24860 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.479532 24916 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:32.479590 24860 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.480721 24881 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.469022 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.426642 25002 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.481375 24881 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:32.482467 25002 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.484148 22748 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:07:32.483705 25016 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.484408 25002 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.464313 25084 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:32.485157 25186 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:32.486222 25186 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:32.486547 25186 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:32.488895 22822 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:32.417152 25162 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.489831 25181 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:32.490459 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.491003 25162 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:32.491808 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48253.9us
I20260709 13:07:32.492619 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:32.456671 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.494520 25140 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.495231 25179 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:32.492386 25084 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.496160 25088 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.427026 24731 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.498487 25084 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:32.498687 25009 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.499707 24731 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.419832 25052 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.500273 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.500838 22747 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:32.501753 25052 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.502907 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:32.503269 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.437230 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.504539 24811 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:32.504824 22827 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:32.411885 24996 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.504933 24811 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.505868 24860 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.506052 24811 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:32.458479 22750 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:32.507462 24996 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:32.448796 25012 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:32.509371 25088 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.509763 25012 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:32.510782 25012 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.511274 25016 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:32.513031 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 64931.2us
I20260709 13:07:32.513875 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:32.514151 24811 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.449877 25125 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.515496 25179 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:32.516196 24811 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.521898 24851 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:32.522581 24851 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.524904 24851 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.526086 25015 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:32.527136 25177 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.526779 25178 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.527784 25177 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.532912 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:32.533651 22750 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:32.514602 25175 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.233s	user 0.004s	sys 0.000s
I20260709 13:07:32.537206 25185 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:07:32.539734 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 131493us
I20260709 13:07:32.597637 22653 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.458930 25063 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:32.609097 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.296188 22652 rpcz_store.cc:276] 0709 13:07:21.865284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.865576 (+   292us) service_pool.cc:224] Handling call
0709 13:07:21.925739 (+ 60163us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b54000ab4d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d65090 00007fcb74e03f3a 00007fcb7abba1cc 00007fcb7acb837e 00007fcb791792a4 00007fcb79179232 00007fcb79179161 00007fcb791790dc 00007fcb79178392 00007fcb7313648c 00007fcb731358c6
0709 13:07:32.295455 (+10369716us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:30.326893 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:30.336921 (+ 10028us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:30.336957 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:07:30.336971 (+    14us) tablet.cc:662] Decoding operations
0709 13:07:30.337952 (+   981us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:30.338013 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:30.338044 (+    31us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:07:30.338793 (+   749us) tablet.cc:701] Row locks acquired
0709 13:07:30.338812 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:07:30.338945 (+   133us) write_op.cc:270] Start()
0709 13:07:30.339046 (+   101us) write_op.cc:276] Timestamp: P: 1783602450338900 usec, L: 0
0709 13:07:30.339066 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:07:30.339671 (+   605us) log.cc:844] Serialized 3605 byte log entry
0709 13:07:30.741880 (+402209us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b18000d5700. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:07:30.860775 (+118895us) op_driver.cc:464] REPLICATION: finished
0709 13:07:31.012597 (+151822us) write_op.cc:301] APPLY: starting
0709 13:07:31.012673 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:31.012854 (+   181us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:31.012873 (+    19us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:31.014029 (+  1156us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:31.014265 (+   236us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:07:31.014327 (+    62us) write_op.cc:312] APPLY: finished
0709 13:07:31.014585 (+   258us) log.cc:844] Serialized 71 byte log entry
0709 13:07:31.014739 (+   154us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:31.014859 (+   120us) write_op.cc:454] Released schema lock
0709 13:07:31.014906 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:31.684202 (+669296us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:31.684273 (+    71us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":76,"reported_tablets":18,"spinlock_wait_cycles":64273792,"child_traces":[["op",{"apply.queue_time_us":151614,"apply.run_cpu_time_us":2625,"apply.run_wall_time_us":2629,"num_ops":9,"prepare.queue_time_us":247358,"prepare.run_cpu_time_us":44860,"prepare.run_wall_time_us":248865,"raft.queue_time_us":116866,"raft.run_cpu_time_us":38640,"raft.run_wall_time_us":824387,"replication_time_us":521579,"spinlock_wait_cycles":27544960,"thread_start_us":1153433,"threads_started":4,"wal-append.queue_time_us":209756,"wal-append.run_cpu_time_us":13879,"wal-append.run_wall_time_us":1547485}]]}
W20260709 13:07:32.609740 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 117232us
W20260709 13:07:32.609764 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 221909us
I20260709 13:07:32.605690 23453 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:32.615916 22653 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.501045 25009 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.630323 22653 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.626487 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:32.526816 25015 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:32.624836 25106 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:32.637480 25186 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.638238 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.638939 22822 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:32.640120 25179 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:32.640637 22653 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.623284 25114 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.644656 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:32.623900 25088 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.653136 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.657279 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:32.657986 22897 raft_consensus.cc:2368] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:32.653335 25052 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:32.654023 25116 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.575269 24947 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.654659 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.655181 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.655381 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:32.655977 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.656440 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.656580 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.656705 25184 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:32.653092 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:32.678066 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:32.681838 22653 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.658278 25015 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:32.684531 22905 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:32.641608 24996 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:07:32.649780 24731 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
W20260709 13:07:32.692839 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 244006us
W20260709 13:07:32.737287 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 113407us
I20260709 13:07:32.767120 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:32.773952 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:32.715368 22896 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:32.811331 22824 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.736876 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:32.773871 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.716094 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.813799 22903 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:32.814533 23449 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:32.737905 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.692277 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.715029 22899 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:32.819701 23453 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:32.750160 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:32.820652 22830 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:32.810544 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:32.666172 25063 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.823151 22829 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:07:32.678715 25127 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:32.823922 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:32.755674 25131 consensus_meta.cc:287] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.713s	user 0.011s	sys 0.000s
I20260709 13:07:32.827661 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.797840 24881 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.716958 25179 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.775735 25186 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:32.841419 22904 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:32.828796 23450 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:32.840974 22653 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.836441 22823 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:32.774129 25087 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.765039 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.816713 25177 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.815908 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.877166 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.864442 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.880776 24997 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:32.881464 25130 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:32.881909 24947 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.882439 24947 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:32.883337 22719 leader_election.cc:400] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4.
I20260709 13:07:32.883656 22719 leader_election.cc:403] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:07:32.869299 24999 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.886056 25192 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.885890 24999 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.886796 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.886569 25192 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.887447 23452 raft_consensus.cc:2368] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
I20260709 13:07:32.887993 24999 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:32.888252 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.888995 23452 raft_consensus.cc:2410] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:32.680528 25088 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.890818 25088 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:32.870759 22653 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 3 to 4, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.891691 25185 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:32.892804 25185 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:32.893173 25185 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:32.893571 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:32.894220 23450 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8.
I20260709 13:07:32.893996 25185 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.892351 22653 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 3 to 4, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.895454 25087 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.895970 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.857143 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:32.896525 23444 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:32.897501 25087 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:32.897701 22749 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:32.871974 24955 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.901880 24955 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.900542 22900 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:32.903915 24955 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:32.905576 23446 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:07:32.764525 25009 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:32.906615 25009 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.902285 22653 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.908597 25009 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.679332 25172 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.912542 23449 raft_consensus.cc:2410] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:32.884862 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:32.878669 22826 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:32.911768 22653 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 2, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 2 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.853776 22830 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:32.824343 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.840898 25178 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.692705 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:32.869395 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.876430 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:32.927699 22747 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:32.890847 24947 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:32.927443 22828 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:32.840682 25162 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:32.896482 22750 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:32.881170 24997 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.863322 25096 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:32.895171 25185 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:32.840946 25128 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.753491 25111 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:32.665723 25114 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.751154 25183 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:32.912147 23448 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:32.920143 22824 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
W20260709 13:07:32.940341 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:32.979885 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:32.980250 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.941341 22752 raft_consensus.cc:2410] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:07:32.943526 22828 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:32.960172 25029 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.927451 25015 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:32.960618 22653 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:32.853587 25131 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.967293 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.969151 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:32.978698 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:32.866818 24881 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:32.936218 25130 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:32.932556 25125 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:32.932372 22827 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:32.986233 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.053349 22750 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:33.050381 22826 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:32.983454 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:33.047434 22747 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:33.035590 22749 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:32.984031 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.939962 24997 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:32.995025 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.983687 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:32.918154 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 181326us
I20260709 13:07:33.059975 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.061870 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.062556 22749 raft_consensus.cc:2368] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
I20260709 13:07:33.063735 22653 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.984493 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:33.066655 22794 leader_election.cc:400] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
W20260709 13:07:33.066424 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 97377.8us
I20260709 13:07:33.068095 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:33.067003 22794 leader_election.cc:403] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:32.991494 25191 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:33.099189 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:32.978524 25106 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.115160 22903 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:33.116204 25127 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:33.115703 22755 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8.
I20260709 13:07:33.118042 22897 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:32.943491 25178 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:32.959594 25096 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:33.119032 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:33.056138 25015 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:33.066267 25111 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:33.119998 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.065536 25029 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.120385 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.120832 24616 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.121821 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.122430 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:33.123231 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:33.209223 25186 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:33.189524 25084 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.209769 22826 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:33.124150 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.210318 25084 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.210863 23452 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:33.212332 25084 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.124747 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:33.124866 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:33.213297 25210 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:33.126099 23450 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:33.213532 23446 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.213912 25210 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:33.126705 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:33.214577 22828 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:33.189810 25015 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:33.214927 25210 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.127406 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:33.221731 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:33.222445 25206 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:33.222638 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:33.130623 25209 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.224516 25209 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:33.189662 25111 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:33.225765 25095 raft_consensus.cc:1064] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:07:33.209129 24616 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.227006 25209 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:33.227952 24616 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.228212 25208 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:33.228596 24616 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:33.228951 25208 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:33.229261 23448 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:33.230365 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.230151 25208 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.231082 25178 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.225793 25111 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:33.232582 24947 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4.
I20260709 13:07:33.138429 23447 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:07:33.066156 25173 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.791s	user 0.012s	sys 0.000s
I20260709 13:07:33.055724 25009 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.233799 25173 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.234217 25009 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.235775 25173 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.236457 22822 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:33.236660 25009 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.120160 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:33.144219 25202 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.258s	user 0.002s	sys 0.000s
I20260709 13:07:32.959478 25162 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.239380 23449 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:33.240619 25162 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:32.977546 25128 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.242928 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.243495 22755 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:33.050938 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:33.244275 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.180625 25106 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:33.245998 23449 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:33.247125 25106 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:33.248031 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.141228 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.176565 22747 raft_consensus.cc:2410] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:07:33.186203 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 70886.9us
I20260709 13:07:33.120196 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:32.918629 25179 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:33.221822 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 94510.9us
I20260709 13:07:33.220975 23446 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:33.189620 24851 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:07:33.228016 25095 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.143224 25192 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.249676 22904 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
W20260709 13:07:33.179576 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 116489us
I20260709 13:07:33.281877 22794 leader_election.cc:400] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8.
I20260709 13:07:33.282269 22794 leader_election.cc:403] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:07:33.072976 22900 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:07:33.284807 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.248369 22752 raft_consensus.cc:2955] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:33.144857 25131 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.258718 22797 leader_election.cc:400] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
W20260709 13:07:33.259785 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 208931us
I20260709 13:07:33.049252 25183 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.277277 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:33.288897 22901 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:33.283063 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:33.291407 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.257251 22756 raft_consensus.cc:2410] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:07:33.285504 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 217452us
I20260709 13:07:33.249034 24616 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:33.258901 25186 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:33.293387 22753 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:33.280978 25098 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:33.261711 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:33.295022 25192 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:33.295560 22755 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:33.285106 25106 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:33.292863 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:33.297102 22797 leader_election.cc:400] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8.
I20260709 13:07:33.209340 25096 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.194649 25127 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:33.310109 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.294942 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:33.308193 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.294685 25087 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.294835 25162 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 13:07:33.340245 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.317641 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.343571 22747 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:33.316956 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:33.367823 25212 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:33.373358 25087 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.373821 25015 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.374292 24881 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.360965 24616 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.374761 25114 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.310326 25192 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.310875 25098 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:33.340660 22903 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:33.379652 25127 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:33.380676 24851 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4.
I20260709 13:07:33.382164 25179 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.382946 25106 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8.
I20260709 13:07:33.429706 24947 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:07:33.383746 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60800.4us
I20260709 13:07:33.383610 25210 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:33.384918 24997 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:33.385708 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 124054us
I20260709 13:07:33.393503 25183 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.445480 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:33.447640 25210 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:33.448053 25210 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.448391 25210 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.454831 25015 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:33.339849 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.365805 25186 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:33.361654 25189 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.308775 25052 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:07:33.419610 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 102690us
W20260709 13:07:33.419955 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:33.420115 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 111989us
I20260709 13:07:33.421227 22747 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:33.420353 22902 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:33.311843 25125 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:33.318060 25009 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:33.422636 25173 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:33.422474 25211 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:07:33.432595 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 138212us
I20260709 13:07:33.431325 25192 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:33.431527 25098 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:33.436460 25087 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:33.424134 24996 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:07:33.446586 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.340188 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.446974 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:33.471895 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:33.471974 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.430371 25111 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.469897 25052 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.475483 25116 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:33.469904 25189 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.482954 22747 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:33.493870 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.481912 22748 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:33.482180 22905 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:33.487637 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:33.479665 25173 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.491210 22899 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:33.495594 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:33.550462 22904 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:33.495805 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.497525 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.480638 22719 leader_election.cc:400] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 3. Current term is 4.
I20260709 13:07:33.529489 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:33.480561 25211 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:33.527845 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.553406 23450 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:33.495227 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.537333 22794 leader_election.cc:400] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:33.555861 22794 leader_election.cc:403] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:07:33.556552 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 76226.4us
I20260709 13:07:33.557533 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.557729 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:33.543192 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.543798 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:33.493772 25131 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:33.544512 25111 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.495779 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.480135 25125 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:33.498308 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:33.551467 22900 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:33.573606 23446 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:33.551800 23449 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:33.578377 23447 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:07:33.578785 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.489470 25052 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.555533 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.578985 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:33.579790 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.552297 23444 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:33.489270 25183 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:33.554394 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:33.580921 25219 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:07:33.496111 24997 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.585569 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.586205 22904 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:33.586987 23453 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:33.588181 25189 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.574528 25096 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:33.497394 24881 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:07:33.558733 25087 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:33.589603 25087 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:33.590349 25087 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.528616 25173 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.496986 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:33.606866 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 109945us
I20260709 13:07:33.607483 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:33.560700 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.574137 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.576980 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.497661 25205 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:33.578331 22748 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:33.608456 22747 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:07:33.586284 25220 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:07:33.610265 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.572906 25131 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:33.601485 25218 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:07:33.493489 25179 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:07:33.605269 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:33.605557 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.554930 22751 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:33.629455 22756 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:33.598147 25127 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:33.609993 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.633298 22902 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:33.551309 25015 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:33.570629 22903 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:33.555899 25211 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.624645 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:33.626586 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 65995.8us
I20260709 13:07:33.572124 25162 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:33.630208 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.631024 22904 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:33.635825 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:33.631042 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.635064 23450 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:33.632841 25140 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:07:33.636883 25127 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:33.634663 24811 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:07:33.638616 22748 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:33.638826 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.644601 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:33.648272 22751 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:33.614910 24881 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:33.632977 25181 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:33.645145 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:33.656251 22828 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:33.628852 25131 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:33.644802 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:33.660382 22826 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:33.654069 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:33.661777 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:33.633100 25130 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.663954 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:33.606041 24996 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:07:33.665182 22827 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:33.639684 25015 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:33.669123 25015 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.666560 25127 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:33.671007 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 117025us
I20260709 13:07:33.633996 25111 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:33.674695 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:33.659373 23453 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:33.682580 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:33.648736 24947 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:33.676961 22825 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:33.677881 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.656366 25179 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:33.676492 25162 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:33.675755 24996 consensus_queue.cc:1243] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:33.717808 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:33.719444 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:33.700026 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 61241.7us
I20260709 13:07:33.671279 25130 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.686997 24947 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.702310 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:33.682088 25084 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:33.679232 25009 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:33.648911 25123 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:07:33.740274 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:33.740959 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.712865 25190 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:33.713227 25179 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.686331 25181 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:33.725188 22756 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:33.704540 24811 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.683810 25210 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:33.739315 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 91277.2us
I20260709 13:07:33.723510 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.737031 25084 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.741757 22748 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:33.742580 24881 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:33.751931 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:33.716250 25162 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:33.742761 25176 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:33.704730 24851 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:33.742844 25189 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:33.771643 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:33.772482 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49037us
W20260709 13:07:33.783576 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54568.4us
I20260709 13:07:33.762683 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 507) took 1108 ms. Trace:
I20260709 13:07:33.736797 24947 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.776285 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:33.752243 25084 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:33.748291 24996 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:33.753677 25223 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:07:33.762910 22748 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:07:33.758301 25190 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:33.760764 25210 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.751173 24811 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.777179 25162 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:33.775839 25186 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:33.747831 25179 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.784088 22901 rpcz_store.cc:276] 0709 13:07:32.653904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.677750 (+ 23846us) service_pool.cc:224] Handling call
0709 13:07:33.762469 (+1084719us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54000d0554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:33.762595 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082865408}
I20260709 13:07:33.891573 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:56774 (request call id 9) took 20183 ms. Trace:
I20260709 13:07:33.785657 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:33.773952 25111 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:33.784168 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:07:33.935331 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 151230us
W20260709 13:07:33.792797 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.779099 24851 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:33.849428 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 193278us
I20260709 13:07:33.752132 25200 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.137s
I20260709 13:07:33.762012 25192 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.780542 25128 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:33.930311 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 141739us
I20260709 13:07:33.943259 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:33.783470 25189 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.938874 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.777621 25176 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.952193 25176 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:33.953552 25185 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:33.953961 25185 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.954334 25185 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:33.955575 24811 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.892060 25205 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.935643 24616 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:33.957669 25205 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:33.958561 25186 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:33.936405 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:33.959822 25205 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:33.889108 25208 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.754477 25181 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.960404 25208 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.960968 25106 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:33.941399 22902 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:33.892508 22752 rpcz_store.cc:276] 0709 13:07:13.708376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.864758 (+156382us) service_pool.cc:224] Handling call
0709 13:07:13.947696 (+ 82938us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b54002bcda0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d78520 00007fcb74e01b42 00007fcb74e00800 00007fcb79dc1d29 00007fcb733401d4 00007fcb73340162 00007fcb73340091 00007fcb7334000c 00007fcb7333f2c2 00007fcb7313648c
0709 13:07:13.998949 (+ 51253us) log.cc:844] Serialized 499 byte log entry
0709 13:07:14.368523 (+369574us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b54002bcda0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d7bb26 00007fcb74d7549a 00007fcb74d74714 00007fcb74d54eaa 00007fcb74d5bd53 00007fcb74d5bcd9 00007fcb74d5bc69 00007fcb74d5bc31 00007fcb74d5adfd 000000000038f354
0709 13:07:14.426681 (+ 58158us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b1800109ba0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:07:33.891426 (+19464745us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20479,"raft.queue_time_us":20846,"raft.run_cpu_time_us":5467,"raft.run_wall_time_us":283559,"spinlock_wait_cycles":20780544,"thread_start_us":67349,"threads_started":1,"wal-append.queue_time_us":139672,"wal-append.run_cpu_time_us":2266,"wal-append.run_wall_time_us":1100662}
W20260709 13:07:33.893733 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 108173us
I20260709 13:07:33.958006 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.963209 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.962761 23451 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:33.940368 25096 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:33.966136 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:33.970513 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:33.971612 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:33.972182 22824 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:33.972577 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:33.973297 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:33.973867 22830 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:33.968638 25128 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:07:33.974601 25111 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:33.977538 25111 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:33.977937 25111 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:33.979102 22751 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:33.981271 24881 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.967108 24616 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:33.985852 24616 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.986922 25192 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.987737 25192 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:33.988777 25236 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.989809 25204 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.989436 25236 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:33.990293 25204 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.992121 25128 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:33.992347 25204 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:33.992635 25128 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:33.994765 25128 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:33.996030 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 180351us
I20260709 13:07:33.997426 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.998803 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:34.004305 25087 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:34.005049 25208 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:33.940387 24851 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.013465 25087 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.014215 22826 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:07:34.015027 25227 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.311s	user 0.001s	sys 0.003s
I20260709 13:07:33.935366 25230 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:34.021277 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:34.021895 22901 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:34.016829 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:33.962355 25189 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.023083 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:34.026994 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:34.029896 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:33.971099 25063 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:07:34.037066 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.018545 25095 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:34.042917 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:07:34.045250 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 74809.4us
I20260709 13:07:34.046635 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:34.016568 23446 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:34.052471 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.053738 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:34.055132 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:34.053194 22829 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:34.057212 23452 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:34.057585 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:34.058091 23449 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.059368 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:33.942413 25232 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:07:33.970687 25015 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:07:34.066733 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:34.067322 22752 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:34.068334 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:34.023914 25230 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.033120 25063 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.071990 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.072630 22751 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:34.066143 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:34.032246 25210 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:34.075048 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:34.020813 25111 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.090381 22829 raft_consensus.cc:2368] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:34.051445 25231 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:34.022228 25087 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.117594 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:34.121454 22903 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:34.064284 25096 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:34.070156 25240 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:34.087893 25063 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:34.134425 25096 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.117763 25095 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:34.079536 25230 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:34.069229 25211 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:34.118883 25052 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.004s
I20260709 13:07:34.128468 24616 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:34.115216 25192 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:34.112984 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:34.167364 23449 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:34.168303 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:34.169588 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 46578.3us
I20260709 13:07:34.088799 25236 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.086972 25210 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.176074 23451 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:34.141291 25116 consensus_meta.cc:287] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.645s	user 0.008s	sys 0.000s
I20260709 13:07:34.121054 25231 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.227713 22755 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:34.228057 22750 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:07:34.194376 24947 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:34.196177 25098 consensus_meta.cc:287] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.001s
I20260709 13:07:34.113456 25206 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:07:34.149149 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:34.175544 25176 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.212742 23447 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:34.210215 25192 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:34.250083 23451 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:34.252840 23416 leader_election.cc:400] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:34.254068 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:34.221006 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1400) took 1283 ms. Trace:
W20260709 13:07:34.274971 25228 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.571s	user 0.000s	sys 0.003s
I20260709 13:07:34.221433 25237 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.228390 22750 pending_rounds.cc:85] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 4
I20260709 13:07:34.275799 22750 pending_rounds.cc:107] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:07:34.276058 22750 raft_consensus.cc:2889] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:34.229000 22829 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:34.206009 24616 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.285429 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.286041 22829 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:34.287345 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:34.274575 22822 rpcz_store.cc:276] 0709 13:07:32.937636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.940003 (+  2367us) service_pool.cc:224] Handling call
0709 13:07:33.051417 (+111414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:33.248900 (+197483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.220794 (+971894us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540043b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:34.220952 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414780800}
I20260709 13:07:34.211025 25052 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.293177 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:34.242888 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 177238us
I20260709 13:07:34.210480 25245 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:34.257232 25173 consensus_meta.cc:287] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.624s	user 0.010s	sys 0.000s
I20260709 13:07:34.296823 25173 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.298077 22826 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:34.298408 25114 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:34.259663 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.260759 25234 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:34.299193 22830 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:34.236168 25231 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.300143 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.300495 25226 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:34.224610 25185 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.262149 22901 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:34.223263 24851 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.223174 25210 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:34.265498 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 847c8a2cc0bb4233a1d55ca04643418d: Illegal state: Replica 6a5191aaff76469bbf6c4735dc9e6509 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:34.302419 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:34.242862 25208 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:07:34.266324 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.303115 22901 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:34.303397 22828 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:34.298940 25173 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.301298 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.304278 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.307099 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.307291 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:34.308444 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:34.309013 22830 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:34.309010 25232 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:34.309649 25232 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.309983 25232 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.311601 24851 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.314504 25210 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:34.315053 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.315644 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:34.316322 22755 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:34.316437 25210 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.316748 22830 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:34.317261 25185 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.318423 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:34.314532 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:34.319515 25208 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:34.319588 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:34.320021 25208 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.320335 25226 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:34.269411 25192 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.321617 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:34.322309 25226 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:34.323758 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:34.324422 25226 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:34.059832 25190 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.326263 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:34.272974 25221 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.550s	user 0.002s	sys 0.000s
I20260709 13:07:34.327804 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:34.273903 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 775) took 1158 ms. Trace:
I20260709 13:07:34.277376 22871 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:34.328121 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:34.265740 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:34.328358 23448 rpcz_store.cc:276] 0709 13:07:33.115741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.118535 (+  2794us) service_pool.cc:224] Handling call
0709 13:07:34.273714 (+1155179us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b540050f854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:34.273821 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44805888}
I20260709 13:07:34.328903 25236 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:34.329774 25236 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:34.330842 25236 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.320374 25208 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:34.298923 25114 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.332391 23449 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:07:34.269052 25131 consensus_meta.cc:287] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.593s	user 0.006s	sys 0.002s
I20260709 13:07:34.296001 25245 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.285175 24616 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.343055 22829 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:34.257256 25111 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.347762 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:34.282629 25240 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:34.333555 25131 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.245204 25098 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.359464 25249 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:34.338624 25106 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:34.361235 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:34.362308 22905 raft_consensus.cc:2368] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:34.362821 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:34.371016 25245 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.292704 25237 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.372326 25189 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.372416 25208 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.385066 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:34.339775 25206 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:07:34.373363 25179 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.377034 23453 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:34.377049 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:34.342136 22822 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.382843 23449 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:07:34.389583 23449 pending_rounds.cc:85] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 7
I20260709 13:07:34.389890 23449 pending_rounds.cc:107] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:07:34.390168 23449 raft_consensus.cc:2889] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:34.390170 22896 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:34.379300 25249 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.387778 22829 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:34.361073 25234 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:34.348337 25116 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:07:34.394387 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:34.380219 25185 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.400455 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:34.401973 22903 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:34.404902 22901 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:34.394145 25249 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:34.375823 25240 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.396754 25234 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.415302 23453 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:34.422415 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:34.408464 23447 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:34.405239 24811 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:34.402632 24881 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:34.402082 25087 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.430658 25247 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:07:34.432183 24947 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.407307 25185 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.435215 25185 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.424094 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:34.408804 23446 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:34.408188 25239 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:34.418541 25181 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:34.395104 25231 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.417335 25125 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:34.418916 25234 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:34.398319 25232 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.468433 25176 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.470256 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:34.480612 22752 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.427959 24811 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.410166 25186 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:07:34.476833 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:34.467236 25181 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.489142 22903 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.490044 22899 raft_consensus.cc:2368] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:34.469983 24881 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:34.503576 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:34.494158 25130 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.748s	user 0.005s	sys 0.005s
I20260709 13:07:34.489357 24811 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:34.512676 22752 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:34.497830 25181 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.532516 22903 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:34.517261 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:34.535040 22751 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.535897 25098 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.522001 25240 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.539412 25232 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.530362 25095 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:34.542873 25125 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.543040 25234 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:34.543347 25125 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.510080 25052 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:34.545765 25206 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:34.472739 25239 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:34.548269 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 270988us
I20260709 13:07:34.545710 22831 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:34.546937 25208 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:34.552109 25230 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.554287 25230 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.555361 22823 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:34.555884 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:34.552111 25063 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.515338 25130 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.557595 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:34.557826 22902 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:07:34.541137 22822 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:34.559307 25095 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:34.559713 25095 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:34.560062 25095 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:34.513919 25192 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.561491 25246 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.562029 25192 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.561951 25246 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.562637 22829 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:34.526746 25084 consensus_meta.cc:287] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.001s
I20260709 13:07:34.564792 25084 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.566506 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:34.567003 23452 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:34.567476 25179 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:34.568261 25237 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.500066 25116 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:07:34.569212 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:34.569891 22826 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:34.570580 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:34.571115 22826 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:34.572831 25063 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:34.574539 25084 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.576220 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:34.571816 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:34.558693 25261 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:07:34.558709 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:34.587245 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:34.584168 23452 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:34.522408 25235 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:34.587514 22751 raft_consensus.cc:2410] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:07:34.588973 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:34.580965 22829 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:34.545178 25234 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:34.590073 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:34.592079 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:34.587323 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:34.593904 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:34.546283 25206 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.596370 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:34.599254 23452 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 3: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:34.595844 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:34.565347 25131 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.603302 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:34.603646 23449 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:34.603302 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:34.604724 22751 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:34.607105 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:34.607880 22899 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:34.609781 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:34.589800 25189 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.618554 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:34.611238 22826 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:34.620714 22756 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.593532 25176 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:34.601153 25106 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:34.600939 25245 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:34.602886 25111 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:34.602138 25235 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:34.608274 25029 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.594969 25190 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 13:07:34.631795 25265 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:07:34.646272 22826 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:34.595731 25192 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:07:34.647897 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:34.648859 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.618131 25125 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:34.625401 25176 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:34.650267 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:34.664449 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:34.667909 23449 raft_consensus.cc:2368] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
W20260709 13:07:34.663813 25268 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:07:34.707792 22756 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:34.678988 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:34.718291 25181 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.718657 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:34.720538 25181 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.723704 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:34.724610 25015 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:34.719556 22751 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:34.728965 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:34.639343 25130 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.687582 25231 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:34.674305 22898 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:34.688705 25235 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.671876 25176 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:34.690979 25239 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:34.674402 25173 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:07:34.692780 25249 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.603145 24881 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:34.694417 25162 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.759s	user 0.008s	sys 0.001s
I20260709 13:07:34.739986 25015 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:34.645135 25206 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.710384 24616 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.712317 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 1751) took 2296 ms. Trace:
I20260709 13:07:34.714422 25127 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:34.673456 23453 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:34.716216 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:34.766005 22747 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:34.749230 25111 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:34.716053 24811 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:34.766705 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 161629us
I20260709 13:07:34.767572 25015 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:34.761741 22754 rpcz_store.cc:276] 0709 13:07:32.415633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.417873 (+  2240us) service_pool.cc:224] Handling call
0709 13:07:32.418210 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:32.774514 (+356304us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:32.931644 (+157130us) log.cc:844] Serialized 14 byte log entry
0709 13:07:33.154179 (+222535us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:33.154224 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:33.154270 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.497549 (+343279us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b180033cc40. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419
0709 13:07:34.712091 (+1214542us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400045454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:34.712136 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.712242 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7588,"prepare.queue_time_us":221893,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":232,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":6219,"spinlock_wait_cycles":808389120,"thread_start_us":324760,"threads_started":2,"wal-append.queue_time_us":382023,"wal-append.run_cpu_time_us":1298,"wal-append.run_wall_time_us":1118790}
I20260709 13:07:34.775009 22903 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:34.775185 25114 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.775566 25015 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.778424 25249 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.779114 24811 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:34.781648 25270 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:34.754619 25130 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.789157 22747 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:34.791105 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:34.754532 25190 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:07:34.765743 25127 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:34.794443 25106 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:07:34.793193 25274 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.136s	user 0.002s	sys 0.000s
I20260709 13:07:34.761301 25162 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.782307 25270 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.803473 24811 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.785696 25245 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:07:34.833913 25267 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.365s	user 0.004s	sys 0.000s
I20260709 13:07:34.843004 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:34.850832 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 516) took 1366 ms. Trace:
W20260709 13:07:34.852748 25258 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.358s	user 0.000s	sys 0.001s
I20260709 13:07:34.854924 25226 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.857265 25029 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.857789 25116 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.858263 25230 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:34.862927 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:35.075753 22748 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:34.868181 25211 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:34.880646 25260 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.406s	user 0.001s	sys 0.000s
I20260709 13:07:34.886683 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 1760) took 1903 ms. Trace:
I20260709 13:07:34.887843 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:34.899574 25123 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:07:34.900892 22905 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 1719) took 1410 ms (client timeout 1615 ms). Trace:
I20260709 13:07:34.909127 25246 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:34.909185 23445 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:07:34.764912 25095 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.001876 22897 rpcz_store.cc:276] 0709 13:07:33.484322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.494724 (+ 10402us) service_pool.cc:224] Handling call
0709 13:07:33.553794 (+ 59070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:33.946607 (+392813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:33.946651 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:33.946704 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:33.946747 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.850491 (+903744us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54001b7454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:34.850551 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.850715 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902864128}
I20260709 13:07:34.793982 25130 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.089721 22750 rpcz_store.cc:276] 0709 13:07:32.982845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.231273 (+248428us) service_pool.cc:224] Handling call
0709 13:07:33.231560 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:34.287142 (+1055582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.886388 (+599246us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b5400046354. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:34.886591 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2290972032}
W20260709 13:07:35.092665 22905 rpcz_store.cc:269] 0709 13:07:33.490884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.554759 (+ 63875us) service_pool.cc:224] Handling call
0709 13:07:34.900667 (+1345908us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b540034b7d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:34.900826 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715161856}
I20260709 13:07:35.097870 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:34.809276 25176 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:34.955499 24811 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:34.809906 25127 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.074510 25226 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.482199 22752 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:35.406757 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2024) took 1049 ms. Trace:
I20260709 13:07:35.405813 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:35.445955 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:35.484591 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:35.333722 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.045s	sys 0.083s
I20260709 13:07:35.118834 22829 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:07:35.490201 22756 raft_consensus.cc:2410] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:07:35.243011 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2018) took 1226 ms. Trace:
I20260709 13:07:35.174001 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:35.371670 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 568) took 1059 ms. Trace:
I20260709 13:07:35.446559 25270 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:35.494010 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:35.494304 22831 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:07:35.495652 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 652647us
I20260709 13:07:35.118598 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:35.494694 22825 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:35.121482 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:35.490042 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.064s	sys 0.098s
W20260709 13:07:35.497392 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 706300us
I20260709 13:07:35.497850 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:35.476979 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:35.498430 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:35.499640 22750 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:35.110953 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:35.500458 22794 leader_election.cc:400] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:35.493726 22753 rpcz_store.cc:276] 0709 13:07:34.312692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.312821 (+   129us) service_pool.cc:224] Handling call
0709 13:07:34.313106 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:34.313431 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.313459 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:34.313873 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 13:07:34.385874 (+ 72001us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:34.385970 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.386143 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.851973 (+465830us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:35.344499 (+492526us) raft_consensus.cc:1685] finished
0709 13:07:35.344527 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.371610 (+ 27083us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2482,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":67,"spinlock_wait_cycles":5166976,"thread_start_us":71691,"threads_started":1,"wal-append.queue_time_us":102372}
I20260709 13:07:35.500684 22828 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:35.502472 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:35.503178 23450 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:35.192203 25125 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.500835 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:35.520586 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:35.191968 25181 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:35.092563 25123 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:35.430790 25240 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:35.192704 25249 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:35.082842 25230 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:35.246881 25130 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:35.521903 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:35.539458 22830 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:07:35.090193 25029 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.092150 25116 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.405128 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:35.103921 25009 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.174113 25095 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:35.524089 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 405510us
I20260709 13:07:35.567786 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:35.528730 23449 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:35.523167 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:35.570148 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:35.570613 25063 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:35.571336 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:35.571405 25063 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:35.572511 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:35.573407 23449 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:35.574079 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 793) took 1212 ms. Trace:
I20260709 13:07:35.570688 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:35.356674 25106 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.530504 25247 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:35.535537 25270 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:35.396486 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:35.251643 24616 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:35.205847 25096 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:35.492781 22824 rpcz_store.cc:276] 0709 13:07:34.016839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.016952 (+   113us) service_pool.cc:224] Handling call
0709 13:07:34.023318 (+  6366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:34.229855 (+206537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.229884 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:34.240765 (+ 10881us) log.cc:844] Serialized 26 byte log entry
0709 13:07:34.394358 (+153593us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:34.496430 (+102072us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.496811 (+   381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.496854 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:34.828957 (+332103us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b180036ee00. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb79d24aa1 00007fcb79d1bb37 00007fcb79d1b9ca 00007fcb79e1326f
0709 13:07:34.830714 (+  1757us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b4800130158. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb79d24aa1 00007fcb79d1bb37 00007fcb79d1b9ca 00007fcb79e1326f 00007fcb79e12f37 00007fcb79e2755c 00007fcb79e27508 00007fcb79e27488 00007fcb79e27443 00007fcb79e266ec 00007fcb78d50a96
0709 13:07:34.879869 (+ 49155us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:35.242727 (+362858us) raft_consensus.cc:1685] finished
0709 13:07:35.242756 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.242937 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":245986,"prepare.queue_time_us":117913,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":38171,"raft.run_cpu_time_us":4193,"raft.run_wall_time_us":337556,"spinlock_wait_cycles":13714688,"thread_start_us":346826,"threads_started":3,"wal-append.queue_time_us":121263}
I20260709 13:07:35.541625 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:35.615864 22829 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:35.501564 25015 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:35.501868 25206 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:35.173198 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 789) took 1125 ms. Trace:
I20260709 13:07:35.520364 25231 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:35.205437 25278 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.496s
I20260709 13:07:35.489410 22826 rpcz_store.cc:276] 0709 13:07:34.356926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.723960 (+367034us) service_pool.cc:224] Handling call
0709 13:07:34.724221 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:34.726492 (+  2271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.726521 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:34.727355 (+   834us) log.cc:844] Serialized 26 byte log entry
0709 13:07:34.816922 (+ 89567us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:07:34.816961 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.817017 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:35.406571 (+589554us) raft_consensus.cc:1685] finished
0709 13:07:35.406591 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.406708 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7,"raft.queue_time_us":673,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"thread_start_us":89239,"threads_started":1,"wal-append.queue_time_us":677429}
I20260709 13:07:35.521155 23453 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:35.542268 24997 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.413244 25245 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.095527 25111 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.538995 25249 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.605621 25123 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.566036 25286 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:35.597729 24996 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:35.611490 22827 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:35.556751 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:35.504347 25125 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:35.652894 25249 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:35.500793 22794 leader_election.cc:403] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:35.708611 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 208244us
I20260709 13:07:35.572636 25063 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:35.538702 25181 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:35.553918 25240 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:07:35.619418 25235 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:35.653748 23449 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:07:35.588449 25009 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.650532 23453 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:35.630604 25015 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:35.614073 25247 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.633965 25206 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:35.627426 22830 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:35.710778 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:35.712841 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:35.713032 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:35.716377 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:35.759874 22826 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:35.717742 23449 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:35.760015 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:35.725436 22756 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:35.726047 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:35.647853 25231 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.762472 22752 raft_consensus.cc:2368] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:35.763600 22828 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
W20260709 13:07:35.763861 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50924us
I20260709 13:07:35.765071 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:35.772611 22754 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:35.772411 22794 leader_election.cc:400] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:35.782716 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:35.714181 25193 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:35.782660 22823 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:35.702183 25123 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:35.791230 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 223501us
I20260709 13:07:35.714315 25176 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:35.694913 25287 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:35.650906 24997 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.795011 22755 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:35.755949 25210 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:35.694746 25286 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:35.805291 25283 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:07:35.810634 25240 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:35.811844 25116 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:35.793876 25273 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:07:35.794080 25279 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:35.772332 25095 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:35.769586 24881 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:35.812750 25029 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:35.761474 25285 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:35.783483 24811 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:35.767280 25226 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:35.813192 25291 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:35.814034 25288 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:07:35.628298 23445 rpcz_store.cc:276] 0709 13:07:34.047271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.065508 (+ 18237us) service_pool.cc:224] Handling call
0709 13:07:34.908625 (+843117us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:34.908854 (+   229us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:35.104140 (+195286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.173068 (+ 68928us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7616256}
I20260709 13:07:35.803561 25247 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:35.728878 25241 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:07:35.831561 23450 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:07:35.832683 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 334863us
I20260709 13:07:35.756748 25235 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.800899 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:35.842777 25029 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:35.838750 25240 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.841148 25116 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:35.829612 25210 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.950660 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 581) took 1005 ms. Trace:
W20260709 13:07:35.967875 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 57405.2us
I20260709 13:07:35.952414 22825 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
W20260709 13:07:35.961978 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.904798 25270 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:35.853510 24881 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.907270 24811 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.972090 22751 rpcz_store.cc:276] 0709 13:07:34.945170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.945306 (+   136us) service_pool.cc:224] Handling call
0709 13:07:35.419676 (+474370us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b5800003398. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:35.419879 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:35.420103 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:35.652808 (+232705us) log.cc:844] Serialized 14 byte log entry
0709 13:07:35.950214 (+297406us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:35.950286 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:35.950347 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.950436 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.950588 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36795,"prepare.queue_time_us":389252,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":54468,"spinlock_wait_cycles":28069120,"thread_start_us":463312,"threads_started":2}
I20260709 13:07:35.973166 23419 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:35.985735 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:35.967643 24996 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.971992 24616 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:35.982815 25270 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:35.969587 25240 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.991516 24881 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:35.997690 25295 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.969710 25125 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:35.968434 25294 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:36.010783 25210 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:35.972019 25239 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.002274 25294 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.001278 25295 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.027393 25294 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.028275 25295 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:35.825158 25287 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.029299 25210 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:07:36.031369 24881 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:36.033285 24881 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.037334 25287 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.038952 24881 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:35.969535 25029 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.911679 25206 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.867518 25291 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:35.975709 25116 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:35.939324 25015 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.015286 25297 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.017374 25087 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:35.987461 24811 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:36.086338 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 293574us
I20260709 13:07:36.050717 25009 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:36.087430 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 303532us
I20260709 13:07:36.052578 25181 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:36.092193 22794 leader_election.cc:400] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:36.092496 22794 leader_election.cc:403] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:07:36.014276 24996 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.016921 24616 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.099633 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.059405 25231 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.084193 25087 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:36.040156 25239 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.119396 25239 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:36.090619 25290 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:07:36.123196 24616 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:36.124027 25230 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.085614 25297 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.126688 25297 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.125077 22826 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:36.130115 25123 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:36.130582 25123 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.131039 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.132437 25123 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.099889 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:36.103436 25009 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:36.119535 25193 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:36.143136 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.105409 25241 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:36.134479 25087 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:36.149142 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.149969 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.152490 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.155437 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.137737 25230 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:36.156040 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.187160 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.187665 22748 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:36.156607 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.102952 25287 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:07:36.105629 25286 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.147648 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.196939 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:36.160693 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:36.133466 25193 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:36.202582 25270 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:36.186607 25300 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.203176 25270 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.203787 25300 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.155620 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.205235 25270 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.206141 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.156647 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.191073 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.206705 22756 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:36.207093 22824 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:36.195243 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:36.208066 22899 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:36.190361 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.209837 22752 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:36.191748 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:36.210683 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50057.3us
I20260709 13:07:36.168668 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:36.203344 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.211580 22902 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:36.197158 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:36.178561 25306 raft_consensus.cc:2764] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:07:36.178733 25123 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.219894 23451 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:36.188623 25176 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:36.242645 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 244322us
I20260709 13:07:36.247589 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:36.253520 25176 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.235481 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:36.234794 25296 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:07:36.225750 25293 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:36.264034 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:36.269829 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 90837.5us
W20260709 13:07:36.263718 25305 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:07:36.210445 25230 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:36.263051 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.220067 25273 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.277804 22797 leader_election.cc:400] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:36.244071 25235 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.296336 25273 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.288969 23451 raft_consensus.cc:2410] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:36.297789 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.298157 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.298460 23451 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:36.298523 25273 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:35.574527 23448 rpcz_store.cc:276] 0709 13:07:34.362119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.374257 (+ 12138us) service_pool.cc:224] Handling call
0709 13:07:34.890585 (+516328us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6eee 0000000000375939 00007fcb79d4a1e0 00007fcb79e130c4 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:34.890768 (+   183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:34.891019 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:35.172999 (+281980us) log.cc:844] Serialized 14 byte log entry
0709 13:07:35.573715 (+400716us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:35.573766 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:35.573809 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.573873 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:35.574006 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55593,"prepare.queue_time_us":213446,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":830,"raft.queue_time_us":96747,"spinlock_wait_cycles":8293120,"thread_start_us":327920,"threads_started":1,"wal-append.queue_time_us":361667}
I20260709 13:07:36.299476 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.300071 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.300490 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.292840 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.301700 22752 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:36.293473 25176 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.245056 25123 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.226964 25303 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:36.295357 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.288278 25311 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:07:36.296780 22797 leader_election.cc:403] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:07:36.303930 22747 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:36.263688 24811 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.295498 25300 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.277772 25293 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.336345 25293 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:36.246196 25294 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:07:36.242712 25309 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
W20260709 13:07:36.271672 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 74818.6us
I20260709 13:07:36.309150 22899 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:36.315764 23451 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:07:36.263060 25240 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.369855 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:36.377485 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:36.401733 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2027) took 1982 ms. Trace:
I20260709 13:07:36.402513 25123 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.403036 25123 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:36.402259 22831 rpcz_store.cc:276] 0709 13:07:34.419180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.783341 (+364161us) service_pool.cc:224] Handling call
0709 13:07:34.795323 (+ 11982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:35.494894 (+699571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:36.401471 (+906577us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b5400081e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:36.401664 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1607094016}
I20260709 13:07:36.410144 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:36.411361 22900 raft_consensus.cc:2368] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:36.412542 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:36.414480 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 202840us
I20260709 13:07:36.418606 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 454) took 1545 ms. Trace:
I20260709 13:07:36.421801 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.422206 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:36.422567 22751 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:36.423506 25300 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:36.423934 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:36.424085 25300 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:36.419014 23452 rpcz_store.cc:276] 0709 13:07:34.872856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.886340 (+ 13484us) service_pool.cc:224] Handling call
0709 13:07:34.886715 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:35.520704 (+633989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:35.520740 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:35.523048 (+  2308us) log.cc:844] Serialized 26 byte log entry
0709 13:07:35.705188 (+182140us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:35.705240 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.705345 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:36.418405 (+713060us) raft_consensus.cc:1685] finished
0709 13:07:36.418428 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:36.418549 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":617117,"raft.queue_time_us":655806,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":21755,"thread_start_us":139352,"threads_started":1,"wal-append.queue_time_us":353588}
I20260709 13:07:36.418627 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.413252 22900 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:36.428647 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:36.375927 25116 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.424937 25016 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:36.431159 25016 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.432502 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:36.433504 25270 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:36.434221 25016 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.434944 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:36.436085 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 158449us
I20260709 13:07:36.438345 25326 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6.
I20260709 13:07:36.439128 25235 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:36.439635 25235 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:36.440685 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:36.441473 22829 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:36.441632 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.442312 25285 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:36.442817 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.442860 25285 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.445372 24811 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:36.445627 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:36.446074 25249 consensus_meta.cc:287] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.717s	user 0.009s	sys 0.000s
I20260709 13:07:36.446517 25249 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.447009 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:36.448371 25249 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:36.417856 25307 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:36.430506 25125 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 13:07:36.451406 25284 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.823s	user 0.003s	sys 0.000s
I20260709 13:07:36.453651 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 788) took 1896 ms. Trace:
I20260709 13:07:36.403383 25123 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:36.453997 23446 rpcz_store.cc:276] 0709 13:07:34.557427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.591628 (+ 34201us) service_pool.cc:224] Handling call
0709 13:07:34.626119 (+ 34491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:34.855209 (+229090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.855241 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:34.855792 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:07:35.352461 (+496669us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:35.352522 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.566002 (+213480us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54007d0fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:35.584376 (+ 18374us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:36.450855 (+866479us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:36.453458 (+  2603us) raft_consensus.cc:1685] finished
0709 13:07:36.453486 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:36.453601 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":531418624,"thread_start_us":439237,"threads_started":1,"wal-append.queue_time_us":697441}
I20260709 13:07:36.454815 25123 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.455904 25123 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:36.442345 22831 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:36.457078 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.440424 25235 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:36.459056 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.459002 23453 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:36.460227 22901 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:36.461424 22826 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:36.465813 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:36.467048 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.424670 25300 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:36.429934 25210 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:36.470219 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.474166 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.475086 22826 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:36.437086 24996 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:36.478216 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.481647 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.472532 24616 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:36.484055 25063 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:36.445245 25285 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.462689 25330 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:36.492712 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:36.462853 25315 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:36.468286 25249 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.462834 25303 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:07:36.449146 25307 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.538309 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:36.523267 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 108026us
I20260709 13:07:36.535701 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.495951 24616 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.536304 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.536314 22901 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:36.539984 24616 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:36.484663 25231 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:36.537688 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.541473 23452 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:36.537995 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:36.542088 22903 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:36.476873 25162 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.541066 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:36.474867 25125 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:36.491562 25247 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:36.539161 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.548378 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.492952 22900 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:36.553534 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.553705 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.573256 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.479945 23447 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:36.470052 25230 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.573218 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:36.542238 25015 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:36.585493 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.585673 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.567278 25285 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:36.565243 25298 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:07:36.587387 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:36.551003 25226 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.535015 25270 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:07:36.574628 25338 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.578678 25116 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:36.595686 25015 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.584393 25326 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.616503 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.551729 25249 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.597911 25345 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:36.495024 25325 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 13:07:36.550726 24811 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:07:36.598927 25285 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.595325 22822 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:36.617003 25327 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:07:36.606060 25341 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:07:36.609684 25338 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.582994 25297 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:36.597704 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:36.618462 25326 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.619336 25111 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:36.620098 25193 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.631273 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:36.521972 25336 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 13:07:36.633673 22901 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:36.551805 25315 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.604575 22903 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:36.615422 25015 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.542676 25330 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.537356 22826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:36.674034 22826 raft_consensus.cc:606] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:36.674480 22826 raft_consensus.cc:614] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:36.657735 25285 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.644510 25087 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.646853 25270 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.621348 25226 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:36.721463 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 90281.9us
I20260709 13:07:36.720968 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:36.724517 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:36.724730 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:36.725976 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 128336us
I20260709 13:07:36.720908 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:36.745601 25307 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.748983 25353 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:36.752856 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.754303 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:36.760102 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:36.720165 25291 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.763134 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:36.750324 25285 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:36.751679 25232 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:36.714829 25270 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.769672 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:36.770162 25015 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:36.770666 25015 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:36.771499 25015 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:36.773005 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet efec6b734c374a169f3dc3b729ef2f07: Illegal state: Replica 7aad11e70ca94fce8f672db05f1c28c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:36.783264 22756 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:36.784943 23419 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:36.764202 25325 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:36.763577 25345 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:36.766778 25291 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:36.765412 25285 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:07:36.783301 22794 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.819s (SENT) [suppressed 5 similar messages]
I20260709 13:07:36.829037 22903 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:36.828922 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:36.839375 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:36.840946 25335 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
W20260709 13:07:36.844038 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.762377 25293 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:36.849164 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 260507us
I20260709 13:07:36.794845 25239 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:36.854588 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.828771 25285 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:36.860914 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.862715 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.865031 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:36.865258 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:36.863593 25291 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:36.870649 25291 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.853084 25193 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.874430 25330 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.875247 25293 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.877080 22754 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:36.879161 25206 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:36.884208 25307 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:36.884778 25307 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:36.885255 25307 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:36.887152 22900 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:36.887578 25226 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.889796 25029 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.893144 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 788) took 2846 ms. Trace:
I20260709 13:07:36.893344 25239 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:36.893572 23444 rpcz_store.cc:276] 0709 13:07:34.046907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.047886 (+   979us) service_pool.cc:224] Handling call
0709 13:07:35.045879 (+997993us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:35.046117 (+   238us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:35.828753 (+782636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:35.828798 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:35.841140 (+ 12342us) log.cc:844] Serialized 26 byte log entry
0709 13:07:36.032725 (+191585us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b48000606d8. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb79d24aa1 00007fcb79d1bb37 00007fcb79d1b9ca 00007fcb79e1326f 00007fcb79e12f37 00007fcb79e2755c 00007fcb79e27508 00007fcb79e27488 00007fcb79e27443 00007fcb79e266ec 00007fcb78d50a96
0709 13:07:36.104592 (+ 71867us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:36.104637 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:36.104696 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:36.892339 (+787643us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b24003ad8e8. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71b949c4 00007fcb71b92acb 00007fcb71d4ddc9 00007fcb71d4da55 00007fcb71d4cfb0 00007fcb749c3488 00007fcb749c3be8 00007fcb7499b398 00007fcb7499676a 00007fcb74995bb6
0709 13:07:36.892908 (+   569us) raft_consensus.cc:1685] finished
0709 13:07:36.892932 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:36.893069 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":508074,"raft.queue_time_us":528027,"raft.run_cpu_time_us":2941,"raft.run_wall_time_us":458336,"spinlock_wait_cycles":89891456,"thread_start_us":429253,"threads_started":2,"wal-append.queue_time_us":436519}
I20260709 13:07:36.894743 25239 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.897586 25299 raft_consensus.cc:2764] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:07:36.897672 25304 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:36.913096 22901 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:07:36.920149 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 55223.5us
I20260709 13:07:36.934480 25087 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:36.952276 25029 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.955646 25270 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:36.957098 25326 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:36.858837 25249 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:36.962194 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 178900us
I20260709 13:07:36.952977 25029 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:36.966244 22823 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:36.966810 22823 raft_consensus.cc:606] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:36.967445 25029 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.967828 25300 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.969576 25338 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.970301 25300 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:36.972451 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:36.974859 25087 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:36.975586 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 67449ba22ccb4938be8a5e28664cc566: Illegal state: Replica 6a5191aaff76469bbf6c4735dc9e6509 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
W20260709 13:07:36.965214 25333 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.383s	user 0.004s	sys 0.000s
I20260709 13:07:36.977428 25326 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:36.977942 25326 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:36.980001 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:36.975320 25087 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.030339 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:37.063347 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 1781) took 1339 ms. Trace:
I20260709 13:07:37.053951 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:37.036823 22904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 1735) took 2433 ms (client timeout 1819 ms). Trace:
I20260709 13:07:37.005307 22903 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:37.098973 22750 rpcz_store.cc:276] 0709 13:07:35.723566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.723683 (+   117us) service_pool.cc:224] Handling call
0709 13:07:35.771516 (+ 47833us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:35.775691 (+  4175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:35.775735 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:36.037484 (+261749us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4c00006ac0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb730c7a20 00007fcb730d8a29 00007fcb7309e644 00007fcb730c5f2d 00007fcb730c5b93 00007fcb74df618b 00007fcb74df6cb5 00007fcb74df8004 00007fcb74df7729 00007fcb74dfcd96 00007fcb74df1adb
0709 13:07:36.038343 (+   859us) log.cc:844] Serialized 556 byte log entry
0709 13:07:36.139837 (+101494us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:36.139874 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.063023 (+923149us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b540011b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:37.063077 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:37.063123 (+    46us) raft_consensus.cc:1685] finished
0709 13:07:37.063141 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.063281 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53993216,"thread_start_us":101215,"threads_started":1,"wal-append.queue_time_us":250203}
I20260709 13:07:37.074489 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:37.030870 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.101706 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:37.030211 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 275981us
I20260709 13:07:37.104540 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.104640 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:36.988750 22753 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:37.105755 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.067028 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:37.106758 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.088999 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:37.108767 25162 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:37.108927 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:37.090174 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:37.099323 22904 rpcz_store.cc:269] 0709 13:07:34.603021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.603119 (+    98us) service_pool.cc:224] Handling call
0709 13:07:37.036602 (+2433483us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540035ded4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:37.036742 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2336502528}
I20260709 13:07:37.111526 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.113348 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.114663 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 805) took 1580 ms. Trace:
I20260709 13:07:37.115152 22823 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:07:37.114791 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 84020.6us
I20260709 13:07:37.116654 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.117580 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.119247 22753 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:07:37.120520 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.121670 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:37.123817 22719 connection.cc:256] RPC call timeout handler was delayed by -1.1459776880000163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 13:07:37.124199 22719 connection.cc:256] RPC call timeout handler was delayed by -1.0370776879999539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
I20260709 13:07:37.125049 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.125399 22748 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:37.115201 23450 rpcz_store.cc:276] 0709 13:07:35.534425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.586986 (+ 52561us) service_pool.cc:224] Handling call
0709 13:07:35.587401 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:35.942072 (+354671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.114273 (+1172201us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400672d54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:37.114576 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560781696}
I20260709 13:07:37.129271 23416 leader_election.cc:400] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:37.131894 23416 leader_election.cc:403] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:07:37.132211 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 78331.9us
I20260709 13:07:37.132807 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.133095 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.131465 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.133524 22826 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:37.060915 25297 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.134406 23450 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:36.987581 25226 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:37.135655 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.135917 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:37.136121 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.141758 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.143287 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.144352 22824 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:37.138170 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:37.144883 23452 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:37.052666 25315 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:37.153746 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.154512 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.155028 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.155292 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:37.155602 22901 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:37.054538 25326 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:37.156425 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.157135 25245 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:37.157598 25326 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:37.148360 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.158809 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:37.162883 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:37.166401 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:37.149710 25249 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:37.150840 25286 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:37.153009 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.153878 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.176870 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:37.144883 22825 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:37.143762 22831 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:37.142573 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.178434 23449 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:37.178612 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.132120 25029 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:37.053980 25087 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:37.137642 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:37.186473 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:37.186885 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49297.8us
I20260709 13:07:37.187456 22899 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:37.142884 25193 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:37.109270 25162 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:37.165848 25111 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.188302 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:37.138025 25343 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:37.168381 25226 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:37.127830 25359 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.169122 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.131455 25340 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:37.170264 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.141609 25297 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:37.137977 25313 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:37.141779 25355 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:37.057242 25307 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.131230 25114 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:07:37.164279 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:37.213466 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49276.6us
I20260709 13:07:37.185913 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:37.116151 25241 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:37.185257 22902 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:37.153828 25327 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:37.189888 22826 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:07:37.193136 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.191929 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.194746 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:37.181155 24616 consensus_meta.cc:287] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.002s
I20260709 13:07:37.131600 25294 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:07:37.198294 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.169466 25096 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:37.176501 25315 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:37.169616 25304 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:37.141553 25285 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.175822 25286 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:37.180173 25245 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:07:37.195787 25176 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:37.203047 25326 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:07:37.194913 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.193532 25291 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:37.232972 23449 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:37.238111 25353 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.196365 25299 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:37.201188 25029 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:37.248795 22900 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:37.250347 25355 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.195920 25306 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:37.202906 25325 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:07:37.197831 25111 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.229991 25241 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.208807 25346 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:07:37.202183 25343 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.235364 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.263348 22823 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:37.262218 25291 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:37.264894 25360 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:07:37.212802 25210 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
W20260709 13:07:37.271751 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57261.2us
I20260709 13:07:37.247166 24616 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.201056 25359 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.272748 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.276456 24997 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:37.277254 25029 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:37.272873 25226 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.278578 25123 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:07:37.279173 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.279213 25162 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:37.279850 25369 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.258028 25315 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:37.281154 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 71268.8us
W20260709 13:07:37.281605 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48496.7us
I20260709 13:07:37.285745 25307 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:37.287222 22822 consensus_meta.cc:287] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.000s
I20260709 13:07:37.282451 25306 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.277141 25285 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:37.295182 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 59914.2us
I20260709 13:07:37.262710 25300 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:07:37.530797 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 534) took 1104 ms. Trace:
I20260709 13:07:37.536418 22905 rpcz_store.cc:276] 0709 13:07:36.426100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.453124 (+ 27024us) service_pool.cc:224] Handling call
0709 13:07:37.519260 (+1066136us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b5800211598. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:37.519477 (+   217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.519695 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:37.519731 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:37.519783 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:37.519828 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.519894 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.530727 (+ 10833us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10721536}
I20260709 13:07:37.321360 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.512171 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:37.382278 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:37.536067 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:37.502543 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:37.382447 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:37.463047 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:37.295754 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 159884us
I20260709 13:07:37.361191 22748 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:37.546846 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:37.536397 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.547729 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 388) took 1101 ms. Trace:
I20260709 13:07:37.550144 22830 rpcz_store.cc:276] 0709 13:07:36.441134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.441348 (+   214us) service_pool.cc:224] Handling call
0709 13:07:37.321251 (+879903us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5800144f98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6eee 0000000000375939 00007fcb79d4a1e0 00007fcb79e130c4 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:07:37.542692 (+221441us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38347008}
I20260709 13:07:37.332365 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.541599 22754 raft_consensus.cc:2955] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 3 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:37.484561 22654 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 3 to 4, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 4 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.551925 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 533) took 1176 ms. Trace:
I20260709 13:07:37.552405 22898 rpcz_store.cc:276] 0709 13:07:36.375539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.420353 (+ 44814us) service_pool.cc:224] Handling call
0709 13:07:37.332215 (+911862us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b5800211598. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:07:37.551867 (+219652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5472512}
I20260709 13:07:37.292443 25241 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:37.542246 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:37.544441 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.558002 22753 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:37.556850 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:37.553066 25315 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:37.420351 25276 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.488s	user 0.006s	sys 0.000s
I20260709 13:07:37.561165 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.520349 25063 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:37.571255 25063 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:37.571712 25374 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:37.572028 25370 consensus_queue.cc:1059] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:37.572167 25063 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.573215 25190 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:37.573504 25063 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:37.574481 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.577000 25370 raft_consensus.cc:3055] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:37.577414 25370 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:37.568262 25282 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:07:37.568396 25377 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:07:37.568107 24811 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.426728 25311 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:37.296222 25291 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:37.538959 25176 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.567874 23444 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:37.497166 25299 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:37.597391 22830 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:07:37.296267 25294 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:37.569820 22831 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:37.556311 25330 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.573634 25359 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:37.578222 25370 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.541482 25306 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:37.566047 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:37.323585 25015 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.324962 25127 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:37.585250 23453 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:37.587357 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.587800 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.588619 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:37.561962 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:37.609140 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47255.8us
I20260709 13:07:37.610404 22901 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:37.539723 25123 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:37.328871 25343 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:37.322861 25111 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:37.568934 23451 raft_consensus.cc:2955] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 3 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:37.562621 25307 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:07:37.362709 25240 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:37.595640 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.630319 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.586179 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.555413 25162 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:37.626569 25353 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:07:37.636087 25162 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.636497 25162 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:37.636669 25353 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:37.637082 25353 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:37.637919 25353 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:37.638546 25111 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.639096 25353 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:37.639467 25378 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:37.547506 22822 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:37.641850 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2051) took 1078 ms. Trace:
I20260709 13:07:37.641671 25240 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.642230 22822 rpcz_store.cc:276] 0709 13:07:36.563262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.565145 (+  1883us) service_pool.cc:224] Handling call
0709 13:07:37.641789 (+1076644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:37.590435 25282 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.642733 24616 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:37.643123 25282 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:37.643752 25240 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:37.645227 25282 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:37.606439 22896 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:37.612547 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.646627 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.647688 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.648428 25162 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.594065 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.649344 25330 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:37.649480 25130 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:37.615052 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.618847 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.594515 25293 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:37.592063 25311 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:37.593463 23445 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:37.622468 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2206) took 3042 ms. Trace:
I20260709 13:07:37.326515 25327 raft_consensus.cc:2955] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index 3 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:37.620297 25359 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:37.623925 22897 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:37.604717 22904 raft_consensus.cc:2955] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 3 to 5, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:37.600158 25299 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:37.627563 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:37.329805 25286 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:37.631251 22752 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:37.685010 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:37.685907 22752 raft_consensus.cc:2410] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:07:37.635557 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.648798 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.556458 25239 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:07:37.651546 23453 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:37.651885 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.619004 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.658264 22749 rpcz_store.cc:276] 0709 13:07:34.579699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.579809 (+   110us) service_pool.cc:224] Handling call
0709 13:07:34.580139 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:34.591333 (+ 11194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:34.591364 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:34.606450 (+ 15086us) log.cc:844] Serialized 26 byte log entry
0709 13:07:34.804329 (+197879us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:34.804382 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.814982 (+ 10600us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:37.399725 (+2584743us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:37.622211 (+222486us) raft_consensus.cc:1685] finished
0709 13:07:37.622231 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:37.622419 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10819,"raft.queue_time_us":10907,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":8200,"spinlock_wait_cycles":55317888,"thread_start_us":164064,"threads_started":1,"wal-append.queue_time_us":1211720}
I20260709 13:07:37.679672 25359 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.691354 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:37.691876 22749 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:37.667142 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.543792 25369 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.649785 25330 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:37.700132 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.687736 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.687678 25285 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:37.657819 25370 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:37.650245 25130 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.695786 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.687351 25240 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.658535 25293 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:37.664842 25193 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.703095 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.628312 25287 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:07:37.705785 22901 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:07:37.706303 22899 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
W20260709 13:07:37.705089 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53298.2us
I20260709 13:07:37.749624 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:37.737756 25293 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:37.620676 25015 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.691263 25359 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:37.695689 25111 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.737205 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:37.747454 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:37.757225 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56838.3us
I20260709 13:07:37.702680 25330 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:37.703662 25378 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:37.713567 25340 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.712983 23446 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:07:37.703776 25338 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:37.713464 25307 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:37.713343 25325 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.747715 22749 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:37.755488 25309 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:37.785900 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:37.777410 22824 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:37.780995 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.781793 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:37.784420 22752 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:07:37.735209 25298 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:07:37.767190 24616 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:37.769793 25285 consensus_queue.cc:1243] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:37.814419 25096 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:37.754657 25249 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:07:37.755191 25278 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:07:37.703512 25123 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:37.773696 25312 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:37.785249 25287 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:37.765839 25235 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:07:37.853792 25369 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:37.859203 25297 consensus_meta.cc:287] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.000s
I20260709 13:07:37.818434 23447 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:37.861965 25390 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:37.825356 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:37.865037 25278 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:37.865408 25278 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.865739 25278 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:37.866046 25379 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:37.868005 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.869258 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.869949 22829 raft_consensus.cc:2368] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6.
I20260709 13:07:37.776778 25189 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:37.849352 25123 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:37.846105 25312 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.846315 25287 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:37.873508 25312 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:37.875136 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.876122 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.877451 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:37.887195 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.887933 22829 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:37.889549 23449 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:07:37.818358 24616 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.820189 25309 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.886703 24997 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:37.904796 25309 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.907018 25309 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:37.904829 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.907969 25190 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:37.908838 25190 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.909328 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:37.734002 25311 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.910746 25190 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:37.910992 25311 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:37.911290 22825 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:37.895794 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.785393 25210 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:07:37.874086 25162 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:37.904232 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.917121 22826 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:37.909484 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:37.896224 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:37.913363 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:37.918668 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 137895us
I20260709 13:07:37.789868 25206 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 13:07:37.872879 25278 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:37.847108 25298 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:37.812311 25373 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 13:07:37.798319 25325 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.920444 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:37.920953 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:37.901443 24616 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:37.956964 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:37.873974 25287 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.884240 25297 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:37.922380 25315 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.922931 25298 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:37.935508 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.937994 25294 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:37.942341 25015 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:37.944047 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:37.961575 25247 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:37.778369 25378 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:37.916397 24997 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:37.973541 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.925032 23445 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:37.918392 25211 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:37.979792 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 267136us
I20260709 13:07:37.935307 25304 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:38.016659 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:38.022270 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.944887 25291 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.051332 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:37.989694 22755 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:38.005843 22750 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:37.978112 25307 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:38.018260 22749 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:07:38.046458 22823 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:37.914708 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:38.049612 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 302190us
W20260709 13:07:38.055792 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 141155us
I20260709 13:07:38.057081 23416 leader_election.cc:400] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6.
I20260709 13:07:38.058926 23416 leader_election.cc:403] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:38.060254 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:38.003623 25247 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:38.062753 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:37.987536 25315 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.060194 23448 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:07:38.056301 25361 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:38.048424 25127 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.073418 22753 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:38.055200 25334 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.061868 25239 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:07:38.083225 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:38.020711 25211 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.046344 25330 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:38.062022 25294 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:38.066036 25226 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:38.065488 25193 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:38.086319 25334 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.088024 25309 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:38.089218 25315 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:38.078262 25340 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:38.063864 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:38.143527 25393 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:38.060478 25298 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:38.174225 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 111523us
I20260709 13:07:38.150255 25390 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:38.167097 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 103293us
I20260709 13:07:38.148567 25193 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:38.194339 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.081965 25127 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.076076 25306 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:38.164492 25325 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:38.146055 25294 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.090631 24997 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:38.201617 25394 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:07:38.213201 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 136672us
I20260709 13:07:38.196532 25278 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:38.175876 25401 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:07:38.223942 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.224458 22748 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:07:38.152871 25359 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:38.155064 25309 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.198722 25193 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:38.234601 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.236925 22756 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:07:38.247342 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 462) took 1143 ms. Trace:
I20260709 13:07:38.247542 23452 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:38.203192 25298 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:38.258379 25211 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:38.229235 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:38.227766 25297 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:38.172024 25312 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.250406 25330 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:38.273025 22749 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:07:38.274626 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.196211 25247 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:38.292650 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:38.270648 25278 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:38.272400 25378 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:38.311286 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.282040 25111 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:07:38.256804 23448 rpcz_store.cc:276] 0709 13:07:37.104163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.104296 (+   133us) service_pool.cc:224] Handling call
0709 13:07:38.247135 (+1142839us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b540024ca54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:38.247284 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2057952640}
I20260709 13:07:38.324662 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:38.319798 25403 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:07:38.317723 22898 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:07:38.331458 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:38.335341 22904 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:07:38.335741 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.336119 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.336278 22903 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:38.336476 22904 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:38.337162 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.338920 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:38.341334 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.344434 22900 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:07:38.342619 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.327090 25111 consensus_queue.cc:1243] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:07:38.349918 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.350713 22897 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
W20260709 13:07:38.341854 25299 consensus_meta.cc:287] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.647s	user 0.010s	sys 0.000s
W20260709 13:07:38.368327 25387 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:07:38.371654 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:38.371579 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:38.375296 22751 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:38.379103 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.346604 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.386322 25359 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:38.392074 25123 consensus_meta.cc:287] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.000s
I20260709 13:07:38.377596 25127 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:38.398239 25404 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:07:38.400019 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.400892 23449 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:38.401819 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.405258 22825 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:07:38.406304 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.407080 22825 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:38.409888 22825 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:38.411471 23449 raft_consensus.cc:2410] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:38.416561 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.416787 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:38.419193 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.410609 22825 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.424405 22825 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:38.382380 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:38.424732 25294 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:38.406364 25312 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:38.425235 25311 consensus_meta.cc:287] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 13:07:38.434304 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.435202 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:38.450816 25340 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:38.453162 25286 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:38.456826 22751 raft_consensus.cc:2410] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:07:38.457463 25211 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:07:38.441331 25420 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:07:38.431083 22903 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:38.461048 22897 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:38.458482 25340 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:38.466287 25340 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:38.467209 25340 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.468405 25340 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:38.430461 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.445216 23449 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:38.445869 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:38.378290 25299 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:38.471539 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:38.427423 25306 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:38.474720 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:38.479696 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:38.478369 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 467) took 1316 ms. Trace:
I20260709 13:07:38.478562 22825 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:07:38.491462 22825 pending_rounds.cc:85] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 5
I20260709 13:07:38.491799 22825 pending_rounds.cc:107] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:07:38.492292 22825 raft_consensus.cc:2889] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:38.501379 22899 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:38.499547 25311 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.513281 25311 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.515417 25311 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:38.472646 25309 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:38.512871 25293 consensus_meta.cc:287] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.744s	user 0.003s	sys 0.004s
I20260709 13:07:38.492574 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:38.490809 23450 rpcz_store.cc:276] 0709 13:07:37.161495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.161659 (+   164us) service_pool.cc:224] Handling call
0709 13:07:37.614057 (+452398us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:37.614312 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:37.614952 (+   640us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:37.614980 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:37.628601 (+ 13621us) log.cc:844] Serialized 26 byte log entry
0709 13:07:37.774106 (+145505us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:37.774165 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.774249 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:38.473350 (+699101us) raft_consensus.cc:1685] finished
0709 13:07:38.473370 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:38.478314 (+  4944us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":254,"raft.queue_time_us":679,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":11998,"spinlock_wait_cycles":12425472,"thread_start_us":74377,"threads_started":1,"wal-append.queue_time_us":322925}
I20260709 13:07:38.476567 25211 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:38.499040 25408 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:38.524649 22904 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:38.527165 25226 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:38.489208 22822 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.502s	user 0.001s	sys 0.005s
W20260709 13:07:38.533048 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.533371 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1283) took 1969 ms. Trace:
I20260709 13:07:38.491878 25206 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.478317 25418 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:38.505607 25377 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:38.537356 22896 rpcz_store.cc:276] 0709 13:07:36.564159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.564327 (+   168us) service_pool.cc:224] Handling call
0709 13:07:36.576398 (+ 12071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.645940 (+1069542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.533159 (+887219us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540034b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:38.533299 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2365143296}
I20260709 13:07:38.535076 25419 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:07:38.539099 22822 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
W20260709 13:07:38.558038 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.535698 25293 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.558923 22754 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:38.559769 25293 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:38.560289 25193 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.562211 25193 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:38.564723 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:38.584133 25299 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:38.588229 22826 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:38.592603 23453 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:38.547259 25206 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:38.658612 25239 consensus_meta.cc:287] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.567s	user 0.007s	sys 0.001s
I20260709 13:07:38.659061 25239 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6.
I20260709 13:07:38.645160 25293 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.659897 25293 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:38.661911 25293 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:38.664085 25206 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:38.667408 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:37.912184 25285 consensus_queue.cc:1175] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3b626c41600d4737a178ebab8a7ebfc2" responder_term: 5 status { last_received { term: 3 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:07:38.669157 24616 consensus_meta.cc:287] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.001s
I20260709 13:07:38.693383 24616 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.695886 24616 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:38.698117 23449 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
W20260709 13:07:38.668815 25241 consensus_meta.cc:287] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.086s	user 0.005s	sys 0.006s
I20260709 13:07:38.649169 25226 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:38.755079 25334 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:38.758355 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:07:38.759440 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.760507 25377 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:38.760893 25377 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:38.761241 25377 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:38.761989 25095 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:38.762480 25095 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:38.764405 25095 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:38.765249 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:38.766584 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.766729 25095 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:38.767567 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:38.767746 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.769109 25377 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.771078 25377 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:38.771337 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.771826 25095 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:38.772050 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:38.774770 25372 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:38.775302 25372 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:38.776110 25372 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.777062 25372 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:38.784423 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.785784 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2058) took 1198 ms. Trace:
I20260709 13:07:38.786577 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 820) took 1091 ms. Trace:
I20260709 13:07:38.807565 22824 rpcz_store.cc:276] 0709 13:07:37.587824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.587924 (+   100us) service_pool.cc:224] Handling call
0709 13:07:37.588166 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.788023 (+199857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.785579 (+997556us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54001f9ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:38.785730 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434266880}
I20260709 13:07:38.808197 23452 rpcz_store.cc:276] 0709 13:07:37.694857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.908435 (+213578us) service_pool.cc:224] Handling call
0709 13:07:38.247278 (+338843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.257700 (+ 10422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.786415 (+528715us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b540024ca50. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:38.786525 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778469248}
I20260709 13:07:38.789404 25249 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:38.790794 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 1798) took 1427 ms. Trace:
W20260709 13:07:38.793763 25282 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:07:38.797618 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:38.814350 22897 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:38.798221 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.799755 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.800194 25270 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:38.801640 22653 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33010 (request call id 222) took 14801 ms (client timeout 15000 ms). Trace:
I20260709 13:07:38.802165 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:38.772921 25245 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:07:38.802834 22755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 588) took 2511 ms (client timeout 2221 ms). Trace:
I20260709 13:07:38.804436 23451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:07:38.793113 22654 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 3, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:38.810652 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:38.815797 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:38.811775 22756 rpcz_store.cc:276] 0709 13:07:37.363023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.363135 (+   112us) service_pool.cc:224] Handling call
0709 13:07:37.365648 (+  2513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:38.253143 (+887495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.789557 (+536414us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b540003ad20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d72dc0 00007fcb74dfea02 00007fcb74dfadd6 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:07:38.790477 (+   920us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b540016b054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:38.790715 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2042190976}
I20260709 13:07:38.818742 25392 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 13:07:38.824918 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.829982 22747 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:07:38.830329 22747 pending_rounds.cc:85] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 3
W20260709 13:07:38.804922 25416 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.481s	user 0.003s	sys 0.001s
I20260709 13:07:38.822403 23451 raft_consensus.cc:606] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:38.830144 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:38.831686 22751 raft_consensus.cc:2410] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:07:38.831727 25392 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:38.832442 25095 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:38.829550 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:38.830580 22747 pending_rounds.cc:107] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:07:38.827771 23450 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:38.835233 22747 raft_consensus.cc:2889] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:38.821419 22755 rpcz_store.cc:269] 0709 13:07:36.291715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.297958 (+  6243us) service_pool.cc:224] Handling call
0709 13:07:38.802622 (+2504664us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54003f0cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:38.802751 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3888823168}
I20260709 13:07:38.847812 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:38.854292 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:38.855506 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:38.848994 22654 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 3 to 5, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 5 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.750006 25226 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.859799 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:38.863927 22824 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:38.860697 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:38.865422 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:38.866547 22654 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 3 to 4, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 4 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.832316 25392 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:38.828189 25401 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:38.885275 22654 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.869558 25418 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:07:38.869815 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.874256 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:38.875358 25293 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:38.861941 25334 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:38.818137 22653 rpcz_store.cc:269] 0709 13:07:23.999816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.999990 (+   174us) service_pool.cc:224] Handling call
0709 13:07:38.801565 (+14801575us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:33.495257 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:33.594479 (+ 99222us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:33.594535 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:07:33.594550 (+    15us) tablet.cc:662] Decoding operations
0709 13:07:33.596088 (+  1538us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:33.596159 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:33.596195 (+    36us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:07:33.624307 (+ 28112us) tablet.cc:701] Row locks acquired
0709 13:07:33.624357 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:07:33.624548 (+   191us) write_op.cc:270] Start()
0709 13:07:33.624731 (+   183us) write_op.cc:276] Timestamp: P: 1783602453624463 usec, L: 0
0709 13:07:33.624750 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:07:33.625583 (+   833us) log.cc:844] Serialized 6234 byte log entry
0709 13:07:34.261782 (+636199us) op_driver.cc:464] REPLICATION: finished
0709 13:07:34.433640 (+171858us) write_op.cc:301] APPLY: starting
0709 13:07:34.433733 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:34.434085 (+   352us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:34.434207 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:34.466794 (+ 32587us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:34.519949 (+ 53155us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:07:34.520005 (+    56us) write_op.cc:312] APPLY: finished
0709 13:07:34.520428 (+   423us) log.cc:844] Serialized 113 byte log entry
0709 13:07:34.520614 (+   186us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:34.520772 (+   158us) write_op.cc:454] Released schema lock
0709 13:07:34.520841 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:34.521246 (+   405us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:34.521276 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":213541,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":171590,"apply.run_cpu_time_us":8256,"apply.run_wall_time_us":87561,"num_ops":16,"prepare.queue_time_us":118121,"prepare.run_cpu_time_us":13541,"prepare.run_wall_time_us":283587,"raft.queue_time_us":213853,"raft.run_cpu_time_us":31458,"raft.run_wall_time_us":133347,"replication_time_us":636864,"thread_start_us":486680,"threads_started":4,"wal-append.queue_time_us":305122,"wal-append.run_cpu_time_us":13422,"wal-append.run_wall_time_us":1581940}]]}
I20260709 13:07:38.828423 25330 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:38.826840 25399 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:07:38.839705 25095 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:38.886795 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:38.868790 25114 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:38.823275 25270 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:38.748951 25241 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:38.995676 22828 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:07:38.996793 23479 heartbeater.cc:646] Failed to heartbeat to 127.21.31.126:35001 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.31.126:35001 timed out after 15.000s (SENT)
I20260709 13:07:38.909741 22749 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:38.916544 22897 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:38.899525 22823 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:07:39.000285 22823 pending_rounds.cc:85] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 5
I20260709 13:07:39.000607 22823 pending_rounds.cc:107] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:07:39.000891 22823 raft_consensus.cc:2889] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:38.905212 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.972815 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.857964 25419 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:38.964573 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:38.969491 25422 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:38.971632 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:38.965247 25235 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:07:38.992069 25291 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.005712 23452 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:39.006426 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:38.982160 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:38.898128 25401 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:39.007568 25373 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:38.971607 25293 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.010429 25095 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.014457 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:39.018421 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:38.890157 25402 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.639s	user 0.000s	sys 0.013s
W20260709 13:07:38.926939 25428 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:39.022919 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:39.023772 25291 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.053771 25291 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:39.054417 25226 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:39.056229 25291 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.056867 25291 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:39.027235 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:39.021576 22654 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 5, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 5 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:39.060019 25287 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:39.060621 22654 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.062526 22654 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.891215 25392 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:38.963856 25330 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.067929 25330 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.068043 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 398) took 1166 ms. Trace:
I20260709 13:07:39.068634 22822 rpcz_store.cc:276] 0709 13:07:37.901800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.909012 (+  7212us) service_pool.cc:224] Handling call
0709 13:07:39.067854 (+1158842us) spinlock_profiling.cc:243] Waited 76.8 ms on lock 0x7b540008b454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:39.067977 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176657152}
I20260709 13:07:39.070187 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:39.057278 25291 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.071996 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 806) took 1942 ms. Trace:
I20260709 13:07:39.045723 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:38.960336 25418 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.071774 25291 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:38.998062 25309 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:07:39.029232 25422 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:39.072576 23447 rpcz_store.cc:276] 0709 13:07:37.129168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.131937 (+  2769us) service_pool.cc:224] Handling call
0709 13:07:37.786351 (+654414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.862341 (+ 75990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.071772 (+1209431us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b5400569fd0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:39.071930 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415182080}
I20260709 13:07:38.971737 25334 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.089186 25095 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:39.101266 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 614) took 1490 ms. Trace:
I20260709 13:07:39.101778 22754 rpcz_store.cc:276] 0709 13:07:37.610724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.712332 (+101608us) service_pool.cc:224] Handling call
0709 13:07:37.712571 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.559264 (+846693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.100985 (+541721us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b5400045954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:39.101197 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1946014976}
I20260709 13:07:39.077533 25226 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:39.080672 25304 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:39.091146 25400 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.883s	user 0.005s	sys 0.000s
W20260709 13:07:39.137679 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.149052 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.082821 25418 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:39.165895 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.168794 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:39.169617 22826 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:39.169852 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 609) took 1581 ms. Trace:
I20260709 13:07:39.170395 22753 rpcz_store.cc:276] 0709 13:07:37.588477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.589123 (+   646us) service_pool.cc:224] Handling call
0709 13:07:37.589433 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.078467 (+489034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.169561 (+1091094us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b5400076cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:39.169777 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112428800}
I20260709 13:07:39.174995 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1434) took 2051 ms. Trace:
I20260709 13:07:39.175472 22828 rpcz_store.cc:276] 0709 13:07:37.123083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.123190 (+   107us) service_pool.cc:224] Handling call
0709 13:07:37.123437 (+   247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:38.996036 (+1872599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.174700 (+178664us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b5400433854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:39.174927 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4305143680}
I20260709 13:07:39.161742 25304 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.182029 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2254) took 1636 ms. Trace:
I20260709 13:07:39.181968 25422 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:39.182479 22748 rpcz_store.cc:276] 0709 13:07:37.545459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.555772 (+ 10313us) service_pool.cc:224] Handling call
0709 13:07:37.556042 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:38.232743 (+676701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.181807 (+949064us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b5400336dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:39.181973 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507509120}
I20260709 13:07:39.184406 22902 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:39.105515 25417 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:39.189622 25417 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:39.191154 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.192020 25417 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.159395 25312 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:39.193423 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 818) took 1598 ms. Trace:
I20260709 13:07:39.193820 23444 rpcz_store.cc:276] 0709 13:07:37.595378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.794695 (+199317us) service_pool.cc:224] Handling call
0709 13:07:37.794954 (+   259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:37.800651 (+  5697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:37.800693 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:37.820663 (+ 19970us) log.cc:844] Serialized 26 byte log entry
0709 13:07:38.005266 (+184603us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:38.007030 (+  1764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.007125 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:39.056979 (+1049854us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:39.193137 (+136158us) raft_consensus.cc:1685] finished
0709 13:07:39.193162 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.193368 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5320,"raft.queue_time_us":5895,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":8167,"spinlock_wait_cycles":7785216,"thread_start_us":143366,"threads_started":1,"wal-append.queue_time_us":289389}
I20260709 13:07:39.195524 25309 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:39.196020 25309 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.196537 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 1811) took 1599 ms. Trace:
I20260709 13:07:39.196903 25293 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.107613 25226 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.197891 25309 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.199035 25226 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.202080 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:39.202207 25343 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:39.203469 25343 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:39.203920 25343 raft_consensus.cc:697] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:39.204552 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.204938 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 810) took 2064 ms. Trace:
I20260709 13:07:39.205307 23446 rpcz_store.cc:276] 0709 13:07:37.140309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.142735 (+  2426us) service_pool.cc:224] Handling call
0709 13:07:37.559612 (+416877us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6eee 0000000000375939 00007fcb79d4a1e0 00007fcb79e130c4 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:37.567286 (+  7674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:37.783039 (+215753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.204691 (+1421652us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400684054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:39.204882 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298819200}
I20260709 13:07:39.206799 25334 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:39.207114 25402 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:07:39.208250 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 401) took 1303 ms. Trace:
I20260709 13:07:39.196897 22752 rpcz_store.cc:276] 0709 13:07:37.600657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.693553 (+ 92896us) service_pool.cc:224] Handling call
0709 13:07:37.784109 (+ 90556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:37.794426 (+ 10317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.196358 (+1401932us) spinlock_profiling.cc:243] Waited 90.3 ms on lock 0x7b540011c1d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:39.196475 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207589632}
I20260709 13:07:39.176841 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:39.215468 22828 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:39.216455 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:39.217198 22828 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:39.218317 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:39.208585 22830 rpcz_store.cc:276] 0709 13:07:37.904440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.905105 (+   665us) service_pool.cc:224] Handling call
0709 13:07:37.905420 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:37.905728 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:37.905752 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:37.906167 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 13:07:38.045246 (+139079us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:38.149813 (+104567us) log.cc:844] Serialized 14 byte log entry
0709 13:07:38.153734 (+  3921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.153800 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:38.517938 (+364138us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b180029ca40. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:07:38.889543 (+371605us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:39.208069 (+318526us) raft_consensus.cc:1685] finished
0709 13:07:39.208089 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.208203 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9003,"prepare.queue_time_us":127937,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":59692,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":38234,"spinlock_wait_cycles":35247872,"thread_start_us":221927,"threads_started":2,"wal-append.queue_time_us":259984}
I20260709 13:07:39.219234 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:39.229789 22899 raft_consensus.cc:2393] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:39.223240 25401 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.223126 25291 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.193034 25312 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.223465 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:39.224187 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.226428 22747 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:39.226668 22752 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:39.229259 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.204711 25343 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.238976 22756 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:39.238867 22830 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:39.249244 22830 raft_consensus.cc:606] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:39.249667 22830 raft_consensus.cc:614] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:39.228857 25375 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:39.242023 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 407031us
I20260709 13:07:39.243661 22750 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 1.
I20260709 13:07:39.250746 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.207844 25270 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:39.250854 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:39.253118 22754 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:39.224071 25282 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:39.228716 25417 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:39.241762 25245 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:39.251339 23448 consensus_meta.cc:287] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.559s	user 0.004s	sys 0.004s
W20260709 13:07:39.266774 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 284665us
I20260709 13:07:39.267563 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:39.267741 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:39.264343 22826 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:39.268649 22828 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:39.269588 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:39.250322 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:39.250274 25241 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.260351 24616 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.242954 25009 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:07:39.287585 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:39.287750 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:39.283090 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:39.290194 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.290755 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.276695 25300 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:07:39.272337 25401 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:39.295648 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:39.297065 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.305363 22751 raft_consensus.cc:2410] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:07:39.271024 24997 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:39.301808 22903 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:07:39.296924 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:39.308526 22900 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:39.294481 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:39.294966 22899 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:07:39.298167 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.297894 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:39.293828 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:39.259922 25375 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:39.270607 23448 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:39.314482 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:39.319734 22901 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:39.288334 25226 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:39.295949 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:39.266268 25282 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.269447 25417 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:39.296617 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:39.286912 24811 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:07:39.292948 25239 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:07:39.326716 25375 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.328155 25291 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.274910 25301 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:07:39.272723 25245 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.346027 25245 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:39.346688 25412 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.052s	user 0.000s	sys 0.003s
I20260709 13:07:39.347066 25301 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:39.347455 25301 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:39.348245 25301 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.348812 22830 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:39.349244 22830 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:39.306842 25401 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.349793 25401 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:39.350258 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2070) took 1301 ms. Trace:
I20260709 13:07:39.350636 22831 rpcz_store.cc:276] 0709 13:07:38.049186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.049289 (+   103us) service_pool.cc:224] Handling call
0709 13:07:38.049563 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:38.062182 (+ 12619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:38.062213 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:38.071161 (+  8948us) log.cc:844] Serialized 26 byte log entry
0709 13:07:38.198871 (+127710us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:38.198910 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.199411 (+   501us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:39.346173 (+1146762us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:39.349995 (+  3822us) raft_consensus.cc:1685] finished
0709 13:07:39.350026 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.350207 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68540,"raft.queue_time_us":16115,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":11287,"spinlock_wait_cycles":5751168,"thread_start_us":101854,"threads_started":1,"wal-append.queue_time_us":170951}
I20260709 13:07:39.287525 25298 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:07:39.319423 25340 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:39.293143 25015 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:07:39.292779 25330 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:07:39.355381 25330 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:39.293776 24616 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.357620 24616 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.358628 22828 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:39.331487 25334 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.358989 25282 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.359328 25334 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:39.363613 25422 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:39.365911 25312 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.367746 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 827) took 1407 ms. Trace:
I20260709 13:07:39.320802 24997 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.368088 23449 rpcz_store.cc:276] 0709 13:07:37.960566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.477392 (+516826us) service_pool.cc:224] Handling call
0709 13:07:38.477664 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:38.698595 (+220931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.367536 (+668941us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b5400672d54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:39.367695 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506424064}
I20260709 13:07:39.336740 25287 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:39.324210 25369 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:07:39.370566 25301 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:39.414162 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.377763 25287 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.440166 25298 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.440801 25298 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:39.443915 25095 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:39.444512 25095 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.446902 25095 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.453486 25377 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.454164 25377 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.456822 25377 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.423758 25334 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:39.370716 25418 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.461915 25287 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:39.527021 25434 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:07:39.588994 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:39.651446 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 379784us
W20260709 13:07:39.698846 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 46515.8us
I20260709 13:07:39.716921 25293 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:39.716817 25330 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.713266 25162 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:39.719208 25442 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:39.824079 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.726099 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:07:39.720767 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 423162us
I20260709 13:07:39.826187 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:39.736712 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 441079us
I20260709 13:07:39.827069 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:39.827410 24616 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:39.827874 24616 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:39.717231 25375 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:39.735790 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 429700us
I20260709 13:07:39.828286 24616 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:39.826272 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 100259us
I20260709 13:07:39.832078 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:39.829142 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:39.844599 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.719691 25189 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:07:39.857877 25190 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:39.859179 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.867857 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.868532 22751 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:39.834421 25162 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:39.840822 25340 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:39.841063 25444 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:39.848424 25312 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:39.828994 25330 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.911660 22755 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:39.913116 24997 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:39.914244 25401 consensus_meta.cc:287] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.564s	user 0.021s	sys 0.010s
I20260709 13:07:39.914640 25401 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.916527 25401 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.842180 25442 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:39.917848 25226 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:39.918506 25226 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:39.919015 25226 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:39.920270 25442 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.920781 25442 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.923935 22830 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.897086 25418 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:39.935171 25162 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.940054 25371 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:39.946118 25371 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.948581 25371 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.881597 25300 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:07:39.949736 22830 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:39.952168 25422 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.588s	user 0.013s	sys 0.018s
I20260709 13:07:39.886940 25417 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:39.849175 25375 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.954631 25375 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.955269 22897 raft_consensus.cc:2410] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:39.955793 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:39.897933 25340 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:39.962450 25340 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.964774 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1449) took 1677 ms. Trace:
I20260709 13:07:39.965131 22825 rpcz_store.cc:276] 0709 13:07:38.287271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.516961 (+229690us) service_pool.cc:224] Handling call
0709 13:07:38.517304 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.665986 (+148682us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:07:38.666014 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:38.666051 (+    37us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:07:38.666083 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.964581 (+1298498us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002319d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:39.964619 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.964725 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":331,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":23274,"spinlock_wait_cycles":341199744}
I20260709 13:07:39.968725 25330 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:39.974510 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 404) took 1205 ms. Trace:
I20260709 13:07:39.974848 22824 rpcz_store.cc:276] 0709 13:07:38.768920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.886211 (+117291us) service_pool.cc:224] Handling call
0709 13:07:38.886479 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:38.886788 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:38.886814 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:38.971421 (+ 84607us) log.cc:844] Serialized 26 byte log entry
0709 13:07:39.256176 (+284755us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:39.731403 (+475227us) log.cc:844] Serialized 14 byte log entry
0709 13:07:39.731688 (+   285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.731789 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:39.972044 (+240255us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b1800407fe0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:07:39.974251 (+  2207us) raft_consensus.cc:1685] finished
0709 13:07:39.974339 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.974458 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":473403,"prepare.run_wall_time_us":31,"raft.queue_time_us":7466,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":68,"spinlock_wait_cycles":214872960,"thread_start_us":715826,"threads_started":2,"wal-append.queue_time_us":725532}
I20260709 13:07:39.896754 25312 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:39.921175 25442 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.987396 25312 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:39.906471 25463 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.989892 25463 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:39.999348 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.917802 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 828) took 1956 ms. Trace:
W20260709 13:07:39.999421 25437 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.269s	user 0.001s	sys 0.001s
I20260709 13:07:40.001523 24616 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:39.882344 25452 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.136s
I20260709 13:07:40.015187 25304 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:40.043401 25301 consensus_meta.cc:287] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.001s
I20260709 13:07:40.034960 25293 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:39.997161 25287 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.063289 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:40.064018 22902 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:40.066959 22753 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:07:39.983428 22830 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:07:40.077520 22830 pending_rounds.cc:85] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 5
I20260709 13:07:40.077879 22830 pending_rounds.cc:107] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:07:40.078202 22830 raft_consensus.cc:2889] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:40.079039 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 411) took 1308 ms. Trace:
I20260709 13:07:40.079619 22830 rpcz_store.cc:276] 0709 13:07:38.770658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.289960 (+519302us) service_pool.cc:224] Handling call
0709 13:07:39.290220 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.348721 (+ 58501us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b540021e1a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d72dc0 00007fcb74dfd1c0 00007fcb74dfe231 00007fcb74dfad8f 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c
0709 13:07:39.923517 (+574796us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b540021e1a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d7906f 00007fcb74df1f60 00007fcb74df37de 00007fcb74dfdb49 00007fcb74dfe416 00007fcb74dfad8f 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62
0709 13:07:40.078761 (+155244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.078975 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161307904}
W20260709 13:07:40.098337 23448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 823) took 2187 ms (client timeout 2139 ms). Trace:
W20260709 13:07:40.098842 23448 rpcz_store.cc:269] 0709 13:07:37.910532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.354614 (+444082us) service_pool.cc:224] Handling call
0709 13:07:40.098123 (+1743509us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400554454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:40.098270 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526966144}
I20260709 13:07:40.096304 25291 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.046767 25226 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:40.148818 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 309276us
I20260709 13:07:40.152272 25452 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:40.152532 25287 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.159842 22902 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:40.000088 23445 rpcz_store.cc:276] 0709 13:07:37.960927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.491713 (+530786us) service_pool.cc:224] Handling call
0709 13:07:38.552944 (+ 61231us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:38.573643 (+ 20699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:38.573680 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:38.574271 (+   591us) log.cc:844] Serialized 52 byte log entry
0709 13:07:38.812718 (+238447us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:38.990094 (+177376us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b18001f3000. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0b455 00007fcb787f33c7 00007fcb74e059a7 00007fcb74e150bf 00007fcb74e15038
0709 13:07:39.265580 (+275486us) log.cc:844] Serialized 14 byte log entry
0709 13:07:39.369383 (+103803us) spinlock_profiling.cc:243] Waited 8.52 ms on lock 0x7b18001f3000. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:07:39.733227 (+363844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:39.733345 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:39.917597 (+184252us) raft_consensus.cc:1685] finished
0709 13:07:39.917614 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:39.917745 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12426,"prepare.queue_time_us":27689,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":4398,"raft.queue_time_us":37859,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":8946,"spinlock_wait_cycles":28161664,"thread_start_us":877612,"threads_started":3,"wal-append.queue_time_us":323075}
I20260709 13:07:40.185266 25282 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:40.185535 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:40.185763 25282 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:40.187918 25226 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:40.189370 25442 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.191787 25463 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.192346 25340 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:40.192406 24616 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.192853 25346 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:07:40.188819 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 356809us
I20260709 13:07:40.137696 25291 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.196099 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:40.202975 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:40.203195 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.203717 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:40.205819 25282 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.186432 25453 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:07:40.194651 25287 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:40.196744 25340 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.211385 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.211642 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.212169 25340 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.213622 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.214839 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.216465 25291 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.216815 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.217490 25442 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:40.217872 25442 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.218199 25442 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:40.217501 23448 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:40.217182 23445 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:40.222635 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2072) took 1260 ms. Trace:
I20260709 13:07:40.222992 22826 rpcz_store.cc:276] 0709 13:07:38.962683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.319784 (+357101us) service_pool.cc:224] Handling call
0709 13:07:39.335244 (+ 15460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.335418 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:39.724153 (+388735us) log.cc:844] Serialized 14 byte log entry
0709 13:07:39.874876 (+150723us) log.cc:844] Serialized 14 byte log entry
0709 13:07:39.896436 (+ 21560us) log.cc:844] Serialized 14 byte log entry
0709 13:07:40.166608 (+270172us) spinlock_profiling.cc:243] Waited 900 us on lock 0x7b1800395380. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e522 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:07:40.222227 (+ 55619us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b1800395380. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb74df3060 00007fcb74e05716 00007fcb74e16287 00007fcb74e16208
0709 13:07:40.222364 (+   137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:40.222406 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:40.222447 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.222501 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:40.222592 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26362,"prepare.queue_time_us":591141,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":87,"raft.queue_time_us":18217,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":16382,"spinlock_wait_cycles":14765696,"thread_start_us":531638,"threads_started":2,"wal-append.queue_time_us":281093}
I20260709 13:07:40.225848 25298 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:40.227509 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.231565 23448 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:07:40.232851 23444 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:40.233484 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.234282 22754 raft_consensus.cc:2410] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:40.236158 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.237507 25285 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.233762 25299 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:40.237910 22749 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:40.238152 25285 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.238312 25299 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.239552 25388 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.240115 25285 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.240048 25388 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.240298 25422 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:40.240769 25299 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.240861 25297 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:40.241355 25297 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:40.241816 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 286017us
I20260709 13:07:40.242002 25453 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.242142 25297 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.242486 25453 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.243152 25297 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:40.244459 25453 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.246480 25176 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:40.246948 25176 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.248893 25176 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:40.251809 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.252725 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.253288 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:40.253883 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.253520 22830 raft_consensus.cc:2410] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:07:40.259722 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.260260 22749 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:40.268530 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:40.270220 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.270656 23448 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:40.279897 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:40.280735 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48202.7us
I20260709 13:07:40.281634 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:40.282153 23448 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:40.196897 25346 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:40.295717 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:40.298048 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:40.299098 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.299891 24616 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:40.300405 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:40.301484 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.230876 25304 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.308436 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.302080 22748 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:40.310236 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:40.310616 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.311370 22904 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:40.312165 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.312343 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.312968 23447 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:40.313442 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 835) took 1068 ms. Trace:
I20260709 13:07:40.313726 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.304718 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:40.306754 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.315414 22902 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:40.307430 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.315804 24616 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:40.315729 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.316332 24616 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:40.317909 24616 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.319293 24616 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:40.302891 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.233212 25063 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:40.313741 23447 rpcz_store.cc:276] 0709 13:07:39.245411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.245554 (+   143us) service_pool.cc:224] Handling call
0709 13:07:40.312743 (+1067189us) spinlock_profiling.cc:243] Waited 6.08 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:07:40.312859 (+   116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:40.313218 (+   359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.313388 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13973888}
I20260709 13:07:40.323717 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.323747 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.324779 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.325219 22902 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:40.326030 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:40.326530 22902 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:40.327471 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:40.328843 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:40.313674 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:40.330857 22899 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:40.231402 25464 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:40.234817 25410 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:40.346633 25410 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.348667 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:40.349390 22903 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:40.308347 25453 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:40.352370 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.353394 25312 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:40.354985 24811 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:40.355422 25422 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:40.355458 24811 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.356082 25422 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.357638 24811 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:40.357947 25422 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:40.301936 25291 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:40.321460 25249 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.359601 25249 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.362126 25249 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.335803 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:40.365962 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.336439 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.330705 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.337633 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.337661 22830 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:07:40.338974 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.340049 25334 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:40.305847 25293 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:07:40.342051 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.245049 25455 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:07:40.345757 25465 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:40.492877 22899 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:07:40.427163 22902 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:40.367470 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:40.465590 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 136824us
W20260709 13:07:40.437218 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 101514us
I20260709 13:07:40.468070 23452 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:40.470422 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.436247 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.438274 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.296887 25463 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:40.480332 22828 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:40.420209 23448 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:40.333360 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:40.512324 22756 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:07:40.321122 25301 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.502741 22751 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:07:40.503608 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.504635 22903 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:40.504940 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:40.520421 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.499792 22823 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:40.514299 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:40.303372 25226 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.180s
I20260709 13:07:40.397256 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:40.517378 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.507382 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.531476 23452 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:40.522454 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.349210 25410 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.439581 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.471089 25450 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:40.466480 25249 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:40.501076 25309 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:40.554903 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.562042 23452 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:40.555611 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:40.529978 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:40.564522 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:40.564600 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 175250us
I20260709 13:07:40.565866 22896 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:40.481770 25453 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:40.543592 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.566874 22830 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:40.551112 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.539371 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:40.368750 25300 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.204s
I20260709 13:07:40.326166 25304 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.512059 25369 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.303504 25452 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.208s
I20260709 13:07:40.568527 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:40.511617 25245 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:40.570093 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.570329 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.570767 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.526623 25291 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:40.610634 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:40.611416 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:40.611444 22828 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:40.368300 25063 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.537259 25312 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.527534 25206 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:07:40.554466 25399 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:40.522123 25463 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.605355 23445 raft_consensus.cc:2410] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:07:40.538328 25422 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:07:40.601178 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:40.605813 22829 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:40.615183 25015 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.649470 24616 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.649925 24616 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.577304 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:40.608893 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.646762 22905 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:40.600733 25226 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:40.614656 25388 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.577579 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:40.594491 25451 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:40.537637 25301 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.602732 25176 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:40.677896 25301 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.678613 23449 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:40.678995 25436 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:40.613235 25369 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.621876 25291 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.681401 25095 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:40.681618 25463 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.576720 25249 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:40.600034 25287 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:40.646901 22830 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:40.656894 22828 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:40.683916 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.684496 22752 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:40.529887 25346 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.608259 25245 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.568512 25096 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:07:40.665158 24616 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.563840 25450 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:40.669204 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 91722.7us
I20260709 13:07:40.694321 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:40.619083 25452 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
W20260709 13:07:40.668583 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 91375.3us
I20260709 13:07:40.657982 25210 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:40.622066 25442 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:40.677958 25466 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:07:40.667263 25226 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.552927 25460 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:40.729730 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.595211 25340 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:40.686022 22829 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:07:40.559618 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.593704 25309 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.695919 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:07:40.733117 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 167006us
W20260709 13:07:40.676177 25471 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.368s	user 0.001s	sys 0.000s
I20260709 13:07:40.733984 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:07:40.676623 25475 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:07:40.706413 25346 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:40.623960 25312 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.667411 25451 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:40.728483 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.729714 25334 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:40.713160 25375 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:40.692077 25298 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.656417 25015 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:40.735930 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.737637 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:40.684947 25444 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.760s	user 0.004s	sys 0.013s
I20260709 13:07:40.730147 25455 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.706704 25463 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:40.703063 25176 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:40.735778 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:40.701879 25210 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:40.727584 25393 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:40.769634 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:40.731035 25245 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.732844 25340 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:40.780754 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.740996 25371 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:40.745384 24811 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:40.825079 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.829350 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.868857 22756 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:40.769428 25334 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:40.859553 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 123827us
I20260709 13:07:40.769582 25483 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:40.737105 25460 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.776721 25369 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.841694 25444 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:40.842629 25453 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:40.843739 25463 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:40.839181 25464 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:40.932381 24616 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.854405 25422 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:40.935004 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 64991.7us
I20260709 13:07:40.872360 25334 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:40.881853 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.827781 25298 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.968175 25298 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.966497 24616 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.970275 25442 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.970389 24616 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.970672 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.937250 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:40.971282 22898 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:40.972167 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.972270 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.855485 25176 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:40.972713 22756 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:40.868427 25340 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:40.920233 25451 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.920535 25369 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:40.973759 25488 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:40.976294 25369 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.977046 25488 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.977602 25488 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:40.978588 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.979172 22756 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:40.979945 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:40.980261 25481 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.314s	user 0.003s	sys 0.000s
I20260709 13:07:40.974615 25451 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:40.973299 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.981664 25176 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.982053 22898 raft_consensus.cc:2410] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:07:40.982892 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.983441 22898 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:40.983955 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:40.984254 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.984983 22898 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:40.974740 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:40.985526 25245 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:40.975139 25462 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:07:40.986342 25015 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.986660 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.974532 25463 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.987233 25239 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:40.987608 25463 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:40.987993 22896 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:07:40.988540 25312 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.989517 25226 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.992221 25309 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:40.987635 25239 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.993492 25239 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:07:40.975286 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.997009 22896 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:07:41.007350 22756 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:40.983198 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:40.986361 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49234.9us
I20260709 13:07:40.978533 25369 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.879038 25413 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:41.009367 22753 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:41.014845 25464 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.013239 22750 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:41.042775 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.043617 22750 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:41.017211 25282 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.041695 25454 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:41.018536 25478 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.365s	user 0.000s	sys 0.001s
W20260709 13:07:41.018672 25480 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:07:40.973691 25422 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.019392 25488 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.021085 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.009601 25486 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:41.023669 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.025421 22751 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:07:41.025830 25482 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
I20260709 13:07:41.032058 25330 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:41.034791 25460 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.071552 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:40.870635 25375 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:41.049876 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:41.070371 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 325899us
I20260709 13:07:40.953928 25063 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:40.879225 24811 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.019188 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:41.075618 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56502.1us
I20260709 13:07:41.075783 22905 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:41.076249 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:40.953719 25210 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.055625 25422 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.087920 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.088637 22750 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:40.938042 25410 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:41.095873 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 326266us
I20260709 13:07:41.071671 24616 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:41.101125 24616 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:40.855000 25455 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.070624 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.072118 22896 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:41.048498 25340 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.080067 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:41.082785 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.111181 23447 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:41.111306 25369 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:41.038187 25413 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.103259 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:41.071226 25330 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:41.115095 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.116422 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:41.116707 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.120285 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.120985 22750 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:41.047995 25454 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.124030 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.082566 25463 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:07:41.141973 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 62016.8us
I20260709 13:07:41.083317 25388 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:41.184039 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:41.174429 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 64438.9us
I20260709 13:07:41.174678 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:41.179636 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.179984 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:41.083406 25486 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:41.049297 25453 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.114606 25413 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:41.218645 22825 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:41.185590 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:41.219677 25282 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:41.220239 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:41.222054 25375 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.173952 25450 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.185457 22902 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:41.184460 25490 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:41.194615 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:41.231281 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:41.187321 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:41.200800 22900 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:41.201176 25095 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:41.201848 25455 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.204576 25239 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.205991 25488 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.209532 25330 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.209946 25300 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:41.128363 25293 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:41.211481 24811 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:40.869441 25393 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.211910 25460 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:41.212965 25484 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:41.225998 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:41.269564 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 43616.8us
I20260709 13:07:41.268643 25484 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:41.230322 25422 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:07:41.271178 25484 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.272778 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:41.231869 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:41.232928 25176 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:07:41.233268 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46037.5us
I20260709 13:07:41.274712 22750 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
W20260709 13:07:41.239467 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.250818 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.275298 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:41.254043 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:41.110947 25410 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.184489 25388 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.261453 25291 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:41.264478 25472 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.586s	user 0.001s	sys 0.000s
I20260709 13:07:41.257061 25436 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:41.233376 25298 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:41.276784 22901 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
W20260709 13:07:41.277676 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45890us
I20260709 13:07:41.234018 25463 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 13:07:41.289790 23451 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.000s	sys 0.030s
I20260709 13:07:41.286394 25388 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:41.278321 25282 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:41.306983 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.310431 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.292496 25436 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:41.296151 23451 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.000s	sys 0.031s
I20260709 13:07:41.283960 25330 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.283376 25346 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:41.289721 25176 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:41.293891 25298 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.300261 25282 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:41.325228 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 836) took 1016 ms. Trace:
I20260709 13:07:41.331831 23452 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:07:41.334833 23448 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:41.321242 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.333283 23451 rpcz_store.cc:276] 0709 13:07:40.308985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.309142 (+   157us) service_pool.cc:224] Handling call
0709 13:07:40.309411 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:40.309591 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:40.613382 (+303791us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.324857 (+711475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:41.324907 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:41.324952 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.325035 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.325161 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":368736,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":5985,"thread_start_us":959968,"threads_started":2}
I20260709 13:07:41.300987 25494 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:41.336328 23444 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:41.341631 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.342501 23450 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:41.324319 25436 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.319979 25340 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.293394 25483 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:41.332883 25298 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:41.338735 25282 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:41.344118 25454 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.344452 25494 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.365329 25410 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.340857 25486 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.387044 25483 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.388296 25413 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.376984 25494 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:41.390295 25413 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.393378 25484 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.393898 25393 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.395917 25484 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.438912 22754 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:41.441704 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2274) took 1232 ms. Trace:
W20260709 13:07:41.444041 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.442329 22755 rpcz_store.cc:276] 0709 13:07:40.209024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.209697 (+   673us) service_pool.cc:224] Handling call
0709 13:07:40.210006 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:40.230832 (+ 20826us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:40.237960 (+  7128us) log.cc:844] Serialized 14 byte log entry
0709 13:07:40.683381 (+445421us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:40.689867 (+  6486us) log.cc:844] Serialized 26 byte log entry
0709 13:07:40.690006 (+   139us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:40.690041 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.690101 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:41.263886 (+573785us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:41.440746 (+176860us) raft_consensus.cc:1685] finished
0709 13:07:41.440952 (+   206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.441378 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48830,"prepare.queue_time_us":7489,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":72507,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":27342,"spinlock_wait_cycles":12913920,"thread_start_us":416596,"threads_started":2,"wal-append.queue_time_us":419031}
I20260709 13:07:41.277292 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:41.456034 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2089) took 1149 ms. Trace:
I20260709 13:07:41.456506 22825 rpcz_store.cc:276] 0709 13:07:40.306364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.312015 (+  5651us) service_pool.cc:224] Handling call
0709 13:07:41.455625 (+1143610us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b540043b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:41.455829 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2083632896}
W20260709 13:07:41.458789 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 395516us
I20260709 13:07:41.288980 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:41.461871 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2082) took 1205 ms. Trace:
W20260709 13:07:41.463371 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 174380us
I20260709 13:07:41.464614 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:41.465607 22831 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.011s	sys 0.016s
W20260709 13:07:41.466239 22831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.011s	sys 0.016s
I20260709 13:07:41.462491 22822 rpcz_store.cc:276] 0709 13:07:40.244239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.245182 (+   943us) service_pool.cc:224] Handling call
0709 13:07:40.247629 (+  2447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:40.247877 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:07:40.248151 (+   274us) log.cc:844] Serialized 14 byte log entry
0709 13:07:40.320778 (+ 72627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:40.327428 (+  6650us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:40.368866 (+ 41438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.448648 (+1079782us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18004639c0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419
0709 13:07:41.448789 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.449105 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59349,"prepare.queue_time_us":559,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":23,"raft.queue_time_us":26674,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":41964,"spinlock_wait_cycles":25362048,"thread_start_us":45739,"threads_started":1,"wal-append.queue_time_us":46084,"wal-append.run_cpu_time_us":1130,"wal-append.run_wall_time_us":1003189}
I20260709 13:07:41.479352 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 639) took 1224 ms. Trace:
W20260709 13:07:41.448874 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 171522us
I20260709 13:07:41.479662 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.480952 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:41.482798 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.486194 22754 rpcz_store.cc:276] 0709 13:07:40.255015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.258168 (+  3153us) service_pool.cc:224] Handling call
0709 13:07:40.258403 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.440006 (+1181603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.478998 (+ 38992us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400045954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:41.479279 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2713423744}
I20260709 13:07:41.487955 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.491119 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:41.518373 25239 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.492475 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.495427 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.497136 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.499745 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 1789) took 1179 ms. Trace:
W20260709 13:07:41.536832 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 218720us
I20260709 13:07:41.501291 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 564) took 1138 ms. Trace:
I20260709 13:07:41.502014 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 567) took 1138 ms. Trace:
W20260709 13:07:41.502539 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 288028us
I20260709 13:07:41.505692 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 485) took 1274 ms. Trace:
I20260709 13:07:41.508019 24616 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.510033 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.511307 25015 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:41.512502 25096 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.323309 24811 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:07:41.513562 25206 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:41.514593 25176 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.517776 25293 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:41.518304 25309 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:41.519647 25369 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:41.520645 25245 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:41.545975 22900 rpcz_store.cc:276] 0709 13:07:40.319986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.780431 (+460445us) service_pool.cc:224] Handling call
0709 13:07:41.499183 (+718752us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400107554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:41.499421 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786996480}
I20260709 13:07:41.546343 22901 rpcz_store.cc:276] 0709 13:07:40.362908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.826505 (+463597us) service_pool.cc:224] Handling call
0709 13:07:41.500822 (+674317us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b5400275454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:41.501053 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938202880}
W20260709 13:07:41.562186 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 71374.8us
I20260709 13:07:41.563191 25401 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:41.565874 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.566689 25455 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:41.567296 25455 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:41.567860 25455 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:41.569554 25484 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:41.554029 25309 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.570148 25484 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:41.570540 25309 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:41.570672 25484 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:41.546875 22904 rpcz_store.cc:276] 0709 13:07:40.363838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.999697 (+635859us) service_pool.cc:224] Handling call
0709 13:07:41.000015 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.000292 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:41.302956 (+302664us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.501177 (+198221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:41.501299 (+   122us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:41.501427 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.501595 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.501817 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":375539,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":15631,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":282,"thread_start_us":499982,"threads_started":2,"wal-append.queue_time_us":78092}
I20260709 13:07:41.574203 23446 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:41.574399 25015 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.554847 25245 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:41.576766 25245 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.578871 25245 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:41.583117 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.376119 25436 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.552405 25206 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:41.588279 25436 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.588785 25206 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.554929 25369 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.592993 25304 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.598754 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.600956 25095 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:41.548579 23445 rpcz_store.cc:276] 0709 13:07:40.230588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.233040 (+  2452us) service_pool.cc:224] Handling call
0709 13:07:41.505077 (+1272037us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b54000b2cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:41.505380 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13295488}
I20260709 13:07:41.596215 25369 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:41.603632 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:41.604271 25442 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:41.604699 22825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:41.608884 22825 raft_consensus.cc:606] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:07:41.618273 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.608645 25442 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:41.619451 25206 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:41.599223 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.607995 25015 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.636636 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:41.643652 23451 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:07:41.645920 25465 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.645967 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:41.651175 25297 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.651791 25297 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.652374 25293 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:41.652855 25293 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:41.646602 25465 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.653468 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.654313 22751 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:41.655495 25465 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.620134 25442 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.657560 25442 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:41.661163 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 416) took 1119 ms. Trace:
I20260709 13:07:41.661635 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:41.661402 25293 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:41.662189 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.642539 22822 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:41.662690 25451 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:07:41.661644 22823 rpcz_store.cc:276] 0709 13:07:40.502231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.611660 (+109429us) service_pool.cc:224] Handling call
0709 13:07:40.622852 (+ 11192us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:40.689692 (+ 66840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:40.689738 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:40.690216 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:07:41.214853 (+524637us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b180044e5a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:07:41.600498 (+385645us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:41.600596 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.601347 (+   751us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:41.606781 (+  5434us) raft_consensus.cc:1685] finished
0709 13:07:41.606803 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.621724 (+ 14921us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65987,"raft.queue_time_us":66325,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":125,"spinlock_wait_cycles":6268160,"thread_start_us":358659,"threads_started":1,"wal-append.queue_time_us":878556}
I20260709 13:07:41.672240 25435 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:41.672634 25297 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:41.672755 25435 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:41.673839 25388 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.674365 25435 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.675685 25435 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.676154 25388 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.401258 25494 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.690619 25369 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.689711 25096 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:07:41.690362 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 78280.5us
I20260709 13:07:41.691987 25309 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.692493 25485 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:07:41.692726 25494 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.693424 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:41.689411 25465 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:41.700151 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.694815 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.695423 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:41.703225 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 95370.3us
I20260709 13:07:41.705731 25282 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.707650 25309 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.707809 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.696587 25369 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.709205 25293 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.709708 25282 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.710685 22755 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:41.721580 23453 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:41.724274 25309 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:41.725133 25346 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:41.726028 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62985.6us
I20260709 13:07:41.725936 25346 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.726809 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:41.727417 24616 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.727967 24616 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.728253 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:41.728806 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:41.729533 22897 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:41.730202 24616 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:41.718699 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:41.734040 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:41.734614 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:41.735127 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.735383 22896 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:41.736258 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:41.739842 25401 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:41.740442 25401 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:41.741734 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:41.742741 25282 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:41.743155 25282 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.743495 25282 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:41.745985 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.747123 25309 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:41.747592 25309 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:41.748026 25309 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:41.748310 25455 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.751174 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.752056 22751 raft_consensus.cc:2410] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:41.734197 22750 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:41.753014 25465 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:41.753459 25465 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:41.753804 25465 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:41.758404 25334 consensus_meta.cc:287] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.718s	user 0.004s	sys 0.011s
I20260709 13:07:41.758891 25334 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.760681 22896 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:41.761201 22901 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
W20260709 13:07:41.764458 25525 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 13:07:41.728274 25346 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.767335 25282 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.770298 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.770951 23453 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:41.773305 22897 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:41.773394 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.773849 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.774384 23451 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:41.774777 25282 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.775230 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:41.776664 23446 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:07:41.778743 25401 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:41.779567 25528 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:41.780027 22753 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:41.780902 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:41.778065 25309 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.784930 23446 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:41.789769 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.790517 23453 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:41.791205 25401 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.793911 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:41.791740 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:41.794328 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 84034us
I20260709 13:07:41.794672 22898 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:41.795053 22896 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:41.795243 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:41.760864 25334 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:41.796625 25526 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:07:41.796804 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:41.794039 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.797518 25297 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:41.798327 25297 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:41.800278 23453 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:41.800750 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:41.802196 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 65184.1us
I20260709 13:07:41.795185 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.798887 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:41.802750 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 59379.9us
I20260709 13:07:41.768690 25298 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.815081 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.803681 25451 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:41.784942 25309 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:41.824926 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:41.821672 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.827617 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.825794 22897 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:07:41.832626 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.823640 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:41.837037 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.822774 25334 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:41.843458 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:41.846686 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50006.2us
I20260709 13:07:41.847865 25210 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.848136 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:41.853680 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.854576 25298 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.855254 22823 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:07:41.855914 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:41.857059 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:41.858069 22900 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:41.847901 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:41.861405 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 44646.2us
I20260709 13:07:41.860271 22829 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:41.840199 25435 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:41.870127 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.874209 22898 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:41.874405 25451 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:41.874976 22822 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:41.874472 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.879535 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:41.880183 23446 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:41.874213 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:41.883782 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:41.884563 25297 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:41.885111 25297 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:41.885669 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:41.885798 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:41.890304 22903 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:41.858944 25455 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:41.841092 25334 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:41.896219 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:41.897045 23446 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:41.898037 22900 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:41.911126 22831 raft_consensus.cc:2368] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:41.913144 25297 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.914754 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:41.919919 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:41.861065 25346 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:07:41.920680 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 52519.3us
I20260709 13:07:41.868386 25485 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:41.869755 25435 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.923710 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:41.891340 25245 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:07:41.929610 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.934563 23453 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:41.909198 25334 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:41.941682 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57984.6us
W20260709 13:07:41.944245 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 62967.7us
I20260709 13:07:41.947779 24811 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:41.947775 25510 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:41.949301 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:41.950912 25484 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.953459 22750 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:41.954099 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38478 (request call id 2280) took 1690 ms. Trace:
I20260709 13:07:41.954607 22747 rpcz_store.cc:276] 0709 13:07:40.263810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.320530 (+ 56720us) service_pool.cc:224] Handling call
0709 13:07:41.952687 (+1632157us) spinlock_profiling.cc:243] Waited 8.9 ms on lock 0x7b54000cf1a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d72dc0 00007fcb74dfee18 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6
0709 13:07:41.953718 (+  1031us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54003cebd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:41.953877 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3139745920}
I20260709 13:07:41.960659 23446 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:41.932523 25435 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:41.948328 24811 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.945699 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:41.932085 25245 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.988129 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:41.989445 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:42.022706 23447 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:41.987955 25346 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:41.990483 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:41.991554 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:41.995646 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:42.000638 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 75244.5us
I20260709 13:07:42.002427 25510 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:42.003026 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:42.007501 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:42.000175 25465 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:41.936563 24616 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:42.017733 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 72661.1us
I20260709 13:07:42.021347 24811 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:42.045387 25346 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:42.045727 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:42.045984 25490 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:07:42.046453 23448 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:07:42.034425 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.022953 25245 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:42.041702 25510 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.042526 23446 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:42.042699 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:42.043687 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:42.040357 23453 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:42.055028 23451 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:42.051548 22750 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:42.047253 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:42.058807 22824 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:42.055891 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:42.032694 25484 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.060187 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:42.043704 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:42.062891 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.063184 23452 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:42.062464 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:42.063884 22831 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:42.066037 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:42.070014 22829 pending_rounds.cc:85] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 2
I20260709 13:07:42.075138 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:42.074183 22829 pending_rounds.cc:107] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:07:42.080273 22829 raft_consensus.cc:2889] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:42.080535 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.081516 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:42.081984 23451 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:42.082307 22747 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:42.083971 25312 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:42.085515 25300 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:07:42.086105 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:42.086609 25300 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:42.086813 22830 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:42.087016 25300 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:42.085212 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.087891 25300 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.089031 25304 raft_consensus.cc:2764] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:07:42.094034 25300 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:42.095947 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:42.096307 25435 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:42.098682 25532 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:07:42.103171 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.077929 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:42.103713 24616 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:42.111505 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 70439us
I20260709 13:07:42.116590 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:42.122241 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 89333.7us
I20260709 13:07:42.126713 23448 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:07:42.074608 25490 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:07:42.127169 23448 pending_rounds.cc:85] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 7
I20260709 13:07:42.133193 23448 pending_rounds.cc:107] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:07:42.133510 23448 raft_consensus.cc:2889] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:42.132037 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:42.135380 22747 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:42.139976 22826 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:42.132507 25346 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:42.146224 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:42.146588 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:42.146919 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:42.148871 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:42.149116 22830 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:07:42.154093 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:42.163794 22826 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:42.158411 24616 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:42.170307 22902 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 4: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:42.158015 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:42.171453 22750 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:07:42.172717 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 54508.7us
I20260709 13:07:42.172881 22900 raft_consensus.cc:2368] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7.
I20260709 13:07:42.166326 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:42.151316 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:42.156678 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:42.176504 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 80681.6us
I20260709 13:07:42.150900 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:42.163339 25535 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:07:42.180155 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:42.181006 22747 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:42.185961 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.149189 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:42.188437 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:42.190176 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:42.178423 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.192873 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:42.196762 22750 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:42.182931 25282 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:42.179682 25455 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:42.185238 25239 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.192826 22905 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:42.191916 25484 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:42.211352 22904 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:42.185469 25465 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:42.188952 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 51579.7us
W20260709 13:07:42.222725 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46364us
I20260709 13:07:42.227242 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:42.220721 22898 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:42.231681 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:42.210815 25346 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.220574 25282 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:42.214035 25304 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:42.245101 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:42.222172 25210 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:42.253564 22751 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:42.236691 25484 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:42.252085 25537 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:07:42.227751 25455 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.259093 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:42.247223 25435 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.248209 25510 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:42.268285 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:42.225188 25239 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.260497 22755 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:42.240974 25346 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:42.273397 25536 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:07:42.273542 25330 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:42.274066 25330 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:42.274891 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.274979 25330 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:42.277719 22752 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:07:42.279999 25533 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.241s	user 0.000s	sys 0.002s
I20260709 13:07:42.281637 25282 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.284732 25301 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:42.260360 25176 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:42.258445 25210 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:42.287988 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:42.267303 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 93843.1us
W20260709 13:07:42.288589 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 67282.6us
I20260709 13:07:42.289893 25176 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.289949 25334 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.291181 25484 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:42.300115 25435 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:42.251113 24811 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:42.309083 22752 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:42.296823 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:42.337957 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 81330.6us
W20260709 13:07:42.342048 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 54150.1us
I20260709 13:07:42.342363 22754 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:42.337684 25485 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:42.345677 25485 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:42.349833 24811 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:42.350811 24616 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.351307 22751 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:07:42.352151 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:42.352715 23449 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:42.356531 25485 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.351303 24616 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.359992 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:42.360826 22751 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:42.360885 23449 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
W20260709 13:07:42.363914 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 134336us
W20260709 13:07:42.364027 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 67354.2us
I20260709 13:07:42.365902 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.364894 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.345963 25096 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:42.375111 24616 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:42.380858 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:42.393285 24811 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:42.396479 22752 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:42.378947 25401 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:42.392586 25465 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:42.415073 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:42.427893 25176 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.428493 25176 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.428987 25176 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.431428 25176 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.412324 25210 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:42.434931 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 91203.7us
I20260709 13:07:42.415895 25455 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.437732 25176 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:07:42.456274 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:42.411919 25301 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 13:07:42.459517 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 118955us
W20260709 13:07:42.463027 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 64647.1us
I20260709 13:07:42.464975 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:42.407660 25282 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:07:42.469555 22747 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:42.459285 25297 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:42.461658 25239 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:42.501957 22719 leader_election.cc:400] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7.
I20260709 13:07:42.502522 22719 leader_election.cc:403] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:07:42.503383 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46570.2us
I20260709 13:07:42.464218 25334 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.464530 25436 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:42.506291 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:42.512040 25552 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:07:42.512439 22900 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:07:42.513218 25547 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:42.513690 25547 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:42.514039 25547 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:42.517968 25548 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:07:42.523545 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88972.9us
I20260709 13:07:42.472854 25297 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.535380 22751 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:42.546025 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:42.545722 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:42.549022 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.549414 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:42.550199 22900 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
W20260709 13:07:42.552019 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 45778.4us
I20260709 13:07:42.552686 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.525941 25401 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7.
I20260709 13:07:42.554760 22899 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:42.526747 25485 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.557760 22902 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:42.544981 25297 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:42.568581 25370 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:42.569141 22899 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:42.570572 25346 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.571097 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:42.571271 22828 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:07:42.571779 23448 raft_consensus.cc:2368] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7.
I20260709 13:07:42.559409 25455 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:42.573874 25451 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:42.574278 25451 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:42.575142 25451 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.576148 25451 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:42.548060 25334 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:07:42.582294 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:42.584942 22828 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:42.586220 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:42.586625 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:42.589900 25551 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:42.590675 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.590891 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:42.591992 25455 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:42.592231 25555 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:42.603103 25455 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:42.609452 25544 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.112s	user 0.003s	sys 0.000s
I20260709 13:07:42.611765 25370 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:42.612282 25370 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:42.612080 25435 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:42.617256 25239 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:42.618772 25239 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:42.619067 25550 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:42.620031 25550 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:42.583563 25245 consensus_meta.cc:287] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.000s
I20260709 13:07:42.621708 25245 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.622159 25554 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.622833 25554 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.623823 25245 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.621531 25551 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:42.626507 25551 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.619217 25239 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:42.630537 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.630293 25239 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.630851 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:42.631464 25239 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:42.631810 25249 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:42.630569 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:42.632696 22828 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:07:42.620595 25550 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.634306 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:42.634859 22825 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:42.635231 25550 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.625157 25554 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:42.617981 25435 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:42.636498 25435 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:42.637401 25435 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.641551 25435 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:42.596957 25304 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 13:07:42.642585 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.642445 25304 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:42.643167 25304 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:42.643299 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:42.644014 25304 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:42.631659 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.593451 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:42.649005 22828 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:42.617254 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.645195 25304 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:42.649960 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:42.654297 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 56053.1us
I20260709 13:07:42.655133 22871 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:42.650585 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 57241.4us
I20260709 13:07:42.657516 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.658650 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:42.659230 23448 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:42.659698 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.660944 25555 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:42.661547 25485 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:42.662168 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.662979 25555 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:42.664157 25555 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:42.666055 25555 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:42.668655 22719 leader_election.cc:400] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7.
I20260709 13:07:42.668968 25096 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:42.678978 25096 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:42.679345 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 577) took 1577 ms. Trace:
I20260709 13:07:42.679821 22905 rpcz_store.cc:276] 0709 13:07:41.102174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.102617 (+   443us) service_pool.cc:224] Handling call
0709 13:07:42.679063 (+1576446us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b54001b6a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:42.679246 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285978496}
I20260709 13:07:42.679914 25096 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.681059 25096 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:42.681628 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:42.683506 25300 consensus_meta.cc:287] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.003s
W20260709 13:07:42.683645 25540 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.411s	user 0.000s	sys 0.001s
I20260709 13:07:42.684358 22754 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:07:42.660997 25330 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:42.691813 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:42.654001 25334 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:42.684690 22754 pending_rounds.cc:85] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 5
I20260709 13:07:42.700258 24616 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:42.705552 22754 pending_rounds.cc:107] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:07:42.706200 22754 raft_consensus.cc:2889] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:42.675575 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.705009 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:42.705543 22871 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:42.716207 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:42.660188 25301 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:42.717504 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:42.746289 24616 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:42.746716 24616 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:42.747538 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 130402us
W20260709 13:07:42.716590 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:42.750015 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 74524.8us
I20260709 13:07:42.691778 25485 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:42.753201 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.758214 25553 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:42.759153 25299 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:42.759317 25490 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:42.759662 25299 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:42.761585 25553 raft_consensus.cc:993] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:42.763005 25245 raft_consensus.cc:1081] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:07:42.765224 22652 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.767531 22652 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 4 to 6, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 6 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.769464 25453 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:42.770561 25550 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:42.770990 25550 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:42.771081 23449 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:07:42.771399 25550 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:42.715081 25554 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:07:42.778260 25554 consensus_queue.cc:1243] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:07:42.778790 25554 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:42.779137 25554 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:42.779986 25554 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:42.781044 25554 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:42.708573 25334 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.769686 22652 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:42.799834 23448 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
W20260709 13:07:42.801646 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50213.4us
I20260709 13:07:42.798478 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:42.803330 22753 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:42.808307 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:42.797680 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:42.810842 22748 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:42.696190 25346 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:07:42.812863 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:42.813176 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:42.813980 24616 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.804350 25334 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:42.840250 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.749164 25301 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.760584 25299 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:42.845432 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 139953us
I20260709 13:07:42.850766 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:42.854633 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:42.839053 25453 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:07:42.856281 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 95071.8us
W20260709 13:07:42.857192 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44084.7us
W20260709 13:07:42.860672 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 117604us
I20260709 13:07:42.841959 25176 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:42.859440 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.860498 24616 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:42.843091 25300 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.865780 25300 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.867825 25300 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:42.864041 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.843222 25301 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:42.869884 22654 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 331) took 13908 ms (client timeout 15000 ms). Trace:
I20260709 13:07:42.856594 25330 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:42.877148 25330 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:42.877899 25435 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:42.878180 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.878935 25245 raft_consensus.cc:2764] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:07:42.879647 25245 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:42.880385 25301 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.881470 25484 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:07:42.882754 25301 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:42.839221 25547 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.888162 25547 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:42.870628 22654 rpcz_store.cc:269] 0709 13:07:28.961312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.963417 (+  2105us) service_pool.cc:224] Handling call
0709 13:07:42.869635 (+13906218us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:39.872303 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:39.921799 (+ 49496us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:39.921837 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:07:39.921851 (+    14us) tablet.cc:662] Decoding operations
0709 13:07:39.923156 (+  1305us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:39.923211 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:39.923238 (+    27us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:07:39.923839 (+   601us) tablet.cc:701] Row locks acquired
0709 13:07:39.923860 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:07:39.923992 (+   132us) write_op.cc:270] Start()
0709 13:07:39.924099 (+   107us) write_op.cc:276] Timestamp: P: 1783602459923947 usec, L: 0
0709 13:07:39.924117 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:07:39.924657 (+   540us) log.cc:844] Serialized 3214 byte log entry
0709 13:07:41.333160 (+1408503us) op_driver.cc:464] REPLICATION: finished
0709 13:07:41.412850 (+ 79690us) write_op.cc:301] APPLY: starting
0709 13:07:41.412909 (+    59us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:41.413044 (+   135us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:41.413124 (+    80us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:41.781723 (+368599us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:41.782024 (+   301us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:07:41.782069 (+    45us) write_op.cc:312] APPLY: finished
0709 13:07:41.782471 (+   402us) log.cc:844] Serialized 65 byte log entry
0709 13:07:41.782687 (+   216us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:41.782885 (+   198us) write_op.cc:454] Released schema lock
0709 13:07:41.782942 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:41.783405 (+   463us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:41.783431 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3098197,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":74241,"apply.run_cpu_time_us":4420,"apply.run_wall_time_us":370473,"num_ops":8,"prepare.queue_time_us":744193,"prepare.run_cpu_time_us":20318,"prepare.run_wall_time_us":300820,"raft.queue_time_us":449361,"raft.run_cpu_time_us":9617,"raft.run_wall_time_us":271591,"replication_time_us":1393304,"thread_start_us":904116,"threads_started":4,"wal-append.queue_time_us":372015,"wal-append.run_cpu_time_us":35281,"wal-append.run_wall_time_us":2469663}]]}
I20260709 13:07:42.894549 25300 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:42.894793 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:42.864410 25401 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:07:42.900588 25546 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:42.901662 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.902661 25490 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:42.902789 25312 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:42.903234 25312 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:42.903615 25245 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:42.879619 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.904165 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:42.904094 25312 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.904994 25490 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:42.906729 22652 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:42.907898 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:42.909024 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:42.909449 25435 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:07:42.860503 25538 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.575s	user 0.004s	sys 0.004s
I20260709 13:07:42.908547 22652 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 4, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 4 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:42.917801 22748 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:42.919342 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:42.861704 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.862911 25176 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.916344 25547 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:07:42.934360 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72720.6us
I20260709 13:07:42.940133 22751 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:42.943209 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:42.943964 22899 raft_consensus.cc:2410] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:07:42.881225 25370 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.925884 25176 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:42.942525 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:42.925527 25096 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:42.931254 22652 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 4 to 5, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 5 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:42.931668 25547 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.953437 25485 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.955219 25547 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:42.941818 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:42.954344 22652 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.944938 25210 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:42.959247 25435 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:42.959944 25451 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:42.972301 25451 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:07:42.972652 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.972635 25451 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:42.973436 25569 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.973907 25569 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.974015 22652 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 1 to 3, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:42.974970 25297 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.976115 23447 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:42.975725 22652 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 4, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 4 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:42.977583 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:42.978261 25210 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:42.961963 25568 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:42.963289 25422 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:42.979117 25422 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.961589 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 582) took 1116 ms. Trace:
I20260709 13:07:42.979508 25422 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:42.979625 22896 rpcz_store.cc:276] 0709 13:07:41.845592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.860169 (+ 14577us) service_pool.cc:224] Handling call
0709 13:07:41.886925 (+ 26756us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:41.990094 (+103169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:41.990137 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:41.990880 (+   743us) log.cc:844] Serialized 52 byte log entry
0709 13:07:42.115632 (+124752us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:42.122995 (+  7363us) log.cc:844] Serialized 14 byte log entry
0709 13:07:42.123403 (+   408us) log.cc:844] Serialized 14 byte log entry
0709 13:07:42.123581 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.123675 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:42.721763 (+598088us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:42.961300 (+239537us) raft_consensus.cc:1685] finished
0709 13:07:42.961397 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.961515 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2244,"prepare.queue_time_us":81199,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":7519,"raft.queue_time_us":54709,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":5970,"spinlock_wait_cycles":31534976,"thread_start_us":124314,"threads_started":1,"wal-append.queue_time_us":238525}
I20260709 13:07:42.982597 25451 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:42.983747 25451 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:42.978140 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:42.955384 25485 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:42.993813 25485 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:42.978788 25550 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:42.976662 25569 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.004663 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:43.011906 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:43.012157 22751 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:43.013113 22828 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:43.025074 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:42.977072 25297 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:43.034284 22828 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:43.034591 22747 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:07:43.042436 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 162866us
I20260709 13:07:43.041967 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:43.045253 22749 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:43.047955 22652 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 3 to 4, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 4 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:43.050107 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:43.050819 23448 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:43.048250 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:43.047184 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 187819us
I20260709 13:07:43.054450 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:43.055063 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:43.041961 25550 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:43.063735 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:43.064729 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:43.061089 25551 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:43.065661 25554 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:43.065951 25551 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:43.066346 25551 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:43.066715 25551 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:43.057215 25571 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:43.068286 25571 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.068607 25571 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:43.057585 25547 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.091609 22753 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:43.099429 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:43.099691 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:43.103500 23446 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:07:43.072202 25299 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:43.100831 25453 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.108562 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:43.108573 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:43.109793 22899 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:43.110347 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:43.061794 22652 catalog_manager.cc:5697] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 5, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66), now has a pending config: VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65), VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66), VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67), NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 5 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:07:43.118393 23416 leader_election.cc:400] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:43.118695 23416 leader_election.cc:403] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:43.103132 25547 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:43.120926 23447 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:43.059561 25568 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:43.104566 25334 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.125396 24616 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:43.119900 22652 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 4 to 5, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 5 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:43.124074 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:43.131938 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:43.132598 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:43.133138 22901 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:43.133988 22904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43998
I20260709 13:07:43.134588 22904 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:43.134922 22904 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:43.136091 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:43.135975 25422 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.117238 25096 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:43.136698 23450 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:43.137207 22905 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:43.137648 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:43.134097 25334 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.141711 23447 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:43.143690 22900 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:43.146598 22904 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:43.147881 22897 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:43.154238 22829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:43.154639 22903 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:43.154754 22829 raft_consensus.cc:606] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:43.155167 22829 raft_consensus.cc:614] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:43.155978 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:43.157514 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:43.117357 25579 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:07:43.161908 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:43.162485 22822 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:43.164491 25312 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.163653 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:43.170905 22899 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:43.174157 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:43.128543 25578 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:43.109479 25453 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:43.166517 25370 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.119477 25484 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:07:43.119249 25574 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:07:43.183156 25571 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:43.184474 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:43.184760 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:43.185227 22831 raft_consensus.cc:2410] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:07:43.186414 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:43.186875 22823 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:43.189677 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:43.193262 25176 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.193789 25568 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.195081 22896 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:43.194283 22825 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:43.198309 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:43.200752 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:43.203586 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:43.206791 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:43.209947 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.212107 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:43.193274 22823 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:43.212955 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:43.185087 25571 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:43.233675 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:43.235064 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:43.236310 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:43.238003 24616 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:43.238709 24616 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:43.240192 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:43.273559 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 144751us
W20260709 13:07:43.297991 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 65723us
I20260709 13:07:43.298763 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:43.297863 22831 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:43.237306 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:43.308223 22823 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:07:43.308768 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 71521.8us
I20260709 13:07:43.311465 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:43.313086 23450 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:43.314128 24616 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:43.212580 25546 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:43.228518 25490 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:43.318142 22903 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:07:43.232167 25510 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:43.231745 25370 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:43.234647 25547 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:43.320914 22903 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:07:43.319813 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:43.322921 23444 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:43.308959 25553 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:43.327250 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:07:43.327486 25486 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:07:43.328584 25486 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:43.328984 25486 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:43.329782 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:43.298838 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:43.335213 25176 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.345991 23449 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:43.314127 25555 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:43.333797 25422 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.350477 22903 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:43.240948 25569 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:07:43.354012 23451 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:43.317838 25245 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.320065 25453 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:43.323999 25370 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.321487 25510 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:43.313863 25573 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 13:07:43.393127 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:43.357272 23446 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:43.366864 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:43.397639 23452 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:43.403374 22905 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:07:43.321614 25554 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:07:43.408540 25486 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.409849 25486 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:43.371265 25245 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.415213 25245 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.373406 25495 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:07:43.418021 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:43.425678 23444 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:43.428298 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:43.393550 25370 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:43.397066 25510 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:43.403999 25555 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:43.446242 23451 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:43.443799 25547 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.450233 25547 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.452215 25547 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:43.458158 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:43.462448 23446 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:43.437419 25484 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:43.466370 25484 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:43.467338 25484 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.373546 25553 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:43.373893 25578 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:43.470739 25578 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.478421 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:43.479219 22747 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:43.481894 22748 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:43.476864 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:43.483846 22753 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:43.489653 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:43.491218 25490 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:43.491683 25490 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:43.492457 25490 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:43.494737 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 175976us
I20260709 13:07:43.501344 22831 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:43.495672 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:43.503088 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:43.503921 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:43.506544 22753 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:43.512601 25594 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:43.513494 25593 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:43.513478 25554 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:43.514400 25554 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:43.514762 25554 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:43.519542 22904 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:43.521512 25568 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:43.513434 25299 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.524704 25299 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.444566 25571 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:43.528975 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:43.437963 25551 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.539546 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:43.555713 25593 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:43.523483 25555 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.544706 25297 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:43.528644 25422 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:07:43.528287 25330 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:07:43.535351 25571 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.544835 25300 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:07:43.604835 22747 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:43.561971 25304 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:43.527830 25597 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:07:43.612273 25584 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:07:43.615481 25299 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.617834 22904 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:43.616020 25370 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.599947 25297 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:43.626843 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 97984.9us
I20260709 13:07:43.644341 22747 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:43.647284 22905 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 2.
I20260709 13:07:43.635241 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:43.635604 25577 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.605s	user 0.002s	sys 0.003s
I20260709 13:07:43.631264 25304 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:43.671347 25451 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:43.692090 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:43.693086 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:43.701844 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:43.721338 23449 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:43.703125 25346 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:43.704275 23444 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:43.708384 22871 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:43.727106 23446 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
W20260709 13:07:43.738436 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 179835us
I20260709 13:07:43.646522 25594 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:07:43.720157 25451 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:43.751973 22904 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:43.734004 25602 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:43.771313 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:43.772961 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:43.774376 22748 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:43.775774 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:43.778510 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:43.779491 22754 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:43.782423 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:43.748152 24811 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:43.747692 25555 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:43.754689 25551 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.800887 22822 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:07:43.805294 22825 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:43.805456 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:43.729656 25346 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.646191 25553 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:43.774181 25560 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.746070 25601 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:07:43.828402 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:43.829340 22822 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:43.784994 25594 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:43.794083 25490 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:07:43.830658 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 81699.1us
I20260709 13:07:43.832695 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:43.833240 22825 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:43.831398 25605 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.803633 24811 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.781129 25584 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:07:43.831904 25490 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.799765 25304 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:43.848613 25584 consensus_queue.cc:1243] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:07:43.859181 23444 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:43.853161 25490 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:43.849494 25555 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.800361 25297 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.866648 22749 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:43.868933 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:43.853399 25370 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.828126 25451 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.869719 23444 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:43.865255 24811 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.838451 25605 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.879343 25605 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:43.831547 25560 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.851862 25510 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.882860 25510 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:43.883788 23447 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:43.845386 25594 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.891206 25554 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.896692 25346 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.868759 25571 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.901736 25560 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:43.928606 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:43.902683 25608 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:07:43.903568 25599 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 13:07:43.910604 25600 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:07:43.911507 25245 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:43.865888 25555 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.899585 25490 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:43.922849 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:43.922806 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:43.924244 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:43.914701 25578 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:43.937675 22829 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:44.150955 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:44.215405 25451 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:44.216377 22897 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:44.217247 25304 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:43.950431 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:44.225849 22751 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:44.226456 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.225901 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.228747 23451 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:44.227305 23444 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:44.157518 25485 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:44.231127 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:44.082528 25594 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.233801 25603 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:44.225200 25249 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.235469 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:44.236395 23448 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:44.237532 25096 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:44.238076 25096 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:44.237970 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:44.241138 22823 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:07:44.230161 25304 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.213739 25245 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:44.065477 25622 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:43.938581 25602 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:44.056248 25616 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.246610 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:44.154461 25554 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.253405 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:44.253407 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:44.255321 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.257827 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:44.256853 25620 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.288s	user 0.005s	sys 0.000s
I20260709 13:07:44.238924 25422 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:07:44.256984 25598 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:44.264771 22897 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:44.265520 25604 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:44.239603 25485 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:44.249096 25495 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:07:44.257189 25346 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:44.272140 25346 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.272541 25346 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:44.273527 25570 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:44.273952 25570 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:44.274329 25570 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:44.252123 25622 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.276394 25614 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:44.276793 25614 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:44.277138 25614 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:44.256634 25616 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:44.277964 25622 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.279119 25584 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:44.280423 25584 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:44.234306 25603 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.250499 25568 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:07:44.266186 25245 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.323413 25245 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:44.324031 25346 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.241211 25249 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.279531 25616 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:44.301992 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:44.327105 25249 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.301849 25604 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:44.327899 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.328495 23444 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:44.330852 25176 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:44.331480 25176 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:44.332623 25573 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:44.333029 25573 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:44.333841 25573 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.302520 25304 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.302775 25485 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:44.304049 25603 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:44.328490 25604 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:44.306881 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:44.335430 25604 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.335647 22754 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:44.335834 25604 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:44.308547 25602 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:44.337504 25346 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.311761 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.312704 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:44.312883 22755 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:44.360143 23446 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:44.312894 25614 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.314179 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:44.301664 25584 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:44.365945 25584 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:44.366237 25614 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.367055 25584 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.368440 25584 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:44.366032 22752 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:44.314467 25553 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.372650 25553 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:44.374800 25553 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:44.376121 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.379835 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:44.380383 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:44.381021 22755 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:44.383636 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:44.383920 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:44.281738 25096 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.320480 25623 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:44.292424 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:44.385583 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:44.337817 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:44.310285 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:44.390405 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 52665.5us
I20260709 13:07:44.392827 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:44.340020 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:44.341651 22899 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:44.345821 25604 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.346419 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.346896 25578 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:44.351934 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 867) took 1112 ms. Trace:
I20260709 13:07:44.353833 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.313205 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.334686 25176 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:44.339417 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:44.402140 23444 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:44.398492 23447 rpcz_store.cc:276] 0709 13:07:43.239731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.311300 (+ 71569us) service_pool.cc:224] Handling call
0709 13:07:44.351768 (+1040468us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b5400620f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:44.351876 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314967808}
I20260709 13:07:44.406502 22824 pending_rounds.cc:85] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 3
I20260709 13:07:44.406929 22824 pending_rounds.cc:107] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:07:44.406540 23448 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:44.410036 22823 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:44.410188 22752 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:44.409986 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:44.412881 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.413167 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.413602 22831 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:44.383018 25603 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.407233 22824 raft_consensus.cc:2889] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:44.420519 22822 raft_consensus.cc:2368] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6.
I20260709 13:07:44.391501 25623 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:44.400947 25553 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:44.424149 22748 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:44.426913 23444 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:44.398420 25578 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:07:44.428607 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51678.7us
I20260709 13:07:44.429548 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:44.431824 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:44.440783 22823 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:07:44.443348 22823 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:44.443122 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:44.424811 24811 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.450001 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.450521 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:44.450836 22831 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:44.451537 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:44.445225 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:44.446074 25570 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.441753 25553 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:44.448649 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.449193 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.434883 25484 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:44.423818 25490 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:44.452329 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:44.424067 25603 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.454457 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:44.444614 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.457871 25551 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:44.458235 25249 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:44.460253 25578 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.462266 25604 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.456449 22830 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:44.468104 25485 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:44.467540 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.492102 22831 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:44.469614 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:44.472873 22902 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:44.453958 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:44.453399 25299 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:44.473753 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:44.477708 25602 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:44.456054 24811 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.455947 25210 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:44.438807 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:44.482340 25346 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:44.495610 22829 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:44.504446 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:44.484009 25249 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:44.521633 25346 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.522912 25346 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:44.523582 25603 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:44.525530 25495 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:44.525938 25495 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:44.526293 25495 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:44.518138 25578 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:44.531111 25249 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:44.531198 24811 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.515695 25604 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:07:44.521777 25613 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:07:44.508359 25465 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.534961 25465 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.535907 25553 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.536973 25465 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.515933 25210 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.542309 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:44.540081 25573 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:44.530761 25623 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.507953 25176 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:44.507609 25304 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:44.546698 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:44.547916 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:44.542613 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:44.540304 25346 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:44.573137 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:44.541585 25570 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.584784 22753 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:44.575114 22748 raft_consensus.cc:2410] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:07:44.485440 25484 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:44.592387 22901 raft_consensus.cc:2368] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:44.573619 23450 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:44.589701 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.594167 23448 raft_consensus.cc:2410] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:07:44.588821 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.596220 23446 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:07:44.549629 25210 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:44.552723 25299 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:44.600801 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:44.602798 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 208769us
W20260709 13:07:44.605289 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.605293 23451 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:44.609088 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:44.610323 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:44.609642 22755 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:07:44.626120 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:44.625141 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:44.629593 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:44.630096 25627 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.392s	user 0.004s	sys 0.000s
I20260709 13:07:44.627564 22827 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:44.627648 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:44.600219 25623 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:44.656683 23446 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:44.663596 25346 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:44.665304 25484 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:44.671366 22828 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:44.671810 22828 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:44.672238 23447 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:44.672689 22828 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:44.617959 25453 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:44.686975 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.687212 22831 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:44.669692 25621 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:44.687747 23446 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:44.689091 25484 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
W20260709 13:07:44.700160 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 75103.8us
I20260709 13:07:44.739813 22829 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:44.714481 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:44.750634 23451 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:44.755398 22831 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:44.692848 25453 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.763268 25453 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.754410 23450 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:44.718972 25647 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:07:44.772469 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.756557 25210 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.773648 22899 raft_consensus.cc:2368] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9.
I20260709 13:07:44.756034 25249 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.775341 25210 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.775621 25249 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.689164 25495 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.778229 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:44.714087 23445 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:07:44.781865 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:44.782559 22755 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:44.782570 22829 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:44.783730 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:44.718513 25096 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.788249 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:44.792945 23450 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:07:44.789124 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:44.802601 23451 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:44.804206 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.806856 22831 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:44.792393 25176 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:44.821055 22829 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:44.830662 25656 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.831210 25656 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.843973 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:44.844605 22753 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:07:44.846088 25334 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.846563 25334 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.848563 25334 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.803881 25096 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.851367 25096 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.851883 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:44.846880 22755 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:07:44.853053 22755 pending_rounds.cc:85] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 7
I20260709 13:07:44.808538 25495 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.853349 22755 pending_rounds.cc:107] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:07:44.853628 22755 raft_consensus.cc:2889] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:44.854144 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:44.854216 25176 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:44.853205 25297 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.855396 25176 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.792471 25484 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
W20260709 13:07:44.860057 25632 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.622s	user 0.000s	sys 0.003s
I20260709 13:07:44.855288 25603 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:44.833099 25656 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.863569 22824 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:44.867277 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1352) took 1035 ms. Trace:
I20260709 13:07:44.867645 22904 rpcz_store.cc:276] 0709 13:07:43.832338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.832599 (+   261us) service_pool.cc:224] Handling call
0709 13:07:43.832876 (+   277us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:43.977882 (+145006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.977927 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:44.214649 (+236722us) log.cc:844] Serialized 52 byte log entry
0709 13:07:44.237412 (+ 22763us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:44.237446 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.237510 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:44.859553 (+622043us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:44.861976 (+  2423us) raft_consensus.cc:1685] finished
0709 13:07:44.861994 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.867223 (+  5229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":56247,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"spinlock_wait_cycles":5921664,"thread_start_us":154958,"threads_started":2,"wal-append.queue_time_us":22744}
I20260709 13:07:44.870903 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2336) took 1176 ms. Trace:
I20260709 13:07:44.871378 22747 rpcz_store.cc:276] 0709 13:07:43.694835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.695000 (+   165us) service_pool.cc:224] Handling call
0709 13:07:43.695375 (+   375us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:43.733789 (+ 38414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.733824 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:43.735430 (+  1606us) log.cc:844] Serialized 52 byte log entry
0709 13:07:44.864418 (+1128988us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b5400037620. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d65090 00007fcb74d726b8 00007fcb74dfb92b 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c
0709 13:07:44.864469 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:44.864510 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.864580 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:44.864616 (+    36us) raft_consensus.cc:1685] finished
0709 13:07:44.864628 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.870826 (+  6198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25426,"raft.queue_time_us":33658,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":3306,"spinlock_wait_cycles":113953536,"thread_start_us":86106,"threads_started":1,"wal-append.queue_time_us":104434}
I20260709 13:07:44.883484 23450 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:44.882755 25484 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.884783 25300 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:44.885252 25484 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.856371 25297 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.869338 25603 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.893394 25300 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:07:44.897900 25297 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:44.899118 25451 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.873944 25096 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:44.904181 25484 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.870137 25334 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.908775 22904 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:44.911562 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.928537 22829 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:44.929207 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:44.933130 25551 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:44.933671 25551 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.935804 25551 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.904732 25603 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:44.937402 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.940203 23453 raft_consensus.cc:2368] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3.
I20260709 13:07:44.940781 22902 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:44.938232 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.938891 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.942456 22829 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:44.943394 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:44.943990 22899 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:44.931454 25096 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:44.951089 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.950138 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.951700 23453 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:44.951825 23451 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:44.950314 22719 leader_election.cc:400] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3.
I20260709 13:07:44.930603 25578 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:07:44.927302 25647 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:44.952442 22719 leader_election.cc:403] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:44.939721 25605 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:44.936728 25661 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:44.929713 25451 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.956867 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:44.938069 25334 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.943985 25574 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:07:44.962164 25656 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:44.959558 25647 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.964183 25656 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.964587 25656 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:44.965629 25334 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.965907 25647 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:44.966820 25300 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:44.967316 25300 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.980608 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:44.981534 22829 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:44.959847 25661 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.988145 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.961378 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.960995 25484 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:07:44.988698 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:44.990018 23446 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:44.969265 25300 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:44.959744 25605 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:44.997677 25661 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:45.029268 22824 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
W20260709 13:07:45.031868 25650 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:07:45.033123 25401 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:45.033532 25401 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:45.034330 25401 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.035259 25401 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:45.044037 22902 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:45.044667 22897 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:45.013899 25643 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:45.051829 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:45.021796 25300 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.056641 25096 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.057240 25643 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.057916 25451 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:45.058645 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.059242 22902 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:45.060015 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.060523 22902 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:45.061452 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.062023 22901 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 2.
I20260709 13:07:45.061883 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.063539 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.064219 22896 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:07:45.065290 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.065956 22900 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:07:45.066505 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:45.067087 22748 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:45.067672 25484 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 2. Current term is 3.
W20260709 13:07:45.068154 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.068861 25605 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.070473 25312 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.070976 25312 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.072875 25312 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:45.075270 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:45.076443 25656 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.058421 25300 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.056569 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:45.070990 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:45.080804 22755 raft_consensus.cc:2368] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5.
I20260709 13:07:45.083122 23416 leader_election.cc:400] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:45.083408 23416 leader_election.cc:403] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:07:45.083982 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:45.086392 22749 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:45.087248 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:45.087090 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:45.091853 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:45.094578 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:45.094847 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.095165 22748 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:45.097117 25593 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:45.097612 25656 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:45.099094 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:45.102212 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:45.103596 22755 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:45.106588 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:45.107849 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:45.093541 25451 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:45.117991 23448 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:45.093357 25623 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:45.092545 25547 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.124449 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:45.126457 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.128762 22900 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:45.131817 22896 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:45.095624 25613 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.141695 25666 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:45.144898 25300 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.105538 25301 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:45.147610 25490 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:45.148034 25490 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.148365 25490 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:45.149446 25613 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.153927 25623 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:45.155192 25623 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:45.156976 25652 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.447s	user 0.003s	sys 0.003s
I20260709 13:07:45.133607 25451 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.157694 25451 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:45.119230 25249 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:45.162511 25673 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:07:45.173621 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.174356 22896 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:45.176816 22828 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:45.176884 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 70364.4us
I20260709 13:07:45.164445 22897 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
W20260709 13:07:45.184783 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.196724 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:45.315114 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 99203.1us
I20260709 13:07:45.203502 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:45.320147 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 116713us
I20260709 13:07:45.320812 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:45.321961 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.183352 22898 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:45.193642 25435 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:45.335599 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:45.164026 25547 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.338460 22822 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
W20260709 13:07:45.364591 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.342396 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:45.343194 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 248667us
I20260709 13:07:45.401109 23416 leader_election.cc:400] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9.
I20260709 13:07:45.401504 23416 leader_election.cc:403] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:45.399417 25665 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:07:45.333858 25495 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:45.333461 25622 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:45.419045 22828 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:45.419725 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2132) took 1614 ms. Trace:
I20260709 13:07:45.428226 23446 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:45.432264 22824 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:45.420102 22828 rpcz_store.cc:276] 0709 13:07:43.804832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.805221 (+   389us) service_pool.cc:224] Handling call
0709 13:07:44.671225 (+866004us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54004c2820. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d72dc0 00007fcb74dfee18 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6
0709 13:07:44.771997 (+100772us) spinlock_profiling.cc:243] Waited 89.7 ms on lock 0x7b18002ffc40. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74e36646 00007fcb74d7058e 00007fcb74df1f9e 00007fcb74df37de 00007fcb74dff27c 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c
0709 13:07:45.160985 (+388988us) spinlock_profiling.cc:243] Waited 61.3 ms on lock 0x7b58006404d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d570aa 00007fcb74dd9447 00007fcb74df1fe7 00007fcb74df37de 00007fcb74dff27c 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c
0709 13:07:45.419660 (+258675us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1962713600}
I20260709 13:07:45.338373 25667 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.448547 25495 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:45.415176 25666 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5.
I20260709 13:07:45.467310 25495 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.413954 25613 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:45.449270 25455 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 13:07:45.482113 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 149612us
I20260709 13:07:45.342569 25299 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 13:07:45.485471 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:45.490152 22871 leader_election.cc:400] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5.
W20260709 13:07:45.470891 25635 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:45.447654 25622 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:45.415323 25603 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.495124 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:45.496764 25578 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:07:45.539620 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 138642us
I20260709 13:07:45.542498 23416 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:45.545500 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 50439.4us
I20260709 13:07:45.508019 25300 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.484468 25667 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.551616 25346 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:45.551887 25668 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:45.510102 25435 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:45.552472 25681 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9.
I20260709 13:07:45.553185 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 454) took 1160 ms. Trace:
I20260709 13:07:45.574319 22825 rpcz_store.cc:276] 0709 13:07:44.393065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.393174 (+   109us) service_pool.cc:224] Handling call
0709 13:07:45.553002 (+1159828us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54004123d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:45.553113 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620485376}
I20260709 13:07:45.561414 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:45.545855 25578 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:45.581475 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.562328 25675 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:45.549714 25299 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:45.563819 23445 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:07:45.551977 25676 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:07:45.566319 25668 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.576233 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:45.576332 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 86473.4us
I20260709 13:07:45.582623 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:45.584746 23452 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:45.593093 22900 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
W20260709 13:07:45.595600 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 258579us
I20260709 13:07:45.586372 25578 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:45.598914 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:45.594540 25299 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:45.605540 25455 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:07:45.606071 25299 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:45.611388 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:45.597783 25677 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:07:45.568195 25300 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.610711 25490 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.621709 25490 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.624805 25435 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.627452 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:45.627772 23446 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:07:45.628109 22754 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:45.629069 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:45.629762 25668 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.630406 25667 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:45.593690 25622 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:45.631733 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 542656us
I20260709 13:07:45.635427 25584 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:45.635914 25584 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:45.637774 25584 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.640113 25584 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:45.640257 25661 consensus_meta.cc:287] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.002s
I20260709 13:07:45.642124 25603 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.642509 23447 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:07:45.643096 25683 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:07:45.643402 25300 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.613037 25621 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:45.593573 25680 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 13:07:45.637794 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.619020 25547 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:45.649298 25401 consensus_meta.cc:287] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.614s	user 0.001s	sys 0.010s
I20260709 13:07:45.653672 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:45.654295 22755 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
W20260709 13:07:45.654863 25659 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.769s	user 0.000s	sys 0.012s
I20260709 13:07:45.655905 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2356) took 1110 ms. Trace:
I20260709 13:07:45.592476 25675 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.659343 25675 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.659451 22898 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:07:45.640588 25584 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:45.663012 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44582.6us
I20260709 13:07:45.666872 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:45.667459 22755 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:45.667689 25584 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:45.669902 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:45.656219 22752 rpcz_store.cc:276] 0709 13:07:44.545728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.547132 (+  1404us) service_pool.cc:224] Handling call
0709 13:07:44.547400 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:44.591075 (+ 43675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:44.591105 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:44.601705 (+ 10600us) log.cc:844] Serialized 26 byte log entry
0709 13:07:44.806106 (+204401us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:44.806179 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.806267 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:45.654361 (+848094us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:45.655650 (+  1289us) raft_consensus.cc:1685] finished
0709 13:07:45.655670 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:45.655852 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1182,"raft.queue_time_us":32508,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":10773,"spinlock_wait_cycles":10743552,"thread_start_us":136618,"threads_started":1,"wal-append.queue_time_us":277217}
I20260709 13:07:45.640703 25661 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.633000 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:45.675632 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:45.678534 22754 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:07:45.596994 25096 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
W20260709 13:07:45.686805 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 115611us
I20260709 13:07:45.688439 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:45.689433 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:45.689479 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:45.661371 25623 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:45.694751 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:45.696309 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:45.697067 22748 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:45.695792 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:45.696967 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:45.685815 25584 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:45.699936 22755 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:07:45.716948 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:45.718300 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:45.720033 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:45.720757 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.717022 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:45.723408 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:45.725543 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:45.661116 25451 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.728994 25451 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:45.698798 25300 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:45.729852 25299 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:45.730228 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:45.730479 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:45.731058 23446 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:45.732272 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:45.732497 25455 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:45.733487 25455 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:45.705838 25593 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:07:45.723340 25578 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.735314 25455 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:45.735595 22822 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:45.735781 25455 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:45.735954 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:45.736312 25578 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:45.730260 25299 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.736748 25455 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.733755 25621 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:45.737311 25299 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:45.743369 25661 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:45.743873 25643 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:45.694195 25603 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:07:45.745293 25299 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:45.694512 25675 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:45.724370 25584 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:45.747485 25584 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:45.733573 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:45.752166 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88603.7us
I20260709 13:07:45.752902 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:45.756799 23446 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:45.760627 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:45.759430 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:45.723345 25484 raft_consensus.cc:2764] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:07:45.757371 25681 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:45.769773 25300 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:45.761806 25679 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:45.770346 25623 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:45.792480 23446 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:45.776010 25573 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:45.781788 25210 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:07:45.818672 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 58116.5us
I20260709 13:07:45.780606 25681 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.786314 25679 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.781744 25621 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:07:45.785917 25593 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:07:45.824571 25210 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.843407 25603 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:45.850075 25679 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:45.889882 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.843313 25681 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:45.891744 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:45.901229 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:45.896041 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:45.903919 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 151100us
I20260709 13:07:45.915756 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:45.915591 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:45.921377 22828 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:07:45.855238 25693 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:45.866194 25210 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:45.863910 25300 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:45.967783 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:45.979096 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 191764us
I20260709 13:07:45.915241 25643 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:45.917721 25656 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:45.915148 25603 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:45.978083 22896 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:45.968194 25661 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:45.923767 25465 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:45.988343 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68170.4us
I20260709 13:07:45.999614 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:45.983491 25643 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.975101 25569 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:07:45.983409 25656 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:46.020004 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 294539us
I20260709 13:07:46.046046 23447 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:07:46.069671 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:46.071370 22896 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:46.075344 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:46.074671 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 75155.9us
I20260709 13:07:46.075966 22902 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:46.080411 22719 leader_election.cc:400] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6.
I20260709 13:07:46.080776 22719 leader_election.cc:403] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:46.083361 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:46.085278 25695 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:07:46.014048 25465 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.088522 25465 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:45.973413 25693 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.013991 25578 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.014125 25670 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:46.092475 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:46.095630 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:46.019683 25643 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:46.076104 25656 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:46.016891 25176 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:45.982770 25623 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.097556 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:46.107563 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.733s (SENT)
I20260709 13:07:46.146745 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:46.110859 25715 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.115870 25300 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.150763 25715 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.090121 25621 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
W20260709 13:07:46.204546 25702 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:07:46.200030 25716 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:46.208803 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 130923us
I20260709 13:07:46.103962 25578 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.196313 25297 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:46.229553 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 171696us
I20260709 13:07:46.116580 25660 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:46.195269 25176 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:46.269418 25697 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 13:07:46.269843 25691 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.354s	user 0.001s	sys 0.001s
I20260709 13:07:46.301286 22902 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:07:46.250875 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:46.115718 25603 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.205751 25726 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:07:46.248546 25297 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:46.312063 25696 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
W20260709 13:07:46.313889 25584 consensus_meta.cc:287] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.000s
I20260709 13:07:46.314033 25297 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:46.318398 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 67631.6us
I20260709 13:07:46.327189 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:46.327764 23444 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:46.331965 25717 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:46.332477 25717 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.226308 25716 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.334406 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:46.334908 25717 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.335001 25716 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.335242 22827 raft_consensus.cc:2410] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:46.337749 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:46.337996 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:46.338685 22824 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:46.339303 25706 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:46.339807 25706 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:46.340516 25679 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.340662 25706 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.314283 25584 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.349807 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:46.354985 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:46.366716 23444 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:46.369125 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:46.353385 25698 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.367s	user 0.000s	sys 0.001s
W20260709 13:07:46.373950 25704 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:07:46.375497 25693 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:46.375629 25700 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.284s	user 0.001s	sys 0.001s
I20260709 13:07:46.376488 23445 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:46.378086 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 529) took 1285 ms. Trace:
I20260709 13:07:46.304278 25660 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6.
I20260709 13:07:46.323556 25621 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:46.410728 23453 rpcz_store.cc:276] 0709 13:07:45.092629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.093114 (+   485us) service_pool.cc:224] Handling call
0709 13:07:45.105796 (+ 12682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:45.489486 (+383690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.489514 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:45.714051 (+224537us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b2c0013bd48. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:07:45.714373 (+   322us) log.cc:844] Serialized 26 byte log entry
0709 13:07:45.998585 (+284212us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:45.998628 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.000462 (+  1834us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:46.377803 (+377341us) raft_consensus.cc:1685] finished
0709 13:07:46.377823 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:46.378014 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46785,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"spinlock_wait_cycles":10703744,"thread_start_us":596644,"threads_started":2,"wal-append.queue_time_us":611335}
I20260709 13:07:46.416044 25730 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:46.418166 25730 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.418521 25730 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:46.371377 25716 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:07:46.414690 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 187949us
I20260709 13:07:46.428299 22824 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:46.421419 25621 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:46.436786 25578 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.448904 25621 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:46.449353 25734 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.450138 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:46.451175 23445 raft_consensus.cc:2410] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:46.453003 25584 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.456889 25578 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:46.458189 25679 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:46.462659 25714 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 13:07:46.464192 25681 consensus_meta.cc:287] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.547s	user 0.000s	sys 0.007s
I20260709 13:07:46.464572 25681 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.461077 25734 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.465507 25711 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:46.430397 25716 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.478032 25716 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:46.559120 25734 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:46.560509 25210 consensus_meta.cc:287] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.000s
I20260709 13:07:46.560914 25210 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.562947 25210 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.569841 23446 raft_consensus.cc:2368] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:46.570624 25681 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:46.571705 25656 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.573729 25656 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.467648 25621 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:46.574604 25621 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:46.576524 25621 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:46.601330 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 132468us
W20260709 13:07:46.602341 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 127302us
I20260709 13:07:46.613353 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:46.614271 23444 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:46.616194 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:46.457194 25643 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.634138 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:46.635838 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:46.639652 23444 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 7: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:46.636421 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:46.623556 25455 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:07:46.655853 25465 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.017s	sys 0.062s
I20260709 13:07:46.547068 25711 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.631662 25730 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:46.691430 23448 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.008s	sys 0.008s
W20260709 13:07:46.691833 23448 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.008s	sys 0.008s
W20260709 13:07:46.693063 22797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.006s	sys 0.017s
W20260709 13:07:46.693450 22797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.006s	sys 0.018s
I20260709 13:07:46.635197 25660 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.631606 25282 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:46.703058 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:46.631767 25555 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:46.642807 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:46.658525 25661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.003s	sys 0.007s
I20260709 13:07:46.718631 22824 raft_consensus.cc:2368] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8.
I20260709 13:07:46.719128 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:46.659268 25713 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:46.698632 25721 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:46.699086 25578 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:46.699292 25621 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:46.706697 25282 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.717801 25584 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:46.745931 22750 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:46.724650 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:46.737802 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:46.751186 22826 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:46.752730 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:46.753515 22750 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:46.743310 25282 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:46.704583 25455 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:46.759526 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:46.712505 25660 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:46.764847 23416 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.016s	sys 0.032s
I20260709 13:07:46.766503 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:46.769033 23416 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.016s	sys 0.032s
W20260709 13:07:46.769444 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 661914us
I20260709 13:07:46.650763 25716 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.750725 23445 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:46.724439 25727 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:46.755224 25584 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.849519 25584 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.851279 25455 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:46.851753 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:46.775683 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:46.750355 25656 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:46.852690 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 77068.3us
I20260709 13:07:46.852898 25656 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.854970 25656 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.809966 22748 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:07:46.856333 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:46.856830 25656 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:46.857105 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:46.857884 22830 pending_rounds.cc:85] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 7
I20260709 13:07:46.858155 22830 pending_rounds.cc:107] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:07:46.858381 22830 raft_consensus.cc:2889] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:46.859865 22753 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:46.860913 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:46.861814 25741 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.862360 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:46.862291 25741 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.863078 22756 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:46.864512 25741 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.770761 25693 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:46.738161 25578 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.864979 25741 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:46.865576 25693 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:46.819408 22751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.012s	sys 0.010s
W20260709 13:07:46.866520 22751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.012s	sys 0.010s
I20260709 13:07:46.738215 25621 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:46.737630 25721 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:46.867383 22905 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.020s	sys 0.009s
W20260709 13:07:46.868825 22905 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.020s	sys 0.009s
I20260709 13:07:46.827589 25546 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:46.832068 25730 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.832490 25727 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:46.730319 25713 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:46.842785 25711 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:46.883003 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:46.844091 25660 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.844848 25643 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:46.819808 25706 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:07:46.859797 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:46.775282 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:46.817642 25699 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:46.777132 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:46.724275 25661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.005s	sys 0.014s
I20260709 13:07:46.883724 23451 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:46.903718 22896 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.008s	sys 0.038s
I20260709 13:07:46.865849 25741 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:46.966446 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:46.829830 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:46.970368 22755 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:07:46.913216 22829 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:46.973865 23451 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:46.930583 22750 raft_consensus.cc:2410] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:07:46.739509 25555 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:46.936702 22896 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.008s	sys 0.038s
I20260709 13:07:46.939261 22749 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:46.903847 25546 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.016692 23444 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:46.911412 25656 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.095922 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:46.819872 22795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.011s	sys 0.011s
W20260709 13:07:47.128086 22795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.011s	sys 0.011s
I20260709 13:07:47.074915 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:46.936462 25661 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:46.927886 25681 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.218s
I20260709 13:07:46.911746 25571 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.236s
I20260709 13:07:47.129235 22899 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:47.409304 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 903) took 1083 ms. Trace:
W20260709 13:07:47.318063 23416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.013s	sys 0.052s
I20260709 13:07:47.073589 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:47.420720 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 627) took 1301 ms. Trace:
I20260709 13:07:47.419445 23447 rpcz_store.cc:276] 0709 13:07:46.325951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.327027 (+  1076us) service_pool.cc:224] Handling call
0709 13:07:46.327289 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.348140 (+ 20851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.348186 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.348642 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.627686 (+279044us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:46.627740 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.627826 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.126059 (+498233us) raft_consensus.cc:1685] finished
0709 13:07:47.126078 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.409228 (+283150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"spinlock_wait_cycles":25984,"thread_start_us":252200,"threads_started":1,"wal-append.queue_time_us":458371}
I20260709 13:07:47.181842 22755 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:47.421253 22896 rpcz_store.cc:276] 0709 13:07:46.119557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.123482 (+  3925us) service_pool.cc:224] Handling call
0709 13:07:46.209629 (+ 86147us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.246015 (+ 36386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.246083 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.251079 (+  4996us) log.cc:844] Serialized 26 byte log entry
0709 13:07:47.009798 (+758719us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:47.418848 (+409050us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.419478 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.419748 (+   270us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.419979 (+   231us) raft_consensus.cc:1685] finished
0709 13:07:47.420069 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.420668 (+   599us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18677,"raft.queue_time_us":25602,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":21361,"thread_start_us":1049172,"threads_started":2,"wal-append.queue_time_us":805939}
I20260709 13:07:47.137401 22756 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
W20260709 13:07:47.313568 23418 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.009s	sys 0.007s
W20260709 13:07:47.424444 23418 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.011s	sys 0.009s
W20260709 13:07:47.126231 23449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.008s	sys 0.002s
W20260709 13:07:47.419445 23416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.022s	sys 0.057s
W20260709 13:07:47.425302 23449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.016s	sys 0.006s
W20260709 13:07:47.425451 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 654695us
I20260709 13:07:46.656922 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:47.428025 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:47.121042 25701 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.684s	user 0.000s	sys 0.001s
I20260709 13:07:47.133666 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.276774 25689 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.101570 25555 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.418362 25455 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:47.451833 25455 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:47.451939 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:47.453027 25455 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.454360 25455 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:47.455134 25282 consensus_meta.cc:287] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.001s
I20260709 13:07:47.455533 25282 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.056809 25546 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:47.062759 25693 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.105582 25569 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:47.145273 25734 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.064863 25621 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.066102 25578 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.409351 25622 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.102337 25613 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:47.467567 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.490180 25716 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.136546 25656 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.124986 25176 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.318252 25736 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:47.094211 25711 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:47.318383 25756 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:47.426738 25729 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.065673 25727 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:47.451037 25689 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.367636 25739 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:47.507350 22830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.005s	sys 0.037s
I20260709 13:07:46.965978 25699 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.483781 25484 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:47.501606 25465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.080s	sys 0.161s
W20260709 13:07:47.526803 22830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.006s	sys 0.041s
W20260709 13:07:47.520740 22827 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.008s	sys 0.032s
I20260709 13:07:47.503424 25282 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:47.522693 25334 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.019s	sys 0.079s
I20260709 13:07:47.478138 25569 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.479669 25734 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:47.559684 25465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.085s	sys 0.175s
W20260709 13:07:47.563050 25334 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.020s	sys 0.080s
W20260709 13:07:47.560839 22827 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.012s	sys 0.043s
I20260709 13:07:47.492772 25613 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.487059 25622 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.523864 25736 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.525163 25711 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.526893 25729 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.553994 25756 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.574901 22899 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
W20260709 13:07:47.573493 25761 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:07:47.558828 25739 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:47.577132 25766 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 13:07:47.578617 25722 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.345s	user 0.004s	sys 0.009s
W20260709 13:07:47.579416 25688 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.894s	user 0.003s	sys 0.006s
I20260709 13:07:47.521487 25661 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.594589 25734 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.597590 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2159) took 1678 ms. Trace:
I20260709 13:07:47.598002 22822 rpcz_store.cc:276] 0709 13:07:45.918836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.921616 (+  2780us) service_pool.cc:224] Handling call
0709 13:07:45.921995 (+   379us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:45.932239 (+ 10244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.932268 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:45.976034 (+ 43766us) log.cc:844] Serialized 52 byte log entry
0709 13:07:46.034028 (+ 57994us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:46.034107 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.034298 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:46.464114 (+429816us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b4800130158. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb79d24aa1 00007fcb79d1bb37 00007fcb79d1b9ca 00007fcb79e1326f 00007fcb79e12f37 00007fcb79e2755c 00007fcb79e27508 00007fcb79e27488 00007fcb79e27443 00007fcb79e266ec 00007fcb78d50a96
0709 13:07:47.578015 (+1113901us) spinlock_profiling.cc:243] Waited 50.1 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:07:47.597388 (+ 19373us) raft_consensus.cc:1685] finished
0709 13:07:47.597407 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.597535 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3409,"raft.queue_time_us":60434,"raft.run_cpu_time_us":2267,"raft.run_wall_time_us":479522,"spinlock_wait_cycles":121932032,"thread_start_us":41817,"threads_started":1,"wal-append.queue_time_us":131269}
I20260709 13:07:47.562701 25484 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.613689 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 911) took 1139 ms. Trace:
I20260709 13:07:47.614284 23449 rpcz_store.cc:276] 0709 13:07:46.474574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.474848 (+   274us) service_pool.cc:224] Handling call
0709 13:07:46.475218 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.475625 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.475664 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.476178 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:07:47.425757 (+949579us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:47.425808 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.425934 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.613386 (+187452us) raft_consensus.cc:1685] finished
0709 13:07:47.613419 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.613598 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2152,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":848604,"threads_started":1,"wal-append.queue_time_us":932250}
I20260709 13:07:47.632328 25736 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:47.630056 25699 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:47.634315 25739 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.635627 25613 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.636301 25656 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.637748 25729 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:47.637964 25694 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.843s	user 0.018s	sys 0.000s
I20260709 13:07:47.627140 25689 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:47.641081 25744 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:07:47.646715 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.648658 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.656088 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 892) took 1735 ms. Trace:
I20260709 13:07:47.658064 25743 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.658650 25743 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.659595 25681 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:47.659986 25681 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:47.660264 25621 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.660670 25743 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.660920 25553 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.658170 25668 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.659636 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1375) took 1009 ms. Trace:
I20260709 13:07:47.666031 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:47.661439 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 233476us
I20260709 13:07:47.669986 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:47.662050 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 210193us
I20260709 13:07:47.705361 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:47.706171 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:47.664947 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:47.665650 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.703596 22898 rpcz_store.cc:276] 0709 13:07:46.650533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.650718 (+   185us) service_pool.cc:224] Handling call
0709 13:07:46.659806 (+  9088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.564516 (+904710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:47.564548 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.564589 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:47.564622 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.659434 (+ 94812us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54003e9754. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:47.659475 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.659589 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2080104192}
I20260709 13:07:47.643172 25569 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.712270 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.713707 23449 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:47.718026 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.720196 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.720748 22898 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:47.709683 25553 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.666325 25675 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:47.701951 25613 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.737787 22822 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:07:47.719491 25682 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:47.660758 25681 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.734876 22830 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:47.664269 25707 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:47.661267 25743 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.719391 25680 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:47.660769 25621 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:47.729825 25734 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.755379 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.746616 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.749446 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.749673 22830 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:47.759239 22755 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:47.760478 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.761687 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.762411 22748 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:47.725092 25455 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:47.656589 23448 rpcz_store.cc:276] 0709 13:07:45.920698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.920803 (+   105us) service_pool.cc:224] Handling call
0709 13:07:45.974578 (+ 53775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.974799 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:46.703432 (+728633us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.145742 (+442310us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.655538 (+509796us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b18000e8c00. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb74df3060 00007fcb74e05716 00007fcb74e16287 00007fcb74e16208
0709 13:07:47.655704 (+   166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.655763 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:47.655825 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.655905 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.656022 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":300029,"prepare.queue_time_us":138014,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":33660,"raft.queue_time_us":511635,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":74603,"spinlock_wait_cycles":279860608,"thread_start_us":1148802,"threads_started":2,"wal-append.queue_time_us":866801}
I20260709 13:07:47.751308 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:47.749159 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1852) took 1842 ms. Trace:
I20260709 13:07:47.764469 22822 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:47.764685 22905 rpcz_store.cc:276] 0709 13:07:45.907310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.907457 (+   147us) service_pool.cc:224] Handling call
0709 13:07:45.907811 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:45.923689 (+ 15878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.923719 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:45.924109 (+   390us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.870215 (+946106us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:46.870266 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.870353 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.748986 (+878633us) raft_consensus.cc:1685] finished
0709 13:07:47.749003 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.749113 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7841,"raft.queue_time_us":22799,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":11746,"thread_start_us":887514,"threads_started":1,"wal-append.queue_time_us":1597728}
I20260709 13:07:47.766036 25553 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.766251 25734 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.768205 25734 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.768290 25621 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:47.759064 22898 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:47.770236 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:47.732074 25689 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:47.771622 22747 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:47.742512 25711 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.761245 22756 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:47.721273 25668 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.786509 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.787587 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:47.787796 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.788424 22896 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:47.773598 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.771749 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.773468 22822 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:47.789803 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2372) took 2136 ms. Trace:
I20260709 13:07:47.791028 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.757536 25622 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.791709 22822 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:47.793599 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.794157 22831 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:47.790256 22751 rpcz_store.cc:276] 0709 13:07:45.647847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.649273 (+  1426us) service_pool.cc:224] Handling call
0709 13:07:45.653560 (+  4287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.668871 (+ 15311us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:45.669091 (+   220us) log.cc:844] Serialized 14 byte log entry
0709 13:07:46.866826 (+1197735us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:46.866870 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:46.866907 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.120726 (+253819us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b2400139aa8. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71b949c4 00007fcb71b92acb 00007fcb71d4ddc9 00007fcb71d4da55 00007fcb71d4d09d 00007fcb7497f10e 00007fcb7499688c 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221
0709 13:07:47.783558 (+662832us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5400406ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:47.783627 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.783769 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43688,"prepare.queue_time_us":2401,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":166370,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":71,"spinlock_wait_cycles":46288128,"thread_start_us":1153567,"threads_started":2,"wal-append.queue_time_us":744740}
I20260709 13:07:47.774370 25680 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:47.773890 25706 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:47.778721 25681 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:47.795609 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.796713 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:47.796931 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.795506 25451 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:47.776845 25756 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.779943 25765 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:47.743737 25613 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.799551 22824 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:47.779357 25707 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:47.805689 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.758577 25743 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.806599 22831 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:47.774874 25455 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.810117 22896 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:47.791388 25763 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:47.792937 25762 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.779098 25689 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.815869 23449 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:47.817088 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.815689 25705 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:07:47.809397 25451 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.811774 25765 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.832867 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.834179 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.834555 25765 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.835300 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.801856 25706 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.818606 25680 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.835953 22896 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:47.836831 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.837370 25706 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.818907 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.817503 23444 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:47.838996 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.830554 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:47.823668 25743 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.826956 25455 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.840282 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:47.818389 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.841768 22830 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 1.
I20260709 13:07:47.806836 25670 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.843449 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:47.847010 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:47.847012 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:47.847679 23444 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:47.842854 25705 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:47.850950 23451 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:47.851549 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:47.850835 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:47.852357 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:47.852847 25668 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.853466 25484 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.852313 23445 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:07:47.829380 25763 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.854712 25249 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:47.855132 25249 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:47.855504 25249 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:47.856055 25763 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.857496 25727 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:47.857782 25451 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.857906 25727 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:47.858345 25727 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:47.853809 25687 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:47.862838 25687 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:47.821131 25299 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.863687 25687 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.865010 25689 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.815079 25699 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:07:47.865924 25495 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:47.865993 25484 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.866261 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.866387 25495 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:47.866545 25484 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.867388 25495 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.868526 25484 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.868867 22899 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:07:47.868947 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.869690 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 1906) took 1110 ms. Trace:
I20260709 13:07:47.848397 25774 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.870060 22749 rpcz_store.cc:276] 0709 13:07:46.759126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.759265 (+   139us) service_pool.cc:224] Handling call
0709 13:07:47.869568 (+1110303us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54003372d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:47.869644 (+    76us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391712128}
I20260709 13:07:47.870433 25774 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.871749 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.872165 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 536) took 1212 ms. Trace:
I20260709 13:07:47.872643 25774 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.873808 23449 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:07:47.875054 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:47.875156 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:47.875711 23447 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:47.875976 22896 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:47.877300 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:47.877746 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:47.877355 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.883255 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.883375 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:47.884403 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.884945 23447 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:07:47.885008 22899 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:07:47.872507 23453 rpcz_store.cc:276] 0709 13:07:46.659533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.660025 (+   492us) service_pool.cc:224] Handling call
0709 13:07:46.685605 (+ 25580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.720007 (+ 34402us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:46.734346 (+ 14339us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.074027 (+339681us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.074090 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:47.074147 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.871968 (+797821us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b5400671454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:47.872008 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.872113 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23678,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":20144,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":28746,"spinlock_wait_cycles":78510336,"thread_start_us":339259,"threads_started":1,"wal-append.queue_time_us":633463}
I20260709 13:07:47.887084 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:47.887655 22750 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:47.848273 25569 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:07:47.838807 25762 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.885963 25693 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:47.886292 25578 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:47.890884 25762 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.890655 25707 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.873605 25670 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.893487 25096 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:47.893914 25096 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:47.894259 25096 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:47.894609 25670 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.895001 23445 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:07:47.895344 25613 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.897437 25455 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:47.897881 25455 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:47.898267 25455 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:47.900506 25096 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.900954 25715 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.902558 25096 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:47.850878 25705 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:47.904531 25705 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:47.906607 22750 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:07:47.909581 23444 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:07:47.910411 23451 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: [6.6-8.7]
I20260709 13:07:47.868532 25495 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:47.847884 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1513) took 1134 ms. Trace:
I20260709 13:07:47.915557 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:47.942785 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 61852.7us
I20260709 13:07:47.942088 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1514) took 1228 ms. Trace:
I20260709 13:07:47.945272 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.945922 22749 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 2.
I20260709 13:07:47.944877 22828 rpcz_store.cc:276] 0709 13:07:46.714015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.714152 (+   137us) service_pool.cc:224] Handling call
0709 13:07:46.714471 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.619792 (+905321us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:47.620061 (+   269us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.806084 (+186023us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.806131 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:47.806173 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.941863 (+135690us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b5400090bd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:47.941913 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.942036 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18278,"prepare.queue_time_us":3352,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":18602,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":5926,"spinlock_wait_cycles":2081535360,"thread_start_us":167427,"threads_started":1,"wal-append.queue_time_us":217896}
W20260709 13:07:47.947666 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 74409.3us
I20260709 13:07:47.884227 25484 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.949801 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:47.950232 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.921708 22822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:07:47.950431 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:47.951339 22822 raft_consensus.cc:606] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:47.952791 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:47.954860 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.956378 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.958529 23449 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:47.962287 23419 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:47.950789 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.921307 25096 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.966393 22896 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:47.967231 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:47.969048 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.971647 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.969141 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.972714 22748 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:47.973774 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:47.974049 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.969615 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.975050 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:47.966302 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:47.977649 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.978204 22748 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:47.978793 22750 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:47.922123 25693 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:47.980949 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:47.975821 25668 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:47.982890 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.982281 25699 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:07:47.980566 25675 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:48.005863 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.006582 22899 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:47.985468 25668 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.982388 25736 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:07:47.979671 25096 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.008957 25096 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:47.955025 25484 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:48.010334 25680 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:48.011132 25484 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:47.987082 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:47.988730 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.011296 25680 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:47.964341 25762 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:48.012132 22751 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:48.012461 25762 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:47.993825 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.013149 25762 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:47.994679 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.996675 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:47.970288 25711 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:48.014878 25711 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:48.015420 25680 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:48.015509 25711 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:47.969787 25743 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:47.990494 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:47.969852 25774 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:47.989575 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.004611 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.018074 25484 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.012449 22750 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:48.019173 25484 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.021368 25484 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.021716 25661 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:48.022336 25661 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:48.022377 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.023762 25484 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:48.023506 25661 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.024515 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:48.025543 25484 raft_consensus.cc:2764] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:07:48.028333 25762 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.031533 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.035233 25711 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:48.035917 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 70771.1us
I20260709 13:07:48.032366 22719 leader_election.cc:400] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8.
I20260709 13:07:47.952214 25705 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:07:47.964092 25670 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:07:47.981426 25569 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:07:48.037451 22719 leader_election.cc:403] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:48.038899 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.039436 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:48.039824 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.050577 22752 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:07:47.968231 25679 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:07:48.052690 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.055672 25762 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.057981 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.058944 25743 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:48.059859 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.062584 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:48.001138 25720 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.000876 25756 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.073913 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:48.074482 22755 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:48.075065 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:48.048348 25613 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:48.079604 25613 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:48.079668 25756 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.080535 25613 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.005776 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:48.081622 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.081766 25613 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:48.082588 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.082743 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:48.083289 23445 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:48.084143 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:48.086621 25569 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:48.086632 25578 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:48.087214 25569 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:48.081785 25756 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.086750 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.088147 25569 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.052309 25699 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:48.089208 25745 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:48.087314 25578 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.085119 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:48.092048 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:48.096021 22896 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:48.096477 22755 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:48.096554 22749 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:48.089547 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:48.099128 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:07:48.106237 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 43708.8us
I20260709 13:07:48.106609 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:48.107788 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:48.062247 25743 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:48.113767 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:48.037298 25711 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:48.053505 25484 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8.
I20260709 13:07:48.127908 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:48.089265 25569 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:48.129781 22896 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:07:48.092006 25643 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:48.090615 25578 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:48.133581 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 129513us
I20260709 13:07:48.134223 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.106091 25745 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:48.138744 22797 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:48.114743 25096 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:48.039918 25249 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.144703 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:48.148787 22797 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:48.158278 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.152211 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.146138 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.156008 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.118530 25706 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:48.125706 25689 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:48.126042 25730 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:48.115281 25668 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.148679 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1849) took 2237 ms. Trace:
I20260709 13:07:48.160194 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.162679 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.166054 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.163156 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.161576 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:48.089744 25699 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.132655 25726 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:48.163473 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.140452 25711 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:48.165366 22904 rpcz_store.cc:276] 0709 13:07:45.898063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.898199 (+   136us) service_pool.cc:224] Handling call
0709 13:07:45.903160 (+  4961us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:45.917429 (+ 14269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.917458 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:47.948144 (+2030686us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b2c00013588. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:07:47.952456 (+  4312us) log.cc:844] Serialized 52 byte log entry
0709 13:07:48.009335 (+ 56879us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:48.047562 (+ 38227us) log.cc:844] Serialized 14 byte log entry
0709 13:07:48.131483 (+ 83921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.131658 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:48.131744 (+    86us) raft_consensus.cc:1685] finished
0709 13:07:48.131986 (+   242us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:48.134745 (+  2759us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14885,"prepare.queue_time_us":38098,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":38806,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":20955,"spinlock_wait_cycles":13884800,"thread_start_us":110098,"threads_started":2,"wal-append.queue_time_us":38989}
I20260709 13:07:48.186573 25734 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:48.186957 25734 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.187311 25734 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:48.180253 25621 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:48.089540 25693 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.189658 25621 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:48.190090 25621 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:48.190742 25622 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:48.191124 25622 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:48.192204 25622 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.161016 25706 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.193646 25706 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:48.162923 25299 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:48.194840 25721 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.195514 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.193249 25622 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:48.196676 23446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 901) took 2305 ms (client timeout 1566 ms). Trace:
W20260709 13:07:48.197069 23446 rpcz_store.cc:269] 0709 13:07:45.891393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.891584 (+   191us) service_pool.cc:224] Handling call
0709 13:07:48.196470 (+2304886us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b5400555d54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:48.196584 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502229248}
I20260709 13:07:48.119338 25720 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.156728 25707 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:48.160861 25593 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:48.148257 25573 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.203512 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:48.147699 25096 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.217415 25096 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:48.218955 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.221697 25573 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.223671 25573 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:48.208516 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:48.161857 25745 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:48.224565 25706 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.224817 25455 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.225450 23447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:07:47.946904 22827 rpcz_store.cc:276] 0709 13:07:46.713413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.718946 (+  5533us) service_pool.cc:224] Handling call
0709 13:07:46.764851 (+ 45905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.813771 (+ 48920us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:07:47.570179 (+756408us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.809081 (+238902us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.810335 (+  1254us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:47.817602 (+  7267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.827863 (+ 10261us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.843953 (+ 16090us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14255,"prepare.queue_time_us":626742,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":25320,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5916,"thread_start_us":855114,"threads_started":2,"wal-append.queue_time_us":226924}
I20260709 13:07:48.225955 23447 raft_consensus.cc:606] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:48.226292 23447 raft_consensus.cc:614] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:48.226408 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.226708 25455 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.227075 22827 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:48.201812 25726 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.214776 25593 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.228014 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.228371 25593 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:48.228884 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.199342 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.184810 25689 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:48.228149 25720 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:48.229740 22825 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:48.216921 22897 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:48.236157 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:48.236934 22755 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:48.165215 25730 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.256064 25730 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:48.255841 25689 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.229136 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.209506 25707 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.264921 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:48.265679 23447 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:48.225131 25745 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.230499 23446 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:48.275923 25621 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:48.231185 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 63184.2us
W20260709 13:07:48.232615 22829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1518) took 1376 ms (client timeout 1576 ms). Trace:
I20260709 13:07:48.234514 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.234867 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:48.279444 22899 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:48.235883 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.236820 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:48.216477 22898 raft_consensus.cc:2376] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10. Re-sending same reply.
I20260709 13:07:48.281970 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.282658 22831 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:48.283524 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:48.279222 22829 rpcz_store.cc:269] 0709 13:07:46.855998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.856101 (+   103us) service_pool.cc:224] Handling call
0709 13:07:48.232371 (+1376270us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b54002c9554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:48.232540 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129240448}
I20260709 13:07:48.286886 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.287797 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.288681 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.284474 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.290427 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:48.290758 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.290717 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.209329 25711 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.309795 22827 raft_consensus.cc:2368] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7.
I20260709 13:07:48.310694 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.312552 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.277417 25707 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:48.314267 25791 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:48.309836 25699 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:48.278724 25734 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.315532 25699 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:48.315881 25699 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:48.314098 22753 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:07:48.316901 25734 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:48.326985 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 178251us
I20260709 13:07:48.328132 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:48.328703 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:48.332398 23416 leader_election.cc:400] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7.
I20260709 13:07:48.332823 23416 leader_election.cc:403] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:48.396050 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 67978us
I20260709 13:07:48.345599 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.414937 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.422930 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.319634 25720 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:07:48.554960 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 226305us
I20260709 13:07:48.555213 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:48.555819 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.557123 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:48.559502 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 142123us
I20260709 13:07:48.561017 22753 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:48.561654 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 123826us
I20260709 13:07:48.568490 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.316674 25699 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.383322 25679 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7.
I20260709 13:07:48.581305 22904 raft_consensus.cc:2410] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:07:48.583076 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1376) took 1865 ms. Trace:
I20260709 13:07:48.415738 25706 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:48.480504 25484 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:48.334408 25789 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:48.424119 25455 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:48.565217 25573 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:48.421782 25711 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:48.589969 22901 rpcz_store.cc:276] 0709 13:07:46.717633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.719056 (+  1423us) service_pool.cc:224] Handling call
0709 13:07:46.719375 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.969610 (+250235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.969681 (+    71us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:46.969742 (+    61us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:07:46.969787 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.582869 (+1613082us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400184a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:48.582921 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:48.583028 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574803200}
W20260709 13:07:48.640137 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 71709.1us
I20260709 13:07:48.589358 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.593704 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.594684 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.594949 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.597658 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.609812 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:48.625662 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.625823 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:48.681295 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 55582.1us
I20260709 13:07:48.682145 23419 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:48.382130 25593 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:48.687335 23445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.020s	sys 0.050s
W20260709 13:07:48.687693 23445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.020s	sys 0.050s
I20260709 13:07:48.689052 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:48.589361 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.670967 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.675045 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:48.675912 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.713114 22750 raft_consensus.cc:2410] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:07:48.714021 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.578122 25553 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:48.682474 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:48.683284 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.565044 25743 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:48.611320 25455 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.677220 25699 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:48.721359 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.596336 25484 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:48.695887 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.738479 22748 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
W20260709 13:07:48.697279 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.698413 25621 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.596614 25797 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:48.700981 25593 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.689530 25789 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:48.678292 25689 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:48.757035 25689 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:48.758190 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:48.724023 25743 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.758852 22896 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:48.760178 25743 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.725044 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:48.725692 22827 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:48.761482 25789 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:48.725771 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:48.763185 22749 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
W20260709 13:07:48.726965 25727 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.868s	user 0.007s	sys 0.004s
W20260709 13:07:48.728125 25096 consensus_meta.cc:287] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.000s
I20260709 13:07:48.640596 25578 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.732883 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:48.690817 25711 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:48.719288 25762 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:48.737443 25299 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:48.743096 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:48.744911 25484 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:48.745298 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:48.745631 22898 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
W20260709 13:07:48.747347 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 152451us
W20260709 13:07:48.747958 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 181180us
I20260709 13:07:48.757961 25689 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.723659 25455 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:48.781836 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 214284us
I20260709 13:07:48.784040 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:48.769491 22896 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:07:48.743767 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:48.775246 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 471) took 2439 ms. Trace:
I20260709 13:07:48.780891 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:48.787326 22823 rpcz_store.cc:276] 0709 13:07:46.332069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.337422 (+  5353us) service_pool.cc:224] Handling call
0709 13:07:48.732772 (+2395350us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b5800144f98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6eee 0000000000375939 00007fcb79d4a1e0 00007fcb79e130c4 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:07:48.771289 (+ 38517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13555840}
I20260709 13:07:48.791152 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:48.791703 23449 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:48.794711 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:48.798074 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.762356 25789 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:48.781142 25569 consensus_meta.cc:287] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.638s	user 0.007s	sys 0.002s
I20260709 13:07:48.812628 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:48.822197 23444 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:07:48.773070 25711 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:07:48.824770 25796 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:07:48.825281 25711 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:48.823722 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.830945 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:48.832767 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:48.822957 25668 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:48.833582 25656 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:48.834033 25656 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.834383 25656 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:48.841572 25675 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:48.842051 25786 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:48.842666 25675 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:48.833588 25668 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:48.843307 25786 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:48.843652 25668 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:48.843971 25675 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:48.844460 25675 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:48.845541 22751 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:48.850286 25711 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.782508 25762 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:48.852818 25656 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.853507 25762 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.855336 25711 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.856189 25656 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:48.857592 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:48.858705 25656 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:48.861739 25656 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.862502 25578 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.778518 25346 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:48.772219 25727 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.763616 25730 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.887548 25730 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.883263 25713 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.882370 25743 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:48.878770 25686 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:48.885013 25593 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:48.885463 25346 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.775624 25096 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.901041 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:48.901803 23453 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:48.899874 25727 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:48.897055 25743 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.904891 25713 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.905407 25713 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.908195 23444 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:48.879634 25656 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:48.858749 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:48.911039 22828 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:48.881848 25622 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.020s
W20260709 13:07:48.913529 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54830.1us
I20260709 13:07:48.914737 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:48.865020 25675 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:48.931286 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.898902 25593 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:48.942572 25707 consensus_meta.cc:287] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.628s	user 0.010s	sys 0.002s
I20260709 13:07:48.924467 25808 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:48.935859 25401 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:48.961053 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:48.944525 25727 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:48.972478 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:48.972846 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:48.974594 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:48.976442 25726 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:48.951875 25707 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.956611 25455 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.983078 25455 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:48.983196 25707 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:48.983630 25727 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:48.984030 25721 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:48.952795 25668 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:48.985209 22829 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:48.950436 25808 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:48.998636 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 628) took 2796 ms. Trace:
I20260709 13:07:48.999034 22903 rpcz_store.cc:276] 0709 13:07:46.202610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.208317 (+  5707us) service_pool.cc:224] Handling call
0709 13:07:46.231337 (+ 23020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.250229 (+ 18892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.250265 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:47.667423 (+1417158us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b1800388960. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a
0709 13:07:48.621814 (+954391us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b2c00013588. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:07:48.622702 (+   888us) log.cc:844] Serialized 26 byte log entry
0709 13:07:48.749439 (+126737us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:48.772037 (+ 22598us) log.cc:844] Serialized 14 byte log entry
0709 13:07:48.777919 (+  5882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.778010 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:48.998418 (+220408us) raft_consensus.cc:1685] finished
0709 13:07:48.998446 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:48.998582 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45911,"prepare.queue_time_us":22319,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":1901,"raft.queue_time_us":54789,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":25935,"spinlock_wait_cycles":104679168,"thread_start_us":126141,"threads_started":1,"wal-append.queue_time_us":277462}
I20260709 13:07:48.971447 25713 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:48.970610 25743 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:49.001741 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 360902us
I20260709 13:07:49.004455 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:48.993597 25808 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:49.014684 22756 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:07:49.014678 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.006358 25809 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:49.012305 25707 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.021823 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.010766 25743 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.013628 25346 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.014918 25721 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:49.025718 25814 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.025525 25578 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:49.034197 25668 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:49.028635 25806 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.216s	user 0.000s	sys 0.004s
W20260709 13:07:49.035338 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 202634us
I20260709 13:07:49.027374 25656 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.054208 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.025019 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:49.020464 25809 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.023412 25775 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:49.055411 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 190785us
I20260709 13:07:49.034560 25096 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:49.047192 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 225784us
I20260709 13:07:49.037237 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:49.035348 25708 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:49.060052 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.058275 22751 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:49.043931 25727 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:49.065750 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:49.066365 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:49.068668 23444 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
W20260709 13:07:49.072753 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 47851us
I20260709 13:07:49.072994 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.073661 22749 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:49.074175 25816 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:49.074687 25816 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:49.075467 25816 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.077841 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:49.079174 25812 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:07:49.079335 23453 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:49.080576 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:49.082264 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.082986 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.083156 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1529) took 1029 ms. Trace:
I20260709 13:07:49.083528 22829 rpcz_store.cc:276] 0709 13:07:48.053646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.288468 (+234822us) service_pool.cc:224] Handling call
0709 13:07:49.082991 (+794523us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b5400231ed4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:49.083105 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600395136}
I20260709 13:07:49.063126 22827 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:49.085120 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:49.085839 25815 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:07:49.086225 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50948us
I20260709 13:07:49.086642 25249 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.057514 25809 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:49.087556 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1532) took 1013 ms. Trace:
I20260709 13:07:49.087908 22828 rpcz_store.cc:276] 0709 13:07:48.074043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.218712 (+144669us) service_pool.cc:224] Handling call
0709 13:07:49.087396 (+868684us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54002c9a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:49.087505 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1587793792}
I20260709 13:07:49.088410 25661 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:49.088797 25661 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:49.089145 25661 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:49.035521 25743 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.091811 25743 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.091878 25713 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:49.092720 25741 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:49.062835 25593 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:49.094424 23449 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:07:49.035549 25707 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.101279 23447 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:49.103345 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:49.103914 22903 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:49.106611 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.107758 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:49.108115 22828 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:49.109139 22822 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:49.086236 25815 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:49.085543 25808 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:49.110533 22749 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:07:49.098474 25682 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:07:49.103201 25762 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.113335 22831 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:49.109508 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.119483 22823 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:49.123580 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:49.120124 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:49.122599 22824 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:49.121235 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:49.120848 22828 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:49.070163 25775 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.130874 22822 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:49.132884 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:49.127836 25451 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:49.122409 25682 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:49.130333 25789 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:49.137265 25451 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:49.137502 25682 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:49.137813 25451 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:49.138525 25682 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.139739 25682 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:49.143038 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 904) took 2816 ms. Trace:
I20260709 13:07:49.143417 23450 rpcz_store.cc:276] 0709 13:07:46.326245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.326366 (+   121us) service_pool.cc:224] Handling call
0709 13:07:46.326670 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.335750 (+  9080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.335793 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.336285 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:07:48.720736 (+2384451us) spinlock_profiling.cc:243] Waited 7.67 ms on lock 0x7b180012f400. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:07:49.023064 (+302328us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:49.023107 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.023174 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:49.142753 (+119579us) raft_consensus.cc:1685] finished
0709 13:07:49.142772 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.142962 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":17635200,"thread_start_us":299570,"threads_started":1,"wal-append.queue_time_us":2769393}
I20260709 13:07:49.145923 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:49.147418 25789 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:49.147966 25824 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:49.148398 25824 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:49.148907 25824 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:49.148355 25789 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.122884 25721 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:07:49.153610 25808 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:49.153716 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:49.155539 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48997.9us
I20260709 13:07:49.156052 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:49.135018 25822 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.131999 23416 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.162359 22749 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:49.155987 25793 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:49.164732 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.164248 25822 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.171489 23446 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:49.172704 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.168970 25451 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.170639 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.174896 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.176981 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.177829 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:49.173733 25727 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:49.179677 25422 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:49.179881 25661 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.173671 25679 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:07:49.182673 25822 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:49.172158 25810 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:49.134547 25484 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.191758 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.195158 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:49.192435 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:49.192732 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.188671 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.169391 25708 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:07:49.205827 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.206545 22752 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:49.207232 23448 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:07:49.208037 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:49.208153 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.200690 25820 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:49.198681 25822 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:49.205863 22747 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:49.209398 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:49.213105 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.213580 22750 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:49.214069 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:49.214973 25775 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:49.198839 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.199194 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.199927 22902 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:07:49.216167 22902 pending_rounds.cc:85] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 5
I20260709 13:07:49.216475 22902 pending_rounds.cc:107] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:07:49.216753 22902 raft_consensus.cc:2889] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:49.210189 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.217887 22902 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:49.209268 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.186908 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.201458 22901 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:49.201689 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:49.204133 23444 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:49.225734 22904 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:49.226616 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.226965 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.228744 22905 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:07:49.229246 22905 pending_rounds.cc:85] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 6
I20260709 13:07:49.229547 22905 pending_rounds.cc:107] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:07:49.229844 22905 raft_consensus.cc:2889] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:49.234287 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.177866 25699 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:49.226554 25096 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:07:49.235947 25699 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.209867 25661 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.236282 25451 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:49.238018 25707 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.157348 25815 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.239598 25815 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:49.242208 25797 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:49.242604 25797 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:49.243419 25797 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.244546 25553 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:49.238025 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.248790 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:49.249447 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.252804 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.249358 23445 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:49.262645 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.264473 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.207985 25656 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:49.265573 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 78733.1us
I20260709 13:07:49.268239 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.269472 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:49.270555 22831 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:07:49.270951 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.271292 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.210971 25484 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.282105 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.226321 25741 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:49.285377 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:49.216020 25809 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.287351 22827 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:49.252364 25808 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:49.236313 25699 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:49.268934 25682 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:07:49.280723 25721 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:49.266607 25820 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:49.266065 25797 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:49.283855 25825 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:07:49.270772 25451 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:49.294117 25809 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.289268 25810 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.283912 25643 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:49.289484 25096 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:49.307974 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:49.310643 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.311161 22905 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:49.311264 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:49.323879 22898 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:49.324198 22831 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:49.311965 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:49.288223 25808 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:07:49.317646 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:49.293355 25593 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:49.327196 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.294615 25789 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:49.294310 25711 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:07:49.337249 22750 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:49.338404 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:49.323609 25096 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.343413 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:49.345854 22822 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:49.346896 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:49.347649 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.349308 22748 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:49.357396 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.348017 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.333931 25593 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.360491 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.334102 25603 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:49.333537 25789 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:49.361435 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.368623 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.364398 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.363869 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.359866 25821 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:49.370573 25821 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:49.370826 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.361954 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.331216 25681 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:07:49.376504 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:49.362321 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.373872 22751 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:07:49.377760 22898 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:07:49.378113 22898 pending_rounds.cc:85] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 6
I20260709 13:07:49.378326 22898 pending_rounds.cc:107] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:07:49.378538 22898 raft_consensus.cc:2889] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:49.370757 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.361240 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:49.377796 22822 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
W20260709 13:07:49.392133 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 47222.1us
I20260709 13:07:49.392688 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.360973 25096 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:49.395453 22797 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.396169 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.400029 22752 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:49.365499 25686 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:49.401782 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:49.369200 25789 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:49.409857 22797 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.375011 25817 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:49.370960 25821 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:49.379642 25801 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.418232 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.425808 22752 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
W20260709 13:07:49.420318 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 139004us
I20260709 13:07:49.385640 25593 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.424154 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.427970 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.424918 25573 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:07:49.391764 25176 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:07:49.419004 25686 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:49.396786 25495 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:07:49.418773 25743 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:49.430902 25675 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:07:49.400454 25765 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:49.397212 25730 raft_consensus.cc:2764] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:07:49.507536 22905 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:49.421789 25707 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:49.433725 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:49.440497 22827 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
W20260709 13:07:49.455868 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.508536 22797 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.424093 25817 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.420171 25711 consensus_queue.cc:1243] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:07:49.512837 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 79175us
I20260709 13:07:49.513659 22797 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.518187 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.520081 22899 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:49.522480 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.523028 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:49.524816 22752 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:49.524340 25681 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:07:49.530614 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:49.530997 25715 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.531029 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.531515 25546 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.436578 25686 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:49.532025 25715 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:49.528086 25730 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:49.532346 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.533435 25578 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:49.534248 25715 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:49.521530 25721 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:07:49.538708 25729 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:49.539181 25729 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.434337 25593 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.527477 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:49.522331 25708 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:49.528385 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.401047 25826 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.128s
I20260709 13:07:49.534363 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.527369 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.542488 25643 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:49.542598 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.543617 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:49.543977 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.544483 25573 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:49.544679 22797 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.507218 25720 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:49.437996 25675 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:49.546217 25675 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:49.546407 25715 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:49.546469 25708 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:49.547473 22822 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:49.547147 25675 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.526006 25603 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:49.548503 25603 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.548880 25603 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:49.550597 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:49.550848 22822 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:49.551292 22823 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:49.554644 22748 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:49.556712 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.433434 25682 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:49.522176 25817 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:49.428901 25451 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
I20260709 13:07:49.431285 25741 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.128s
I20260709 13:07:49.432582 25484 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 13:07:49.444128 25824 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.565680 25656 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.571694 25686 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.574142 25643 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:49.577800 25832 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:07:49.560475 25661 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:49.616171 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 222333us
I20260709 13:07:49.618558 23445 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:07:49.567987 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49911.8us
I20260709 13:07:49.594933 25834 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:07:49.603657 25789 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.606329 25762 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:49.607772 25809 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:49.611918 25686 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:49.643510 25809 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:49.637176 25762 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:49.608850 25699 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.639583 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.640286 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:07:49.641424 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 95981.8us
I20260709 13:07:49.653790 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:49.641615 25817 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.596616 25808 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:49.648540 22753 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:49.625214 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.654789 22756 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:49.624323 25793 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:07:49.639807 25824 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.635267 25573 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:07:49.647244 25346 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:49.656240 25816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.001s	sys 0.011s
I20260709 13:07:49.649179 25687 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.687319 22900 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:07:49.635439 25825 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:07:49.642704 25727 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:07:49.683270 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.637642 25096 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:49.684966 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67972.9us
I20260709 13:07:49.655504 25808 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:49.696784 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.687497 25810 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:49.710186 22827 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:07:49.686605 25816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.002s	sys 0.011s
I20260709 13:07:49.706200 25814 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:49.690119 25346 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.720398 25675 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:49.721089 25346 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.721424 25817 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:49.722362 25841 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:07:49.730104 22830 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:49.685916 25844 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:49.741873 22755 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:49.743800 22898 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:07:49.689236 25451 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:07:49.746691 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:49.740834 25815 consensus_meta.cc:287] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.000s
I20260709 13:07:49.754794 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:49.741144 25838 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:07:49.695852 25687 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.732366 25668 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:07:49.761723 22823 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:07:49.762135 22823 pending_rounds.cc:85] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 7
I20260709 13:07:49.762367 22823 pending_rounds.cc:107] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:07:49.762674 22823 raft_consensus.cc:2889] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:49.761152 25844 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:49.765193 25844 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:49.732527 25711 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:07:49.749691 25299 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.781124 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.781936 22755 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:49.756628 25495 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.697751 25809 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.756935 25825 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.777726 25810 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:49.745188 25821 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.711707 25762 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.782907 25656 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:49.806488 25727 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:49.806877 25727 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:49.797118 25789 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:49.779332 25808 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.810523 25808 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:49.810824 25096 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.798388 25687 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:49.797188 25851 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:07:49.799975 25603 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:49.816937 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.817955 23449 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:07:49.801281 25801 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:49.819439 25801 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:49.820243 25801 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.757151 25850 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:07:49.822306 25801 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:49.822407 25850 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:49.822979 25850 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:49.823927 25850 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.825735 25699 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.829514 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.787385 25495 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.833876 25603 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:49.835136 25495 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.837301 25661 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:07:49.844883 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.935160 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 238437us
I20260709 13:07:49.936231 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:49.937012 25495 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:07:49.939062 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 211847us
I20260709 13:07:49.793357 25825 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.948109 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.951514 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.933836 25850 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:49.951437 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.958530 22755 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:49.922583 25810 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:49.954980 22752 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:49.955722 22756 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:49.964480 22756 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:49.956375 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:49.956490 25852 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:07:49.961840 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.966303 22753 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:49.807693 25727 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.962138 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:49.966363 22747 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:49.962984 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:49.963711 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.965256 22756 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.974409 22756 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:49.978936 22755 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:49.979877 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:49.981071 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 43690.8us
I20260709 13:07:49.981765 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:49.985821 22900 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:49.985948 25852 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.986516 25852 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.987016 25687 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:49.986960 25668 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:49.987529 25687 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:49.987883 25668 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:49.988217 25668 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:49.988593 25852 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:49.988369 25687 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:49.989705 25687 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:49.993471 25828 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.651s	user 0.002s	sys 0.005s
I20260709 13:07:49.994428 25689 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:49.995707 25495 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:49.996716 25495 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:49.996484 25859 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:49.997089 25495 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:49.997637 25299 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:49.998063 25299 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:49.998325 25495 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:49.998850 25299 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.000003 25299 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:50.000509 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:50.002851 25789 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:50.001972 25249 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:49.996232 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:50.015182 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:50.003999 25249 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:50.017655 23445 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:50.018456 25249 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.019583 25249 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:50.019738 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:50.020148 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:50.020376 23451 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:50.010864 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:49.785173 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:49.973548 25656 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:50.004240 25849 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:07:50.025287 22652 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 337) took 11272 ms (client timeout 15000 ms). Trace:
W20260709 13:07:50.029182 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47487.5us
I20260709 13:07:50.041967 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:50.009331 25775 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:50.029341 25622 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.043074 25775 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:50.043573 25622 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.044011 25775 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.045120 25775 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:50.045549 25622 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:50.045938 25853 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:50.046363 25853 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.046711 25853 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:50.025636 22650 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.048951 22650 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.063585 22904 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:50.065130 22831 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:07:50.040078 25721 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:07:50.070039 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:50.070612 22822 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:50.073947 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.036119 25495 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:50.091086 22650 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 4, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:50.055333 25801 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:50.034097 25825 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:50.030220 25689 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:50.102000 25689 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.037827 25826 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:50.104302 25546 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:50.104853 25693 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:50.078722 23451 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
W20260709 13:07:50.083369 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 71761us
I20260709 13:07:50.107012 22897 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:50.108008 22831 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:50.083256 25176 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:07:50.109483 22828 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:50.109684 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:50.110153 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:50.107745 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:49.989033 25668 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.085299 25857 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:50.123365 25668 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:50.085675 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:50.039783 25817 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.084959 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:50.090324 25762 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:50.039780 22652 rpcz_store.cc:269] 0709 13:07:38.753140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.759055 (+  5915us) service_pool.cc:224] Handling call
0709 13:07:50.025233 (+11266178us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:43.575447 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:43.830533 (+255086us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:43.838533 (+  8000us) write_op.cc:435] Acquired schema lock
0709 13:07:43.843955 (+  5422us) tablet.cc:662] Decoding operations
0709 13:07:43.906439 (+ 62484us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:43.906531 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:43.906555 (+    24us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:07:43.907618 (+  1063us) tablet.cc:701] Row locks acquired
0709 13:07:43.907667 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:07:43.907821 (+   154us) write_op.cc:270] Start()
0709 13:07:43.907968 (+   147us) write_op.cc:276] Timestamp: P: 1783602463907758 usec, L: 0
0709 13:07:43.907991 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:07:43.908651 (+   660us) log.cc:844] Serialized 4982 byte log entry
0709 13:07:44.359661 (+451010us) op_driver.cc:464] REPLICATION: finished
0709 13:07:44.479970 (+120309us) write_op.cc:301] APPLY: starting
0709 13:07:44.480114 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:44.480448 (+   334us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:44.480614 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:44.662252 (+181638us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:44.662489 (+   237us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:07:44.662538 (+    49us) write_op.cc:312] APPLY: finished
0709 13:07:44.662802 (+   264us) log.cc:844] Serialized 89 byte log entry
0709 13:07:44.662965 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:44.663109 (+   144us) write_op.cc:454] Released schema lock
0709 13:07:44.663155 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:44.663505 (+   350us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:44.663531 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":18644,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":84590,"apply.run_cpu_time_us":4473,"apply.run_wall_time_us":190531,"num_ops":12,"prepare.queue_time_us":204381,"prepare.run_cpu_time_us":49373,"prepare.run_wall_time_us":477987,"raft.queue_time_us":51304,"raft.run_cpu_time_us":36659,"raft.run_wall_time_us":56133,"replication_time_us":451526,"thread_start_us":299682,"threads_started":4,"wal-append.queue_time_us":343826,"wal-append.run_cpu_time_us":9566,"wal-append.run_wall_time_us":1469719}]]}
I20260709 13:07:50.090631 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:50.139472 22749 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:50.111922 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:50.113312 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:50.141664 22831 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:50.093256 22650 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 5, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 5 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:50.115064 23445 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:50.118409 22828 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:50.122349 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:50.149902 22828 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
W20260709 13:07:50.151646 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 66760us
I20260709 13:07:50.153108 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:50.133597 22822 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
W20260709 13:07:50.135917 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.157739 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:50.058049 25622 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:07:50.156821 22650 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:50.163722 22650 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.166466 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:50.167423 23445 raft_consensus.cc:2410] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:07:50.172868 22650 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 4 to 7, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:50.111641 25859 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:07:50.154639 25455 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:50.197145 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:50.200373 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:50.065526 25810 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.168524 25422 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:50.219485 25297 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:50.219975 22650 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 5 to 8, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 8 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:50.145401 25817 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.223694 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:50.231849 22650 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 3, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 3 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.232807 23448 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:50.233486 22650 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 5 to 6, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 6 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.235044 22650 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.236438 22650 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.226009 25553 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:50.238157 25553 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.238189 22650 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 3 to 5, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 5 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.240154 25553 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:50.240728 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:50.241161 22826 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: [5.6-6.7]
I20260709 13:07:50.239738 22650 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 5 to 8, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 8 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.171329 25713 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:07:50.253132 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:50.202930 25821 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:50.259979 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 101318us
I20260709 13:07:50.225023 25686 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:50.256016 23448 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:50.266088 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:50.260955 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:50.277666 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:50.205585 25826 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:07:50.342201 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:50.279119 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:50.249907 25810 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:50.337482 22650 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 3 to 4, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.212728 25793 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:07:50.351641 22650 catalog_manager.cc:5697] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 4, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 4 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:50.353510 22650 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:50.279728 25249 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:50.263105 25686 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:50.358429 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:50.368721 22903 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:07:50.369889 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 90760.2us
I20260709 13:07:50.370796 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:50.261476 25401 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:07:50.337106 25816 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:50.341130 25850 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:50.340909 25821 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:50.376485 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:50.401058 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:50.375574 22650 catalog_manager.cc:5697] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index -1 to 5, VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added, no longer has a pending config: VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65), VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66), VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67), NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 5 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:50.410140 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:50.411356 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:50.376124 25249 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:50.414037 22650 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 3, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:50.403635 25801 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:50.429702 25681 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:50.429913 25861 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:07:50.430277 25681 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.431918 25711 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:50.399509 25675 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:07:50.435556 25729 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:50.432386 25711 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:50.438609 25809 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:50.438401 25711 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.439417 23453 raft_consensus.cc:2410] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:07:50.432394 25681 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:50.416932 25789 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:50.394461 25816 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:50.527498 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.541831 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:50.543051 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:50.525012 25817 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:50.570549 25874 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:07:50.571859 25871 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:07:50.501302 25853 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.568642 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:50.582091 25873 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:07:50.589020 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 177736us
I20260709 13:07:50.570426 25825 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:50.625279 22900 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:07:50.590677 25708 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:50.626854 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:50.592754 25825 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:07:50.635218 25862 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 13:07:50.639680 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.646656 25824 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:50.647354 25824 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:50.647862 25824 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:07:50.652776 25881 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:07:50.648661 25824 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:50.657775 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 255352us
I20260709 13:07:50.641387 25553 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:50.657935 25816 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:50.662812 25249 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:50.663228 25249 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.663440 25656 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:50.663573 25249 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:50.663878 25656 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:50.665336 25801 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:50.700649 25801 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:50.704730 25822 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:50.705225 25822 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:50.706532 25879 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.074s	user 0.001s	sys 0.001s
I20260709 13:07:50.707250 25822 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:50.713523 25299 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:50.714051 25299 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:50.715047 25299 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.716480 25299 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:50.664690 25656 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.721810 25727 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:50.660701 25762 consensus_meta.cc:287] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 13:07:50.678560 23445 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:50.679162 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:50.724639 25593 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:50.725004 23450 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:07:50.727655 25826 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:50.728349 25826 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.729971 25815 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:50.730345 25815 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:50.730721 25826 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:50.731189 25815 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.732617 25815 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:50.729974 25593 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:07:50.684376 25686 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:50.685071 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.739106 25699 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:50.739228 25693 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:50.739552 25699 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:50.739806 25693 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.740365 25699 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.741839 25693 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:50.688586 25495 consensus_meta.cc:287] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.596s	user 0.003s	sys 0.004s
I20260709 13:07:50.692842 22749 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:50.693305 25825 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:07:50.699601 25868 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:07:50.700206 25853 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:50.749626 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.753347 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:50.753654 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:50.753944 22897 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:50.754287 22896 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:50.750849 25346 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:50.755560 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:50.757400 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:50.758913 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:50.760751 25346 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.762967 25888 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:50.764145 25887 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:50.751497 25699 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:50.764454 25888 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:50.750855 25822 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:07:50.771241 25888 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:50.772717 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 559) took 1263 ms. Trace:
I20260709 13:07:50.773087 23449 rpcz_store.cc:276] 0709 13:07:49.509620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.509769 (+   149us) service_pool.cc:224] Handling call
0709 13:07:49.524848 (+ 15079us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:49.818398 (+293550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.772496 (+954098us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54000b2054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:50.772659 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672772480}
I20260709 13:07:50.764603 25887 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.764693 25346 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:50.810804 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:50.796976 25887 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:50.823210 25890 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.824450 25249 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.830470 25401 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:07:50.834187 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.834764 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:50.836740 25808 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:50.837364 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:50.819211 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:50.856456 23449 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:50.841586 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:50.841034 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:50.842244 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:50.836552 25801 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.848670 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:50.850557 22754 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:50.857381 22828 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:50.796821 25775 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:07:50.852965 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:50.852367 22756 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:50.857751 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:50.858467 22749 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:50.861650 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:50.864490 22752 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:50.861027 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:50.868892 25845 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:50.869426 25845 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.850006 25401 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:50.849959 25689 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:50.880126 22903 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:50.877729 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:50.853577 25455 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:50.877507 25546 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:50.874424 25826 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:50.855656 25578 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:50.854126 25890 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.884335 25675 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:50.884850 25578 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.883227 25546 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:50.886325 25546 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.887827 25546 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:50.888458 25249 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:50.893968 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:50.894833 23449 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:50.859076 25808 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.891795 25730 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:07:50.904584 25870 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.346s	user 0.005s	sys 0.000s
I20260709 13:07:50.911226 23449 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:50.911777 25801 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:50.913456 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:50.890348 25455 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.919221 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:50.919679 25808 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:50.893765 25721 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:07:50.920400 25455 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:50.920665 25727 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:50.920976 25721 raft_consensus.cc:3055] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:50.921243 25727 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:50.921514 25721 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:50.883376 25689 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.922248 25727 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.922407 25721 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:50.923583 25721 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:50.924074 25689 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:50.905083 25730 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:50.927328 25730 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:50.930374 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:50.930927 22750 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:50.932679 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:50.933398 22896 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:50.931998 25656 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:50.890645 25843 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:07:50.935714 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:50.943532 22750 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:50.942144 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:50.927109 25888 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.951727 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:50.949071 25729 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:07:50.946321 25741 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:50.953547 25699 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:50.953884 25741 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:50.954145 25699 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:50.954559 25741 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:50.948908 25743 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:50.954643 25699 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:50.982455 25887 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.985025 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:50.985569 22748 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:50.987023 25401 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:50.988263 25401 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:50.990291 25890 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:50.990831 25888 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.005971 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:50.991783 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.006880 22749 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:50.992388 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:51.005952 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:50.994720 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.005133 22748 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:50.996084 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.007115 22903 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:07:50.996886 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:50.995913 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:51.009202 22826 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 9: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:51.000943 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.009701 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:51.009977 22754 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:50.999584 22828 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:07:51.000609 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:51.005920 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:51.011644 22896 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:51.009052 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:51.007905 22822 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:51.006723 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.022260 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:51.022305 22902 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:51.010975 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:51.024706 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:51.029563 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.031479 25346 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:51.032084 25573 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:51.032450 25573 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:51.032783 25573 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:51.033666 25578 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.035183 25825 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:51.035526 25299 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:51.036373 25455 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:51.036800 25455 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.037107 25455 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:51.038282 25686 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:51.039822 25845 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.021531 25711 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.041867 25810 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:51.041788 25711 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.043298 25810 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:51.043679 25810 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:51.044132 25711 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.044519 25810 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.045717 25810 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:51.048959 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:51.049544 23445 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:07:51.051792 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:51.054141 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.054646 22902 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:51.055769 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:51.056759 22896 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:51.062037 25890 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:51.067305 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:51.070345 22901 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
W20260709 13:07:51.072865 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 63884.3us
I20260709 13:07:51.073877 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:51.078651 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:51.075282 25578 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:51.020738 25887 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.041213 25573 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:51.088855 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 89466.2us
I20260709 13:07:51.093381 25762 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:51.076982 25727 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.077816 25546 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:51.077311 25888 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:51.101118 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.103250 25834 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:07:51.104837 25727 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.104691 25857 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.106499 25857 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.106789 25730 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.108994 25730 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.114920 25762 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:51.115914 25762 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.118019 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.118755 22749 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:51.120736 22871 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:51.124380 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:51.124711 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:51.125387 22829 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:51.126709 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:51.127391 22749 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:51.129982 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:51.127923 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.117304 25727 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:51.134060 22754 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:51.117223 25573 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.109962 25455 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.135954 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:51.137063 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:51.135826 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.138208 25546 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:51.142935 22902 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:51.075795 25699 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.143849 25546 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.146992 25699 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.148965 22754 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:51.149659 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:51.150501 23450 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:51.110580 25888 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.156407 25593 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:51.156548 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:51.156948 25593 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.157403 25593 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:51.125386 25857 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.163450 22896 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:51.171432 22826 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:51.153905 25888 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:51.141776 25727 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:51.175769 22899 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:51.167160 22829 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:51.183662 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:51.189045 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:51.189596 22901 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:07:51.175976 25834 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.161790 25859 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:07:51.187731 25727 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.208882 25727 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:51.210446 22750 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:07:51.210829 22750 pending_rounds.cc:85] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 6
I20260709 13:07:51.211038 22750 pending_rounds.cc:107] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:07:51.211263 22750 raft_consensus.cc:2889] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:51.197743 25899 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:07:51.220834 25715 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:51.222018 25743 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.222509 25743 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.222774 25715 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.226191 25455 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.205791 25834 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.231957 25834 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.233135 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:51.234757 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:51.190621 25801 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:51.208014 25741 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.190222 25675 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:07:51.192375 25730 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:51.236788 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.256335 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:51.256971 23445 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:07:51.263207 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:51.263967 22901 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:51.235566 23450 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:07:51.243090 25801 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:51.265957 23449 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:51.221239 25727 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.253782 22752 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:51.268440 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:51.269558 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:51.285110 22901 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:07:51.285306 23445 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:07:51.288103 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:51.250053 25834 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:51.277854 25693 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:51.297761 25902 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:07:51.298172 25573 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:51.298684 25573 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:51.299983 25573 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.279064 25730 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:51.273384 25451 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:07:51.319861 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:51.331887 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:51.330071 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:51.349648 23450 raft_consensus.cc:2410] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:07:51.349916 22749 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:07:51.316884 22897 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:07:51.349138 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.352360 22752 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:51.353617 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:51.354285 22829 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:51.303175 25741 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.364051 22797 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:51.368785 22901 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:51.291031 25834 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:51.292807 25689 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:51.378047 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:51.378690 22824 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:51.318738 25743 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.316627 25727 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:51.375044 22749 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:51.292447 25721 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:51.376317 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:51.383236 22871 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:51.301247 25593 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.301190 25693 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.399369 25593 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.400106 25693 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.400290 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:51.400883 22901 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:51.400880 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:51.405465 22752 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:51.411423 25730 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.320708 25825 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:07:51.375520 25834 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:51.421088 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.434417 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:51.435549 22826 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:51.302927 25888 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.438331 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:51.301613 25801 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.394675 25708 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:51.389832 25720 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:51.390690 25859 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:51.394649 25689 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:51.458868 25708 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.449666 25743 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.449724 25762 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:51.441345 22749 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:51.475625 22901 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:51.476285 22752 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:51.476641 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:51.479715 22824 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:51.471407 25708 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:51.475474 25762 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.472151 25825 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:51.473278 25720 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:51.473083 25859 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:51.485987 25729 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:51.489589 25603 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.516968 25720 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.499922 25346 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:51.497820 25687 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:51.548554 25346 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.548646 25687 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.550916 25346 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.551115 25687 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.491367 25762 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:51.560738 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:51.474332 25743 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.481813 25727 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:07:51.556002 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:51.556804 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:51.560030 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.565855 22750 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:51.565024 23445 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:51.561662 22824 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:51.564883 23449 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:51.492940 25401 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:07:51.521847 25729 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:51.493743 25825 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.586606 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:51.589628 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:51.578588 25176 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.592007 25859 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.597764 25299 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:51.546331 25603 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.611950 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.614161 25808 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.599174 25687 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:51.625384 22897 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:51.616669 25299 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:51.547145 25689 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.631037 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:51.635092 22897 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:51.624303 25797 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.629113 25715 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.629030 25687 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:51.616168 25176 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.634560 25888 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.635155 25299 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:51.665586 22897 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:51.645398 25825 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:51.671789 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:51.625375 25808 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.646714 25682 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:07:51.661497 25687 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:51.670212 25656 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.681448 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.685822 22752 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:51.686725 22753 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:51.655903 25797 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.678697 25888 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:51.662271 25729 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.707242 25808 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.707670 25721 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:51.708117 25721 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:51.709491 25603 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.686625 25656 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.704234 25888 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:51.723277 25708 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.714270 25743 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.727252 25888 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.790405 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:51.789932 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.791847 22824 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:51.794682 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:51.819120 25176 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.828428 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.834847 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.835819 22899 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:51.838068 25455 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.838835 25455 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.845661 22824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43622
I20260709 13:07:51.853485 22824 raft_consensus.cc:606] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Received request to transfer leadership
I20260709 13:07:51.850195 25682 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:51.852281 25656 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.855863 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:51.856806 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.857717 22751 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:51.857738 22749 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:51.856845 22826 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:51.848872 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:51.865702 25708 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.859323 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.867138 22901 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:51.868042 22797 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:51.858639 25656 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:51.873548 25656 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.873796 25451 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:51.874498 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.864054 25687 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.874010 25656 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:51.847709 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:51.874435 25451 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:51.887908 25920 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 13:07:51.888238 25916 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:07:51.891939 25451 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:51.894423 25921 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:07:51.902581 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:51.908829 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 91382.2us
I20260709 13:07:51.910652 25797 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.918865 22899 raft_consensus.cc:2368] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
W20260709 13:07:51.907874 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60760.2us
I20260709 13:07:51.922720 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:51.923559 22752 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:51.905169 25715 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:51.932189 25715 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.932595 25715 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:51.939047 25852 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.939891 25852 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.940992 25859 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.942915 25850 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.943720 25850 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.946102 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.946923 22896 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:51.950163 25687 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:51.951073 25495 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:51.951527 25495 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:51.951956 25495 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:51.953140 25910 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.329s	user 0.000s	sys 0.003s
W20260709 13:07:51.938292 25907 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.555s	user 0.006s	sys 0.008s
I20260709 13:07:51.960608 25821 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.941656 25859 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.967247 25859 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:51.968448 25821 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.970571 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.971151 25821 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:51.971328 22896 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:07:51.972082 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:51.972985 22902 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:51.974702 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.975824 22753 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
W20260709 13:07:51.981199 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 96197.8us
I20260709 13:07:51.981931 22871 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:51.983338 25686 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:51.983749 25686 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.984144 25686 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:51.987949 25852 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:51.990607 25845 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:51.991088 25845 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.991463 25845 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:51.995416 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:51.996146 22753 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:51.996940 25451 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:51.999868 25451 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.001123 25721 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.003814 25721 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:52.004843 22752 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:52.007386 25843 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:52.008129 25843 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:52.009451 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 51383.5us
I20260709 13:07:52.008270 25814 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.016166 25814 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.017447 25455 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.017850 22754 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:52.018865 25814 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.023828 25715 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:51.817040 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:52.040403 22871 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:52.044533 25852 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:52.044978 25852 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.045497 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:52.012213 25656 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.052014 22824 raft_consensus.cc:2368] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:52.052558 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:52.052932 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:52.052989 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.053982 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:52.055950 25762 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.057718 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.060002 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:52.048440 25852 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:52.067327 22828 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43622
I20260709 13:07:52.049804 22797 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:52.100874 25834 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.651s	user 0.013s	sys 0.004s
I20260709 13:07:52.111891 22828 raft_consensus.cc:606] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Received request to transfer leadership
I20260709 13:07:52.111740 22896 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:52.090052 22826 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:07:52.113443 22902 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:52.113543 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.114264 22830 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:52.116182 22753 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:52.117873 22831 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:52.127744 22827 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:07:52.126953 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 77213.2us
I20260709 13:07:52.090052 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:52.039886 25686 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.132058 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:52.132725 22896 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:07:52.138511 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:52.139369 25689 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:52.140089 25689 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:07:52.142215 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 73562us
W20260709 13:07:52.145816 22638 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 4d8a8f3cf129492eb679da24703f5b84 already in progress [suppressed 8 similar messages]
I20260709 13:07:52.147257 22903 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:52.148319 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:52.149969 22827 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:52.124713 25814 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.157202 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:52.122797 25834 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.162736 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:52.163419 22897 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:07:52.163745 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:52.163980 25834 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.164458 22902 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:52.164651 25775 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.165211 25775 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.163486 22900 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:07:52.167629 25775 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:52.167547 25299 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.171521 25762 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.172699 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:52.173539 23449 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:52.164009 25814 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.177009 25814 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:52.158787 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:52.156708 25843 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:52.186363 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53687.8us
I20260709 13:07:52.189616 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:52.189934 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:52.194119 25455 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:52.195252 25845 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.141203 25689 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.204898 25852 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.198670 22896 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:52.201673 25455 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.202055 22750 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:52.202232 22900 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:52.214876 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:52.215917 23449 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:52.216522 25299 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:52.213696 25821 raft_consensus.cc:993] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:52.212214 25455 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:52.217784 25857 raft_consensus.cc:1081] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:52.218572 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:52.219657 23452 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:07:52.221606 25927 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.123s	user 0.002s	sys 0.000s
I20260709 13:07:52.221789 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:52.225692 25850 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:52.227692 22752 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:07:52.229063 25656 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.212749 22830 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:52.236542 23445 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:52.240182 25495 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.242880 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:52.243571 23445 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:52.244433 25821 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.245147 25821 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.245464 22903 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:52.247202 25821 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.247771 25715 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.248875 25845 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.252549 23445 raft_consensus.cc:2410] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:07:52.256062 23449 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:52.257230 22794 leader_election.cc:400] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:52.257743 22794 leader_election.cc:403] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:52.258540 25857 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:52.258687 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:52.259666 25857 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:52.260174 25857 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.260602 25857 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:52.267681 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 60819us
I20260709 13:07:52.268409 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:52.271059 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:52.271528 25844 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:52.272061 25844 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:52.272691 25844 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:52.273406 25686 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:52.274504 25852 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:52.276155 25495 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:52.272019 23449 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
W20260709 13:07:52.278667 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 65902.2us
W20260709 13:07:52.281066 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 81507us
I20260709 13:07:52.283970 23453 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
W20260709 13:07:52.284452 22638 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 138764us
I20260709 13:07:52.289227 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:52.290076 25299 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.006s
W20260709 13:07:52.295046 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48825.2us
I20260709 13:07:52.294436 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:52.304006 25845 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:52.304849 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:52.303869 25573 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:07:52.305702 25845 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.306032 25176 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:52.306459 25176 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.306849 25176 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:52.306254 25797 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:52.307937 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:52.308918 25852 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:52.310778 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:52.314850 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.315162 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.315598 25834 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:52.315912 22822 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:52.316032 25834 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.316483 25834 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:52.317431 22749 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:52.317428 22868 leader_election.cc:400] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:52.321663 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:52.321839 25455 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.322327 22752 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:52.323071 25852 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:52.323362 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:52.323242 25834 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.324100 22750 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:52.324291 22749 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38488
I20260709 13:07:52.324594 25299 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:52.324836 22749 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:52.325089 25299 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:52.325335 22749 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:52.328406 22900 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:07:52.330265 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:52.330890 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.331297 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.325227 25834 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:52.331400 22900 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:52.334486 22823 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:52.336338 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:52.336028 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:52.337692 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:52.335754 25573 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:07:52.341070 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.351634 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:52.350602 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:52.353120 22896 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:52.354733 22828 raft_consensus.cc:2368] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:52.355286 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.355471 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:52.355589 25923 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.157s
I20260709 13:07:52.356208 22753 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:52.356581 22900 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:07:52.356940 25553 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:52.357384 25553 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.357740 25553 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:52.361968 22897 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:07:52.363098 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:52.363699 22899 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:52.365864 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:52.366467 22904 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:52.367123 25726 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:52.367704 25726 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:52.350108 25821 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:07:52.374030 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:52.374570 23450 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:52.375204 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:52.376062 25762 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:52.377916 25762 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:52.378419 25762 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:52.379083 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:52.379740 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:52.379766 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:52.380662 22750 raft_consensus.cc:2368] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:52.379410 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:52.381384 22754 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:52.380736 22756 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:52.380589 22751 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:52.386740 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:52.306214 25845 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.390437 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:52.353964 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:52.405498 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:52.368656 25726 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.413513 25593 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:52.415808 25593 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.416145 25593 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:52.410773 22754 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 3.
I20260709 13:07:52.392624 25455 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.327257 25857 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.340049 25299 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.411993 22654 catalog_manager.cc:5697] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 7, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 7 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:52.427886 25299 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:52.428277 25176 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.428333 22654 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:52.429874 22654 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 5, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 5 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:52.431346 25844 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.431308 22654 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.379360 25762 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.421687 22904 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:52.436873 22830 raft_consensus.cc:2368] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10.
W20260709 13:07:52.419406 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 65510.6us
I20260709 13:07:52.426222 25495 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:52.426647 22751 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:52.425653 22750 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:52.432369 22822 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:07:52.411439 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60907.5us
I20260709 13:07:52.433269 22749 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.445683 22899 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:52.437846 25857 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.449733 25495 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:52.450682 25495 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.464530 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:52.470306 22828 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43622
I20260709 13:07:52.471037 22828 raft_consensus.cc:606] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Received request to transfer leadership
I20260709 13:07:52.475164 23450 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:52.486711 22871 leader_election.cc:400] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:52.442068 25844 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:52.487893 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:52.496260 22868 leader_election.cc:400] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:52.513386 25821 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:52.502758 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:52.514312 25817 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.527796 22753 raft_consensus.cc:2368] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:52.517521 22868 leader_election.cc:403] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:07:52.444222 25553 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.518006 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:52.519421 22897 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:52.531438 22899 raft_consensus.cc:2410] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:52.534658 25553 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.535379 22829 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:52.538635 22654 catalog_manager.cc:5697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 4, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 4 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:52.489566 25859 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:07:52.542454 25845 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:52.560369 25850 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:52.572597 25852 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:52.568938 22654 catalog_manager.cc:5697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 3 to 7, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:52.590272 22822 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:07:52.600244 22650 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33010 (request call id 223) took 12017 ms (client timeout 15000 ms). Trace:
W20260709 13:07:52.600575 25932 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.174s	user 0.000s	sys 0.002s
I20260709 13:07:52.610595 23449 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:52.563782 25821 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.587400 25850 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.623205 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:52.626195 22750 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:07:52.621248 22654 catalog_manager.cc:5697] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 8, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 8 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:52.569069 25817 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:52.632995 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 136896us
W20260709 13:07:52.600821 22650 rpcz_store.cc:269] 0709 13:07:40.593422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.656857 (+ 63435us) service_pool.cc:224] Handling call
0709 13:07:52.585749 (+11928892us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:50.889705 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:50.900471 (+ 10766us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:50.900529 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:07:50.900560 (+    31us) tablet.cc:662] Decoding operations
0709 13:07:50.973519 (+ 72959us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:50.973579 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:50.973608 (+    29us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:07:50.975089 (+  1481us) tablet.cc:701] Row locks acquired
0709 13:07:50.975104 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:07:50.975237 (+   133us) write_op.cc:270] Start()
0709 13:07:50.975349 (+   112us) write_op.cc:276] Timestamp: P: 1783602470975191 usec, L: 0
0709 13:07:50.975368 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:07:50.976373 (+  1005us) log.cc:844] Serialized 7436 byte log entry
0709 13:07:51.356700 (+380327us) op_driver.cc:464] REPLICATION: finished
0709 13:07:51.644568 (+287868us) write_op.cc:301] APPLY: starting
0709 13:07:51.644681 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:51.650165 (+  5484us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:51.650315 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:51.707281 (+ 56966us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:51.707590 (+   309us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 13:07:51.707636 (+    46us) write_op.cc:312] APPLY: finished
0709 13:07:51.707986 (+   350us) log.cc:844] Serialized 132 byte log entry
0709 13:07:51.708166 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:51.708340 (+   174us) write_op.cc:454] Released schema lock
0709 13:07:51.708393 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:51.708776 (+   383us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:51.708810 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1262448,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":272876,"apply.run_cpu_time_us":5486,"apply.run_wall_time_us":64216,"num_ops":19,"prepare.queue_time_us":222270,"prepare.run_cpu_time_us":50774,"prepare.run_wall_time_us":249624,"raft.queue_time_us":137454,"raft.run_cpu_time_us":36424,"raft.run_wall_time_us":226225,"replication_time_us":381196,"thread_start_us":541156,"threads_started":4,"wal-append.queue_time_us":217651}]]}
I20260709 13:07:52.642112 22654 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 6, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 6 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:52.628831 25850 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:52.632807 25176 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.647943 22654 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 6, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 6 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.660167 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:52.669214 25741 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:52.671294 25817 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:52.669744 25775 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.672703 25821 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.672665 25775 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:52.674919 25775 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.659677 22654 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 3 to 5, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 5 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.680140 22654 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 3 to 8, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 8 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.692436 22654 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 8, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 8 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:52.695516 22750 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
W20260709 13:07:52.740398 25934 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:07:52.744500 22654 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:52.748488 22749 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:52.779083 25929 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:07:52.786365 22654 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 7, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 7 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:52.788749 22654 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.794503 22654 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:52.797712 25938 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:07:52.805915 23449 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f881fed251c4393ac053086bd572549"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:07:52.806526 23449 raft_consensus.cc:606] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:52.850793 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 362952us
I20260709 13:07:52.852407 22871 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:52.858657 22871 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:52.859874 22654 catalog_manager.cc:5697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 5, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 5 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } }
I20260709 13:07:52.863978 22654 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 9, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 9 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:52.865861 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 328144us
I20260709 13:07:52.867621 22871 leader_election.cc:400] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
W20260709 13:07:52.857455 25938 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:07:52.793630 25593 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.867239 25299 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:52.876889 25299 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:52.876055 22654 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 5 to 7, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 7 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.877781 25299 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:52.877928 22654 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.878659 25593 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.879321 25952 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:52.879830 22654 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 6, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 6 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.883231 22654 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 10, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 10 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:07:52.944497 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 58492.9us
I20260709 13:07:52.876047 25495 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.953959 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:52.875483 25726 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:07:52.953328 25852 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.978566 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:52.952091 25944 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:07:52.950907 25495 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.008682 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:53.008867 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:52.974161 25682 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.009227 22748 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:53.009991 25495 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:53.009632 25682 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.010669 22903 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:52.981743 25852 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.020918 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:53.027107 25741 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:53.027359 25852 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:53.027722 25741 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:53.029717 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:53.019069 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:53.032676 23450 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:53.032783 22748 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
W20260709 13:07:52.987490 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:52.988034 25951 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:07:52.988811 25946 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:07:52.991806 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:52.993026 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:52.993438 25950 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 13:07:53.000299 25299 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:53.005646 25176 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:53.027149 22826 raft_consensus.cc:3055] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:53.033738 22749 raft_consensus.cc:2368] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3.
I20260709 13:07:52.985842 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:53.018687 22827 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:53.051069 22826 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:53.034720 22797 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:53.048941 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57202us
I20260709 13:07:53.058661 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:53.059616 22749 raft_consensus.cc:2410] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:53.012084 25682 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:52.979930 25850 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.063884 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:53.065711 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:53.064646 22901 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:53.068646 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:53.070120 25834 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:53.070446 25721 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:53.071128 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:53.073880 23450 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:53.078197 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:53.048554 25299 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:53.053258 25656 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:53.082368 22897 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:07:53.087697 23445 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:07:53.094166 23445 pending_rounds.cc:85] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 7
I20260709 13:07:53.094489 23445 pending_rounds.cc:107] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:07:53.094822 23445 raft_consensus.cc:2889] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:53.088449 25944 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:53.095331 25299 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.099340 25721 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:53.099922 25721 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:53.071161 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:53.101742 22898 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:53.108582 25850 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:53.114022 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:53.114435 25299 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:53.114581 22899 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:53.119372 23449 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:53.070120 25176 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:53.122601 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 44407.9us
I20260709 13:07:53.132687 22899 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:07:53.133354 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:53.134609 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:53.135567 22900 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:53.137650 23450 raft_consensus.cc:2410] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:07:53.130424 25944 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:53.143033 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:53.144236 22897 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:53.100796 25721 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.147100 23445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:07:53.149251 23445 raft_consensus.cc:606] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:53.149679 23445 raft_consensus.cc:614] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:53.150653 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:53.138440 22826 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.167886 22826 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:53.169909 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 101324us
I20260709 13:07:53.177635 22898 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:53.189253 25834 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:53.189632 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 72922us
I20260709 13:07:53.215070 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:53.216117 23450 raft_consensus.cc:2410] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:53.225816 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:53.230475 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:53.231357 22899 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:53.231738 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:53.231786 25944 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.132350 25741 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.234256 25741 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:53.237687 25762 raft_consensus.cc:993] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:53.238157 25762 raft_consensus.cc:1081] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:07:53.240558 25949 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:53.243270 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:53.249759 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:53.253938 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:53.241061 25949 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.191941 25682 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:53.182900 25952 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:07:53.251706 25941 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:07:53.269625 25656 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:53.273588 25715 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:53.268040 25741 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:53.270924 25762 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:53.277510 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:53.269147 25845 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:53.182588 25850 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:07:53.272212 22797 leader_election.cc:400] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3.
I20260709 13:07:53.293260 22797 leader_election.cc:403] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:53.299458 22752 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:07:53.281363 25969 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:53.305488 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:53.306329 22752 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:53.307906 22904 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:07:53.310446 22752 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:53.313964 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:53.314689 22830 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
W20260709 13:07:53.317485 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48759.8us
I20260709 13:07:53.327724 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:53.329146 22901 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:53.330638 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:53.337297 22825 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:07:53.306080 25948 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:07:53.340191 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 83492.5us
I20260709 13:07:53.335983 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:53.341785 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68861.9us
I20260709 13:07:53.346038 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:53.316422 25176 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:53.279331 25834 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.352851 22904 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:53.326280 25969 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:53.356767 25890 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:53.357357 25969 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.358361 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:53.359247 22755 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:53.360419 25891 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:53.360889 25762 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:53.361371 25762 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:53.361755 25762 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:53.362944 25656 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.345644 25963 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:53.364786 25963 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.370458 22826 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:53.365164 25963 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:53.391942 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50868.6us
I20260709 13:07:53.393445 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:53.395546 22897 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:53.343335 25741 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:53.398283 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 110828us
I20260709 13:07:53.399395 23450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:07:53.399940 23450 raft_consensus.cc:606] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:53.400279 23450 raft_consensus.cc:614] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:53.401530 22900 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:07:53.360462 25682 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:53.403654 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:53.403333 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:53.405910 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:53.413719 25593 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:53.425498 25573 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:53.426085 25573 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:53.427026 25573 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.438696 22755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f881fed251c4393ac053086bd572549"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:56780
I20260709 13:07:53.439225 22755 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:53.439659 22755 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:53.404446 25176 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:53.394439 25797 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:07:53.405009 25682 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.485047 22749 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:53.441146 25859 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:53.477689 25741 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.483243 22897 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:53.476415 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:53.489528 25965 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:07:53.490108 25797 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:53.347124 25952 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.484761 25948 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 2. Current term is 3.
I20260709 13:07:53.486915 25682 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:53.500947 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 158559us
I20260709 13:07:53.493947 25687 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:07:53.496531 25797 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:53.511821 22719 leader_election.cc:400] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10.
I20260709 13:07:53.515997 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:53.532356 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 82219us
I20260709 13:07:53.509519 25845 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:53.561602 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:53.558830 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:07:53.562767 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 112296us
I20260709 13:07:53.565090 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:53.566655 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 125903us
I20260709 13:07:53.570007 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:53.571370 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:53.577412 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:53.578256 25495 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:53.581610 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:53.581830 22826 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:53.584267 25495 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:53.585743 25845 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.587499 25656 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:53.518643 25979 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:07:53.588757 22755 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.634953 22756 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
W20260709 13:07:53.631186 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 55316.8us
I20260709 13:07:53.638931 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:53.573999 25687 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:07:53.677109 22755 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:53.671259 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 705) took 1172 ms. Trace:
W20260709 13:07:53.685822 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 118258us
I20260709 13:07:53.679299 22896 rpcz_store.cc:276] 0709 13:07:52.481747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.481991 (+   244us) service_pool.cc:224] Handling call
0709 13:07:52.499235 (+ 17244us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:52.507161 (+  7926us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:52.507193 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:52.799839 (+292646us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b180004d4a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:07:53.042330 (+242491us) spinlock_profiling.cc:243] Waited 8.76 ms on lock 0x7b2c00013588. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:07:53.053758 (+ 11428us) log.cc:844] Serialized 52 byte log entry
0709 13:07:53.181134 (+127376us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:53.181186 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.181265 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:53.653513 (+472248us) raft_consensus.cc:1685] finished
0709 13:07:53.653533 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:53.653663 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11092,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":33427456,"thread_start_us":109593,"threads_started":1,"wal-append.queue_time_us":157763}
W20260709 13:07:53.686784 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 47939.1us
I20260709 13:07:53.654088 25495 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:53.796908 25976 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.160s
W20260709 13:07:53.799156 25987 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:07:53.566143 25797 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.835753 25986 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:53.794528 25593 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:53.801949 25979 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:53.887753 25989 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:53.837728 25859 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.891963 25859 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:53.892439 25998 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:07:53.687915 25984 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.191s
I20260709 13:07:53.824704 25495 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:53.879832 25986 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:53.876437 25686 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.877146 25976 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:53.940840 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 2022) took 1432 ms. Trace:
I20260709 13:07:53.823022 25887 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.394s
I20260709 13:07:53.918287 25989 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:53.961503 22751 rpcz_store.cc:276] 0709 13:07:52.492170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.985599 (+493429us) service_pool.cc:224] Handling call
0709 13:07:53.924294 (+938695us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b54000771d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:53.924450 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057958656}
W20260709 13:07:53.943863 25762 consensus_meta.cc:287] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.003s
I20260709 13:07:53.942777 25593 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:53.986851 25686 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.002727 25686 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:54.006181 25682 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.007704 23452 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:07:54.039918 25994 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:07:53.988497 25976 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.060369 25495 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:54.061255 25989 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.010226 25762 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.064563 25762 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:54.111145 23449 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:07:54.111822 23449 raft_consensus.cc:606] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:54.112207 23449 raft_consensus.cc:614] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:54.121287 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 447185us
W20260709 13:07:54.135895 25963 consensus_meta.cc:287] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.734s	user 0.008s	sys 0.000s
I20260709 13:07:54.136317 25963 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.137486 22898 raft_consensus.cc:2368] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:07:54.065200 25986 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.043139 25682 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:54.139799 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:54.140625 25455 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:54.150768 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:54.151739 23449 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:54.152168 23449 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:54.166482 25593 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.168427 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:54.147781 25682 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:54.169538 25682 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.170445 22825 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:54.171386 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:54.171756 23450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:07:54.172297 23450 raft_consensus.cc:606] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:54.175033 25721 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.175561 25721 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.175845 25991 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:54.176362 25991 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.169207 22827 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:54.203424 25689 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:54.203851 25689 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:54.204591 25689 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:54.456372 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.274693 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:54.458205 22827 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:07:54.275230 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:54.205410 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:54.469470 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:54.306982 23449 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.472726 23449 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:54.471776 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:54.208416 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:54.208353 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:54.208540 25924 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.457202 25797 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:54.205576 25977 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.205452 25721 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.455224 25976 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.464514 25682 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:54.204821 25991 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:54.617916 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 139447us
I20260709 13:07:54.648554 22827 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:07:54.655040 22905 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:54.655400 25952 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.647177 25721 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:54.656000 25952 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.656210 25721 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.656430 25686 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.656965 25686 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.615118 25455 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.658010 25952 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.658536 25952 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:54.658716 25762 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:54.655262 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:54.658962 25686 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:54.658900 25923 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:54.481561 25593 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:54.615599 25963 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.693426 25299 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:54.658572 25603 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:54.507579 25954 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.186s
I20260709 13:07:54.655289 25964 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.775614 25964 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.777710 25859 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.777792 25964 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.778340 25859 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.778403 25964 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:54.778923 25859 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.779065 25964 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.781009 25859 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:54.717976 25603 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.720724 25593 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:54.695351 22702 auto_rebalancer.cc:893] Removing replace marker failed: Replica 7aad11e70ca94fce8f672db05f1c28c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:07:54.804764 22702 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 4d8a8f3cf129492eb679da24703f5b84 already in progress
I20260709 13:07:54.666567 25762 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.729784 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2251) took 1129 ms. Trace:
I20260709 13:07:54.658073 25455 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:54.473942 25995 consensus_queue.cc:1175] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aad11e70ca94fce8f672db05f1c28c3" responder_term: 4 status { last_received { term: 4 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 4 index: 7 } }
I20260709 13:07:54.668678 25834 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:07:54.805150 22702 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 4d8a8f3cf129492eb679da24703f5b84 already in progress
I20260709 13:07:54.717561 25997 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.693992 25852 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.661808 25991 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.771854 23449 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:54.774571 25845 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:54.716773 25299 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:54.781454 25682 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:54.782401 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 1970) took 1700 ms. Trace:
I20260709 13:07:54.617689 25977 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.670070 25887 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:54.678519 26006 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:54.795826 25963 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:54.593225 25924 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.671471 25923 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.820030 22826 rpcz_store.cc:276] 0709 13:07:53.586852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.591914 (+  5062us) service_pool.cc:224] Handling call
0709 13:07:53.592235 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:53.667444 (+ 75209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:53.667475 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:53.670750 (+  3275us) log.cc:844] Serialized 26 byte log entry
0709 13:07:54.116093 (+445343us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:07:54.116129 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.116193 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:54.716134 (+599941us) raft_consensus.cc:1685] finished
0709 13:07:54.716165 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:54.716352 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113355,"raft.queue_time_us":47828,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5269,"thread_start_us":445004,"threads_started":1,"wal-append.queue_time_us":784083}
I20260709 13:07:54.659127 25952 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.832880 22905 rpcz_store.cc:276] 0709 13:07:53.082081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.125026 (+ 42945us) service_pool.cc:224] Handling call
0709 13:07:53.125357 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:54.655497 (+1530140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.782172 (+126675us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54003e9754. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:54.782343 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3092085504}
I20260709 13:07:54.842185 23450 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:07:54.838615 22795 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.017s	sys 0.012s
I20260709 13:07:54.830806 25845 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:54.843101 22795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.017s	sys 0.012s
I20260709 13:07:54.834442 25966 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:54.836943 26012 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.838352 25741 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.859050 25850 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:54.859715 25964 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.860304 26013 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.861145 25960 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.890245 25960 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.890671 25762 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:54.890942 26007 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.891260 25996 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.861930 26018 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:54.862540 25726 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.864558 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:54.865975 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:54.830700 25997 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.867638 25603 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:54.867914 25923 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:54.869941 25952 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.870496 23446 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:54.872103 25299 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:54.874315 25593 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.833552 25852 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.906287 25970 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.906437 25960 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:54.854079 25954 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:54.907162 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:54.878103 25845 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.833931 25682 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.829854 25834 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.906800 25970 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.882189 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:54.834375 25991 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.889189 26013 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.880432 25966 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.846307 25687 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.895684 22904 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:54.895751 22896 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:07:54.880043 22795 connection.cc:256] RPC call timeout handler was delayed by -1.1156973659999494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:07:54.854966 25859 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:54.837143 25887 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.845522 25963 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.884814 25817 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.915393 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:54.882085 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.842975 26006 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.888001 25850 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.925515 22825 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:54.910256 25952 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:54.930462 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:54.933583 22896 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:07:54.917984 25960 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:54.887005 25741 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.941443 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:54.885066 26012 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:54.928634 25845 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:54.891502 26007 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.924642 26014 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:54.932720 25852 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.956499 22904 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:07:54.948866 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:54.899631 25726 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.897336 25996 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.984081 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:54.986243 25681 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:54.919665 25954 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.987515 25834 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:54.987900 26014 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:54.931841 25859 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.929370 25687 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.897820 26018 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:55.004215 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:54.993860 22904 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:54.945569 25952 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.004969 22825 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:54.948944 25817 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.009846 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 955) took 1812 ms. Trace:
I20260709 13:07:54.950502 25960 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:54.994704 25850 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:54.993208 25681 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:55.018471 23452 rpcz_store.cc:276] 0709 13:07:53.190892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.191041 (+   149us) service_pool.cc:224] Handling call
0709 13:07:53.476271 (+285230us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:07:55.002864 (+1526593us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b5400555d54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:55.003026 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1188923392}
I20260709 13:07:55.022121 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:55.024636 22896 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:07:54.998303 26019 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:55.022861 25924 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.041874 26013 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.042502 26022 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:55.044386 25960 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.045440 26034 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.026473 25681 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:55.026253 26004 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:55.064033 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:55.098416 22899 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:07:55.056519 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.103837 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:55.107172 22830 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:55.107278 22898 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:55.107749 22898 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:55.103950 26034 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.062354 25954 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.056385 25817 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.111521 25682 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.112375 26034 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.113652 26036 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:55.114310 26036 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.116802 26036 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.075865 25949 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.119580 25984 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.119676 25682 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.120229 25682 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.120180 25984 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:55.056486 26016 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:07:55.139873 22899 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:55.127826 25960 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:55.032011 25850 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:55.062032 25997 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.128381 25682 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:55.062388 25852 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:55.033806 25923 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.128085 25952 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.040083 25977 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.077491 25741 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.126011 22898 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.068050 25887 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.071794 25970 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.076537 26007 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.252519 23452 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:55.137224 25954 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.130927 25984 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.073006 26018 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:55.219298 22898 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:55.074867 26012 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.074054 25687 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.249497 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:55.264675 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.214000 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:55.265784 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.267436 23449 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:55.142540 25775 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.270701 23448 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:55.212795 25960 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.034793 26019 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.128237 26022 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:55.123091 26004 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:55.219841 25852 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.223616 26013 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.211766 25850 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.065637 26014 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.262210 25726 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.321938 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:55.272264 26012 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.256925 25949 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.370733 25986 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.388554 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:55.339355 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.371819 26006 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.311151 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1031) took 1027 ms. Trace:
I20260709 13:07:55.269348 25954 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.390836 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.425346 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.245131 25923 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.391312 22753 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:55.425962 22752 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:07:55.392277 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.276923 26007 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:55.392755 25852 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.337453 25988 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.271950 25970 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:55.435895 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:55.393957 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.419323 25996 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.436890 22754 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:55.394750 26004 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.261615 25997 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.271378 26036 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.285995 25966 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.396304 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.396764 26019 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.398185 25960 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.269785 25963 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:55.399233 26022 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.284462 26032 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:55.321740 26039 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.265058 25977 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.399335 25775 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.388592 22898 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:07:55.401739 26028 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:55.414289 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 92324.9us
I20260709 13:07:55.371222 25859 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.489574 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:55.424093 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.425238 22755 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:55.322335 26024 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.299674 26034 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:55.422578 23444 rpcz_store.cc:276] 0709 13:07:54.271965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.275934 (+  3969us) service_pool.cc:224] Handling call
0709 13:07:54.276248 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:54.276577 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:54.276601 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:54.277051 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:07:54.837365 (+560314us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:54.837414 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.837483 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:55.299320 (+461837us) raft_consensus.cc:1685] finished
0709 13:07:55.299339 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:55.299463 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138549,"raft.queue_time_us":30106,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":7239,"thread_start_us":377614,"threads_started":1,"wal-append.queue_time_us":580990}
W20260709 13:07:55.431489 26037 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:07:55.419763 26012 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:55.433297 25923 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.546458 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.419235 25603 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:55.394368 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.404470 26013 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.451592 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.429550 25984 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:55.454183 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 71026.5us
I20260709 13:07:55.455509 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.421188 25949 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.464695 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.487425 22898 pending_rounds.cc:85] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 7
I20260709 13:07:55.457475 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.488955 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:55.536358 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.534257 23452 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:55.537720 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.538481 26039 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.534816 25977 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.436481 25986 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.437037 26007 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:55.545132 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.630975 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.663506 22898 pending_rounds.cc:107] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:07:55.644903 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.663914 22898 raft_consensus.cc:2889] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:55.664347 22754 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:07:55.664446 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.665119 22755 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:55.665351 22750 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:55.662955 23451 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:55.668291 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:55.654153 26053 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 13:07:55.547224 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.433380 25924 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.574481 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.486550 26036 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.451357 25970 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.575172 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.595470 23444 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:55.578761 26012 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.580911 25603 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:55.581146 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.582789 26013 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.583509 25721 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.548274 25859 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.451855 25988 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:55.674050 22751 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
W20260709 13:07:55.549655 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.674360 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:55.568075 26049 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:07:55.651686 25845 consensus_meta.cc:287] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.681s	user 0.010s	sys 0.000s
I20260709 13:07:55.681471 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:55.646341 25954 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:55.682351 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.662313 25923 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:55.645191 25850 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.675046 22754 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 2.
I20260709 13:07:55.676225 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.675568 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:55.676929 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:55.688441 22897 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:55.689850 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:55.690450 25681 consensus_meta.cc:287] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.000s
I20260709 13:07:55.690523 22900 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:55.690929 25681 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.691483 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:55.691743 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.692875 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.692986 25681 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:55.587513 22794 leader_election.cc:400] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:07:55.673624 25603 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:55.679472 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.679754 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:55.676824 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:55.694481 25970 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:55.682060 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 193173us
I20260709 13:07:55.670807 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.457558 25997 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.684303 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.685089 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:55.681944 25859 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:55.700176 26013 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.700373 25960 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.700927 26013 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.704295 26013 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.704432 25988 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:55.677824 25924 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.707500 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:55.700958 25960 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:55.709946 25960 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:55.693854 25923 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.681537 25689 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:55.689256 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:55.683871 26036 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.585952 26034 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.719790 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 958) took 1512 ms. Trace:
I20260709 13:07:55.719872 26034 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.720176 23446 rpcz_store.cc:276] 0709 13:07:54.207159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.208930 (+  1771us) service_pool.cc:224] Handling call
0709 13:07:54.209219 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:54.908808 (+699589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.719564 (+810756us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54006dea54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:55.719716 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463218432}
I20260709 13:07:55.684784 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.695472 22753 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:55.645140 25984 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.697273 22898 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:55.726589 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.745757 22749 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:07:55.732156 25603 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.713284 25986 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:55.714133 25859 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:55.715003 26046 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.310s	user 0.002s	sys 0.000s
I20260709 13:07:55.535220 26028 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.748488 25845 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:55.686582 26014 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:55.749035 25845 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.749655 22747 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:55.749665 26014 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:55.715924 22897 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:55.695837 22794 leader_election.cc:403] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:55.723521 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:55.728118 25775 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:55.732131 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.737689 25966 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.732394 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:55.723147 25970 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:55.734828 25721 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.736789 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:55.737603 26036 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.747053 26012 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.698379 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.586122 26024 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:55.547405 26032 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:07:56.099629 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 512250us
I20260709 13:07:56.100937 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.106279 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.106715 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.107077 22897 raft_consensus.cc:2410] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:07:56.107262 22749 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:07:56.107594 26064 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:56.107928 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.108629 22898 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:56.088770 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.110141 22753 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 5: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:55.753369 25967 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.336s
I20260709 13:07:56.089455 25923 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:56.089694 22755 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:56.115525 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.091182 26024 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:55.899411 25986 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:55.753769 25944 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:56.092248 26036 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:55.786659 23452 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:56.093256 26012 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.019654 25741 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:56.117849 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:55.751169 26014 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:55.752872 25979 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.342s
I20260709 13:07:55.866425 25995 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:56.172330 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:56.096072 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 372561us
I20260709 13:07:56.096211 25954 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:56.019021 26039 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:55.891311 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.100744 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:07:56.179917 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 79256.3us
I20260709 13:07:56.181541 25689 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:56.181663 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.181941 25689 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:56.182277 22823 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:56.105177 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:55.751282 25984 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.182716 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:56.111840 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.183262 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.183749 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.184090 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.110368 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.184717 23446 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:56.184880 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.116159 22754 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:56.117964 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.103084 25891 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.187716 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:56.187853 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 82801.6us
I20260709 13:07:56.097501 26007 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.097108 25603 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.118775 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:56.194272 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 75577.5us
I20260709 13:07:56.104254 25969 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:07:56.120769 25890 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:56.202605 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.203229 22754 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:07:56.127126 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.171007 22753 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:55.744050 25963 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.207268 25963 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:56.209390 25963 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:56.174487 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.182350 25689 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:56.213920 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.184183 26004 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.111600 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.121261 25970 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:56.116194 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.185427 23449 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:56.184271 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.186703 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.188648 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:56.178460 25954 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:56.187211 22752 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:56.154106 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:56.194953 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:56.220304 23446 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:56.206053 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.193602 25950 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:56.112313 25721 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:56.173823 25986 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:56.113644 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.214571 25887 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:56.112967 25966 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.173105 25741 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.190470 25821 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:07:56.247094 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 58483.5us
I20260709 13:07:56.214937 22753 raft_consensus.cc:2410] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:07:56.219797 23445 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:56.273052 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:56.222347 23419 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:56.262357 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.123122 25944 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:56.264947 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.216717 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 479924us
I20260709 13:07:56.222038 25954 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:56.267575 26036 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.222071 25967 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.268843 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.179324 25995 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.262791 25963 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:56.213655 25850 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:56.219283 26028 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:56.273453 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.287166 22830 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:07:56.265084 22825 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:07:56.268559 26061 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 13:07:56.289143 26032 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:56.275882 25762 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:56.283654 25963 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:56.290516 25944 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.290684 25963 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:56.291271 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.291394 26012 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:56.293015 25986 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.287276 25967 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:56.294342 22753 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:56.295228 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.295759 22823 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:56.296609 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.271906 25887 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:56.274454 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:56.276809 22756 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:07:56.268435 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.298885 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.299935 23449 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:56.300253 22828 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:56.262454 25603 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:56.285830 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:56.289575 25970 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.249540 25950 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:56.274958 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 52709.5us
I20260709 13:07:56.298944 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.282907 25775 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:56.309278 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.310920 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.312356 25721 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.302058 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:56.305791 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.286087 25850 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.285701 22749 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:56.222397 26004 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:56.190079 25891 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.306389 25887 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:56.307446 25859 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:56.307718 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.312577 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.218067 25890 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:56.332332 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:56.338204 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 149094us
I20260709 13:07:56.306947 25603 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:56.345880 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.349295 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.353103 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.353691 22830 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:07:56.354038 22830 pending_rounds.cc:85] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 5
I20260709 13:07:56.287585 26019 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:07:56.354266 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.354264 22830 pending_rounds.cc:107] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:07:56.309435 25986 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:56.360309 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.354136 22824 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:56.310154 25455 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.318542 25950 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:56.319428 25850 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:56.359617 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.358292 22830 raft_consensus.cc:2889] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:56.362651 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.371696 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:07:56.339017 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 66031.4us
I20260709 13:07:56.374271 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:56.377556 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.379762 22900 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:07:56.331248 25859 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:56.381224 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.382316 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:56.343508 25952 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:56.381836 22831 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:56.385490 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 957) took 2163 ms. Trace:
I20260709 13:07:56.382885 22749 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:07:56.354365 25603 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:56.386317 23450 rpcz_store.cc:276] 0709 13:07:54.206877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.207531 (+   654us) service_pool.cc:224] Handling call
0709 13:07:54.207768 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:54.842516 (+634748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:56.369969 (+1527453us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54006b08d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:56.370123 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457283456}
I20260709 13:07:56.396570 22901 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:56.407596 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.409153 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.411232 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.422273 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.386916 25859 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:56.381667 25986 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:56.385900 25952 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:56.424332 25721 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.430137 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:56.430698 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56520.4us
I20260709 13:07:56.432623 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:56.431371 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:56.438010 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 68264.7us
I20260709 13:07:56.438175 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:56.433899 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.413146 25775 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:07:56.439579 22719 connection.cc:256] RPC call timeout handler was delayed by -1.0526460600000291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
I20260709 13:07:56.442688 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.443710 25455 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:56.448764 25455 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:56.449169 25455 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:56.439119 25775 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.450798 25775 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:56.443732 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:56.444619 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:56.445477 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 114723us
I20260709 13:07:56.454779 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:56.436800 25952 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:56.454952 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:56.460131 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:56.438198 26032 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:56.463001 22899 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:07:56.441113 26014 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:56.453614 25944 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:56.464681 26014 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:56.440042 25721 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:56.463933 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:07:56.837378 22749 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:07:56.505246 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66347.6us
I20260709 13:07:56.475678 25455 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:56.534039 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 78744.1us
I20260709 13:07:56.559127 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:56.535553 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 62462.5us
I20260709 13:07:56.863974 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.935639 22902 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
W20260709 13:07:56.552904 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 145668us
W20260709 13:07:56.944093 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 480331us
I20260709 13:07:56.944774 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:56.931782 22871 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:56.467499 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.461510 25970 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:56.952677 25455 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:56.449182 25967 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.461544 26075 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:56.463306 25984 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:56.956847 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:56.957504 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.969473 22902 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
W20260709 13:07:56.959966 25950 consensus_meta.cc:287] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.547s	user 0.006s	sys 0.000s
W20260709 13:07:56.971025 25887 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.615s	user 0.008s	sys 0.000s
I20260709 13:07:56.970685 25950 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.971472 25887 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.972306 23445 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:56.972631 23449 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:56.535024 25715 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.439s
I20260709 13:07:56.461278 25741 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:56.955384 25970 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.568420 25890 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.975046 25950 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.954519 25762 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:56.975829 25762 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:56.976337 25762 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:56.976395 25944 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:56.977149 25721 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.977797 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:56.982015 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:56.967841 25954 consensus_meta.cc:287] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.682s	user 0.006s	sys 0.002s
I20260709 13:07:56.982735 25954 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.974939 26027 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:56.954663 26074 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:56.985693 26027 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.961275 26075 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:56.960748 25859 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.521s	user 0.003s	sys 0.003s
W20260709 13:07:56.654233 26068 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:07:56.655390 26071 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:07:56.463999 25995 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:56.840483 25963 consensus_meta.cc:287] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.003s
I20260709 13:07:56.992600 22822 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:07:56.991271 25984 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.991659 26075 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:56.546424 25176 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:56.997587 22823 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:07:56.950862 26014 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:07:56.999200 22898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.005s	sys 0.010s
W20260709 13:07:56.993577 25775 consensus_meta.cc:287] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.001s
I20260709 13:07:56.975050 25970 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:56.993927 25952 consensus_meta.cc:287] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.525s	user 0.005s	sys 0.004s
W20260709 13:07:56.991932 25603 consensus_meta.cc:287] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.573s	user 0.007s	sys 0.001s
W20260709 13:07:57.002303 22898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.006s	sys 0.010s
I20260709 13:07:56.985246 25954 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:57.002339 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 522250us
I20260709 13:07:56.466686 26032 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.005447 22831 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:07:57.005647 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.002002 25984 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:57.018362 22869 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.010s	sys 0.013s
W20260709 13:07:57.018690 22869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.010s	sys 0.013s
I20260709 13:07:57.020432 25977 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:56.648135 25689 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:57.022360 22896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1452) took 2146 ms (client timeout 1672 ms). Trace:
W20260709 13:07:57.022935 22896 rpcz_store.cc:269] 0709 13:07:54.870383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.941138 (+ 70755us) service_pool.cc:224] Handling call
0709 13:07:57.015981 (+2074843us) spinlock_profiling.cc:243] Waited 82.5 ms on lock 0x7b54000d0554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.016124 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189703552}
I20260709 13:07:57.003615 25176 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:56.993834 25859 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:56.546079 25986 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.059927 22823 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:07:57.063161 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 131429us
W20260709 13:07:57.075476 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.002050 25963 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.007738 26083 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:57.064801 22826 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:07:57.005422 25775 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.108656 26083 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:57.110929 26083 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:57.111954 25775 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:57.109057 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 718) took 2242 ms. Trace:
I20260709 13:07:57.113123 25845 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.114142 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.028805 25176 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:57.116132 25859 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.061129 25887 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:57.011883 25952 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.119445 25952 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.120182 25762 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:57.109673 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 156150us
I20260709 13:07:57.099921 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:57.010113 25603 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.127712 22822 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:57.128407 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.128618 22901 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:57.135711 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:57.137717 22755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38478 (request call id 2475) took 1732 ms (client timeout 2163 ms). Trace:
I20260709 13:07:57.134902 22828 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
W20260709 13:07:57.138129 22755 rpcz_store.cc:269] 0709 13:07:55.396440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.667727 (+271287us) service_pool.cc:224] Handling call
0709 13:07:57.128650 (+1460923us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b5400336dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.128745 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":922525568}
W20260709 13:07:57.140877 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 207892us
I20260709 13:07:57.144217 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.140581 25967 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:57.107933 26081 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:57.149026 22868 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.022s (SENT) [suppressed 16 similar messages]
I20260709 13:07:57.146610 25967 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.151935 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.113610 22903 rpcz_store.cc:276] 0709 13:07:54.863637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.863749 (+   112us) service_pool.cc:224] Handling call
0709 13:07:54.864041 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:54.933100 (+ 69059us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:54.984628 (+ 51528us) log.cc:844] Serialized 14 byte log entry
0709 13:07:55.007351 (+ 22723us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:55.007400 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:55.007438 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.105675 (+2098237us) spinlock_profiling.cc:243] Waited 68.8 ms on lock 0x7b5400206a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:57.105740 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.105895 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7383,"prepare.queue_time_us":76514,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":87,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":158342016,"thread_start_us":58608,"threads_started":2,"wal-append.queue_time_us":266680,"wal-append.run_cpu_time_us":1897,"wal-append.run_wall_time_us":1315513}
I20260709 13:07:57.153045 26090 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:57.153621 26090 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.156162 25967 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:57.153879 25689 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.157045 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.157768 22831 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:57.144176 26091 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.165163 26088 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:57.165830 26088 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:57.166690 26088 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:57.165863 26091 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:57.165624 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 101687us
I20260709 13:07:57.168445 26081 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.170949 26032 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.142508 25991 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:57.173791 26051 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.718s	user 0.010s	sys 0.014s
I20260709 13:07:57.174577 25762 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:57.175488 25850 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.778s	user 0.007s	sys 0.003s
I20260709 13:07:57.175994 25850 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.178220 25850 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.168849 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.179490 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.180336 22822 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:57.180606 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.181207 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.181442 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:57.181722 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.173465 25970 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.182988 22830 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:07:57.179576 25850 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:57.140815 25859 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:57.185936 25970 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:57.189878 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.184010 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:57.184960 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.190719 26086 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:57.191107 26086 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.191431 26086 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:57.188858 25850 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:57.186457 22822 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:57.183557 25603 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:57.202430 26087 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:57.203955 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.183653 26084 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:57.206120 26084 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:57.206496 26084 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:57.188203 25859 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.198695 22831 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:57.171820 26081 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.156098 26090 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:57.168840 26091 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:57.226177 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 839) took 1684 ms. Trace:
I20260709 13:07:57.226557 22750 rpcz_store.cc:276] 0709 13:07:55.542007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.270562 (+728555us) service_pool.cc:224] Handling call
0709 13:07:56.270799 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:56.287951 (+ 17152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:56.287969 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:56.287998 (+    29us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:56.288015 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.225961 (+937946us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b540011add4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:57.226007 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.226127 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39095680}
I20260709 13:07:57.228438 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38478 (request call id 2487) took 1050 ms. Trace:
I20260709 13:07:57.228802 22752 rpcz_store.cc:276] 0709 13:07:56.177773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.183553 (+  5780us) service_pool.cc:224] Handling call
0709 13:07:57.228284 (+1044731us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b540016b054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.228386 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6850816}
W20260709 13:07:57.232172 22751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 842) took 1689 ms (client timeout 1639 ms). Trace:
W20260709 13:07:57.232535 22751 rpcz_store.cc:269] 0709 13:07:55.542780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.580930 (+ 38150us) service_pool.cc:224] Handling call
0709 13:07:57.232000 (+1651070us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b54003f0cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.232107 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6134528}
I20260709 13:07:57.233675 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1035) took 1985 ms. Trace:
I20260709 13:07:57.234020 23453 rpcz_store.cc:276] 0709 13:07:55.247853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.247987 (+   134us) service_pool.cc:224] Handling call
0709 13:07:55.248214 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.591113 (+342899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:55.591156 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:55.591214 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:55.591259 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.233479 (+1642220us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54007d0fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:57.233520 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.233626 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":788030720}
I20260709 13:07:57.234616 26088 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.235394 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.236426 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.236920 23453 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:07:57.239142 26088 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.240439 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.243376 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.243628 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.204852 25891 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.217484 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.174681 26027 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.251812 25924 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:57.251917 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.253340 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.253969 22824 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:07:57.254796 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.255314 22824 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:57.256168 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.208034 25850 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:57.259338 26084 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.263373 25949 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:57.221375 25859 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:57.222033 26078 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.358s	user 0.006s	sys 0.004s
I20260709 13:07:57.266989 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.222541 25966 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.269641 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2261) took 1069 ms. Trace:
I20260709 13:07:57.269775 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.270366 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.270905 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.270018 22827 rpcz_store.cc:276] 0709 13:07:56.199946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.200336 (+   390us) service_pool.cc:224] Handling call
0709 13:07:56.220176 (+ 19840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:56.298283 (+ 78107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:56.298310 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:56.298347 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:56.298374 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.269444 (+971070us) spinlock_profiling.cc:243] Waited 77.9 ms on lock 0x7b54001edcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:57.269494 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.269592 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179224320}
I20260709 13:07:57.272230 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.272589 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2254) took 1163 ms. Trace:
I20260709 13:07:57.272756 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.273043 22903 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:57.275821 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.276372 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.277022 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.244442 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.272233 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.278162 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.278476 22827 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:57.278612 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.278986 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.279238 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.279668 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.219187 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.244634 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.281234 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.281339 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.282362 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.282517 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.248515 25997 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:57.282943 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2045) took 1615 ms. Trace:
I20260709 13:07:57.283486 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.284857 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.285517 23444 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:07:57.286063 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1037) took 2037 ms. Trace:
I20260709 13:07:57.286549 23447 rpcz_store.cc:276] 0709 13:07:55.248439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.248553 (+   114us) service_pool.cc:224] Handling call
0709 13:07:55.265075 (+ 16522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.451789 (+186714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:55.451819 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:55.451862 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:55.451892 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.285668 (+1833776us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540024ca54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:57.285728 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.285858 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428875008}
I20260709 13:07:57.288220 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.290272 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.291285 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.292222 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.185875 25963 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.280197 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.280148 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.292946 23446 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:57.279245 23448 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
W20260709 13:07:57.293684 22747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 838) took 1752 ms (client timeout 1552 ms). Trace:
I20260709 13:07:57.293579 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.293939 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.294337 23450 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:07:57.294127 22747 rpcz_store.cc:269] 0709 13:07:55.541691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.574232 (+ 32541us) service_pool.cc:224] Handling call
0709 13:07:57.293508 (+1719276us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b540032bc54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.293622 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182134016}
I20260709 13:07:57.294968 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.296029 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2258) took 1142 ms. Trace:
I20260709 13:07:57.296088 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.249996 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1455) took 2378 ms. Trace:
I20260709 13:07:57.296674 22825 rpcz_store.cc:276] 0709 13:07:56.153647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.153802 (+   155us) service_pool.cc:224] Handling call
0709 13:07:57.295814 (+1142012us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b54002f0dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.295967 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2803072}
I20260709 13:07:57.298063 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:57.298317 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38478 (request call id 2486) took 1182 ms. Trace:
I20260709 13:07:57.298727 22756 rpcz_store.cc:276] 0709 13:07:56.115626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.121545 (+  5919us) service_pool.cc:224] Handling call
0709 13:07:57.298064 (+1176519us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b54000776d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.298173 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153216256}
I20260709 13:07:57.297488 22905 rpcz_store.cc:276] 0709 13:07:54.871228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.072209 (+200981us) service_pool.cc:224] Handling call
0709 13:07:55.072494 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.391777 (+319283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:55.391805 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:55.391844 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:55.391888 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.249717 (+1857829us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b5400071a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:57.249762 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.249933 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733866240}
I20260709 13:07:57.283334 22753 rpcz_store.cc:276] 0709 13:07:55.667487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.283927 (+616440us) service_pool.cc:224] Handling call
0709 13:07:56.285086 (+  1159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:56.294667 (+  9581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.282753 (+988086us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b540011a8d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:57.282893 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20879104}
I20260709 13:07:57.299999 23446 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:57.300760 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2047) took 1631 ms. Trace:
I20260709 13:07:57.301463 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.301626 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:57.301098 22748 rpcz_store.cc:276] 0709 13:07:55.669394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.283807 (+614413us) service_pool.cc:224] Handling call
0709 13:07:57.299586 (+1015779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.299735 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:57.299756 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:57.299790 (+    34us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:07:57.299816 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.299861 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.300615 (+   754us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000776d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.300709 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2335296768}
I20260709 13:07:57.298363 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.302611 25991 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:57.303047 25991 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:57.285102 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.303433 25991 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:57.303467 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.304605 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.241140 23449 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:57.306327 25989 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:57.306690 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:57.307471 25967 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:57.307103 26014 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.307883 25967 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:57.308295 25967 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:57.309543 26014 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:57.310768 23452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 969) took 1656 ms (client timeout 1850 ms). Trace:
W20260709 13:07:57.311141 23452 rpcz_store.cc:269] 0709 13:07:55.654587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.683062 (+ 28475us) service_pool.cc:224] Handling call
0709 13:07:57.310627 (+1627565us) spinlock_profiling.cc:243] Waited 74.1 ms on lock 0x7b5400698f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.310720 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170364800}
I20260709 13:07:57.311689 23447 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 5: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:57.311861 25176 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.312458 23452 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:57.313936 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2049) took 1630 ms. Trace:
I20260709 13:07:57.314360 23447 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:57.314291 22754 rpcz_store.cc:276] 0709 13:07:55.683477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.282304 (+598827us) service_pool.cc:224] Handling call
0709 13:07:57.245144 (+962840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.245354 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:57.245377 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:57.245417 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:57.245435 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.245468 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.313787 (+ 68319us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b54003f0cd0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.313880 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2211281792}
I20260709 13:07:57.311010 23451 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:07:57.306690 25997 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:07:57.317863 22823 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:57.295457 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.318099 25997 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:57.318544 25997 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:57.319999 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.253091 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.322041 22831 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:07:57.321894 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:57.323539 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:57.324183 22905 raft_consensus.cc:2368] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:57.306730 25989 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:57.325376 25989 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.326331 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.326663 25989 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:57.257198 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.299248 25966 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:57.328644 25966 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:57.329013 25966 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:57.328964 26031 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:57.329376 26034 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:57.330261 26034 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:57.272998 22826 rpcz_store.cc:276] 0709 13:07:56.108755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.108860 (+   105us) service_pool.cc:224] Handling call
0709 13:07:57.272405 (+1163545us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b54001f90d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.272514 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2021353344}
I20260709 13:07:57.330683 26034 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:57.328230 22824 raft_consensus.cc:2376] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5. Re-sending same reply.
I20260709 13:07:57.331513 26034 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.286339 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.332648 26034 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:57.327703 26029 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:07:57.335453 26029 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:57.335789 26029 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:55.669570 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:57.327685 26004 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:57.336541 26029 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.296901 23448 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:07:57.281404 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.338011 26029 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:57.338781 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.339628 26075 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.319375 25997 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.294768 26083 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.341223 25997 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:57.341782 22904 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:57.342183 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.343190 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 724) took 2475 ms. Trace:
I20260709 13:07:57.343387 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.343534 22904 rpcz_store.cc:276] 0709 13:07:54.867910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.029689 (+161779us) service_pool.cc:224] Handling call
0709 13:07:57.016353 (+1986664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:57.342130 (+325777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.342620 (+   490us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54000d0554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:07:57.343043 (+   423us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54000d0550. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:57.343146 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5254374016}
I20260709 13:07:57.344587 23452 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:57.343950 22748 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:57.346122 26083 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.347380 22825 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:57.348536 23453 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:57.348351 25966 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.351660 25176 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:57.352695 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 105625us
I20260709 13:07:57.353346 25966 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.351274 25967 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.358983 25967 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.350415 25850 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.361187 25850 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.354456 25991 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.362802 22754 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:57.363232 25991 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.293123 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.365375 26086 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.367012 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.367900 23453 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:07:57.367300 26086 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:57.374771 22753 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:57.375103 22753 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:57.375073 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:57.377735 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.378428 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.369475 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.352743 23450 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:57.381363 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.380167 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.382733 22754 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 3.
I20260709 13:07:57.382566 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:57.385279 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.386085 23453 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:57.376180 22753 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.388326 22753 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:57.323741 25984 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.391484 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.398542 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.399237 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.393052 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.400139 22754 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:57.395294 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:57.402750 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.403628 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.404157 22749 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:57.405016 25455 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:57.405535 25455 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:57.405946 25455 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:57.387668 25963 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:57.400628 22650 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 6 to 9, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 9 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:57.414165 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.407893 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.409063 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.410219 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.410492 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.415964 23446 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:57.411177 25991 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:57.412447 25967 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:57.415171 22756 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:57.390005 25952 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:57.421977 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.446239 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.433161 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.434703 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.436259 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.445744 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.447412 22751 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:07:57.447602 22828 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:57.447899 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.438676 22650 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.451025 22650 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 5 to 7, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.455359 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.456131 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.446363 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:57.457705 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 49881.8us
I20260709 13:07:57.459228 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.460457 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.461170 22751 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:57.461889 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.462466 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.462806 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.463656 25966 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:57.463886 22831 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:07:57.464298 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.465451 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.465879 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.465976 22828 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:07:57.466228 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.466820 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:57.467399 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.467813 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.467268 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.466882 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.473970 22826 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:57.466006 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:07:57.475828 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.480144 25455 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.404008 26075 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:57.390185 26083 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:07:57.489617 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.489286 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:57.490598 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.482776 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.481876 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.492223 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:57.422495 26077 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:57.495914 22754 raft_consensus.cc:2410] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:07:57.496431 25455 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.497431 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:57.483363 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.487720 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.487968 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.413894 25963 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:57.481752 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50516.1us
I20260709 13:07:57.491976 23452 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:57.491952 23453 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:57.496049 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:57.510989 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:57.459122 22650 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 5, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 5 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.504307 22826 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:57.506379 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:57.507128 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.507282 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.507758 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:57.502723 22830 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:07:57.514281 22823 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:57.514917 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.515941 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:57.515318 22650 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 8 to 9, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 9 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:07:57.517470 25455 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:57.518167 25455 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:57.488724 26027 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:57.528630 26027 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.528997 26027 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:57.481304 25966 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:57.530349 25984 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:57.520287 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:57.519285 26041 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:57.522645 22654 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 332) took 7489 ms. Trace:
I20260709 13:07:57.525519 26015 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:57.526121 25741 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:57.499106 26077 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:57.527406 26084 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.528244 26054 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:57.560873 22822 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:57.513136 22828 raft_consensus.cc:2376] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7. Re-sending same reply.
I20260709 13:07:57.538192 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:57.510882 26014 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:57.564601 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.540742 22749 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:07:57.512477 25963 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:57.507067 26034 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:57.559567 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.542795 26041 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.518473 22650 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 6 to 9, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 9 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.571873 22650 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 4 to 5, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 5 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:57.512332 25996 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:57.520006 26086 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:57.481287 26038 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:07:57.529166 26019 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:07:57.579255 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 386247us
W20260709 13:07:57.507841 26078 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:07:57.566392 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.564823 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:57.498032 25952 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:57.560339 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.570533 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:57.573186 22650 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 8, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 8 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:57.511756 25960 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:07:57.574368 25176 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:57.556969 26077 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:57.580397 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:57.543002 22654 rpcz_store.cc:276] 0709 13:07:50.033853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.034023 (+   170us) service_pool.cc:224] Handling call
0709 13:07:57.522567 (+7488544us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:53.410874 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:53.411618 (+   744us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:53.411663 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:07:53.411681 (+    18us) tablet.cc:662] Decoding operations
0709 13:07:53.481740 (+ 70059us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:53.481814 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:53.481845 (+    31us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:07:53.483545 (+  1700us) tablet.cc:701] Row locks acquired
0709 13:07:53.483594 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:07:53.483760 (+   166us) write_op.cc:270] Start()
0709 13:07:53.483891 (+   131us) write_op.cc:276] Timestamp: P: 1783602473483704 usec, L: 0
0709 13:07:53.483914 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:07:53.517123 (+ 33209us) log.cc:844] Serialized 8566 byte log entry
0709 13:07:54.455874 (+938751us) op_driver.cc:464] REPLICATION: finished
0709 13:07:54.906025 (+450151us) write_op.cc:301] APPLY: starting
0709 13:07:54.906126 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:54.906587 (+   461us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:54.906701 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:55.129226 (+222525us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:55.129828 (+   602us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:07:55.129897 (+    69us) write_op.cc:312] APPLY: finished
0709 13:07:55.130605 (+   708us) log.cc:844] Serialized 152 byte log entry
0709 13:07:55.435849 (+305244us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:55.462640 (+ 26791us) write_op.cc:454] Released schema lock
0709 13:07:55.490331 (+ 27691us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:55.649576 (+159245us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:55.649619 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":18694,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":440105,"apply.run_cpu_time_us":29483,"apply.run_wall_time_us":743479,"num_ops":22,"prepare.queue_time_us":168361,"prepare.run_cpu_time_us":26999,"prepare.run_wall_time_us":242499,"raft.queue_time_us":620400,"raft.run_cpu_time_us":14779,"raft.run_wall_time_us":382907,"replication_time_us":971759,"thread_start_us":709994,"threads_started":5,"wal-append.queue_time_us":472322,"wal-append.run_cpu_time_us":21871,"wal-append.run_wall_time_us":2541709}]]}
I20260709 13:07:57.564903 25991 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:57.573603 26041 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:57.588470 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:57.565253 26054 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:57.590039 26054 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:57.546142 25990 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:07:57.592824 26088 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:57.581306 26019 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:57.597447 26019 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.598516 26019 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:57.565115 26090 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:57.571379 25969 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:57.588466 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.593396 26088 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:57.583258 22650 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 11, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 11 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:57.600759 26088 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.600971 22650 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:57.595113 25966 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:57.602597 22650 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 6 to 9, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 9 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.605342 22650 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:57.565922 25859 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:07:57.606556 22650 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 3 to 5, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 5 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:57.607439 25859 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:57.607831 25859 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:57.607864 22650 catalog_manager.cc:5697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 5 to 6, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 6 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } }
I20260709 13:07:57.608657 25859 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:57.594249 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.620776 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.609277 22650 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:07:57.622906 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.624243 22753 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:07:57.624835 22756 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:07:57.625886 22824 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:07:57.626340 22756 pending_rounds.cc:85] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 3
I20260709 13:07:57.626410 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.625814 23446 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:07:57.626860 22756 pending_rounds.cc:107] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:07:57.582161 25952 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:57.627175 22756 raft_consensus.cc:2889] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:57.578496 26086 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:57.632019 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:57.632894 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44498.4us
I20260709 13:07:57.633880 22752 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:07:57.634115 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.629094 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.629525 22753 pending_rounds.cc:85] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 5
I20260709 13:07:57.635174 22753 pending_rounds.cc:107] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:07:57.635560 22753 raft_consensus.cc:2889] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:57.636809 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.637709 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.626164 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:57.589303 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:07:57.638509 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 49277.2us
I20260709 13:07:57.639000 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:57.639569 22871 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:57.645762 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.647030 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:57.647807 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.648690 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.647418 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.648615 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.625530 25741 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:07:57.626029 26007 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:57.627787 25967 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:57.653786 22750 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:07:57.634950 26086 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:57.697111 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:57.698025 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:07:57.693223 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 67140.4us
I20260709 13:07:57.698542 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:07:57.693929 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 45373.6us
I20260709 13:07:57.653769 25455 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:57.632302 25991 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:57.654891 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.628458 25952 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:57.647384 26075 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:57.643795 25986 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:07:57.704278 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49470us
I20260709 13:07:57.742635 22903 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:07:57.743427 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.648994 26034 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:57.651075 25997 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:07:57.752691 22900 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:57.752990 22826 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:07:57.708649 25963 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.711050 25573 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:57.711849 26032 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:57.712904 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.713627 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.760996 22756 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:57.713770 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.713882 25991 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.648594 25976 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:57.651975 25960 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:07:57.721796 25952 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.725479 26015 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:57.726850 25859 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:57.729158 26027 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.694892 25989 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:07:57.733636 26075 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.738925 26070 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:57.692276 26004 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:57.694906 25967 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.695399 26081 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:57.740545 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.741173 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:57.744727 22903 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:57.790441 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.745563 22896 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:57.692772 25845 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:57.653739 26033 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:57.794754 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:57.745923 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:57.647858 26083 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:07:57.748693 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 96738.1us
I20260709 13:07:57.751477 23448 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:57.754453 22905 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:57.795413 22898 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:57.695678 25741 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:57.796291 25741 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:57.758478 22904 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:57.766392 25593 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
W20260709 13:07:57.761914 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48217.2us
I20260709 13:07:57.808467 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.765345 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.752426 25721 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:57.764823 25715 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:57.746896 25966 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:07:57.826802 25715 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:57.827260 25715 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:57.827601 25963 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:57.785285 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.766662 25960 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:57.828900 25960 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:57.829568 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:57.830142 22904 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:57.804436 26070 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:57.839006 26070 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.788007 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:57.785876 26075 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:57.704142 25817 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:57.804981 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.805567 22756 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:07:57.857651 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.812039 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.813973 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.862169 22905 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:07:57.789549 25967 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:57.854592 22900 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:57.789731 26081 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:57.868610 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:57.804510 26083 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:57.813149 25990 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:57.868112 26033 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:57.877804 25741 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.878378 26033 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:57.878808 26033 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:57.874095 26081 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:57.880300 22752 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:07:57.877043 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:57.879839 25741 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:57.893541 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:57.894198 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.901523 22901 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:07:57.865165 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68842.2us
I20260709 13:07:57.909124 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.900045 26083 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:57.925783 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 113806us
I20260709 13:07:57.927472 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:57.927659 26041 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.929875 26041 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.914891 26110 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:57.933868 26090 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:57.934258 26090 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:57.935055 26090 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.936734 22751 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:57.938234 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:57.938900 26097 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.210s	user 0.000s	sys 0.003s
I20260709 13:07:57.940231 26077 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.942412 26029 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:57.942776 26029 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:57.943528 26029 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.944337 26086 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.944859 26029 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:57.946677 26027 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.947284 26027 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:57.924944 22904 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:07:57.951462 22897 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:07:57.940954 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.952903 22898 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:07:57.955700 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:57.957623 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:57.963320 22897 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:57.924955 25715 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:57.966042 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.947598 26027 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.967003 22897 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:57.968755 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:57.970896 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 221965us
I20260709 13:07:57.959982 26084 raft_consensus.cc:2764] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:07:57.972795 22824 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:57.960798 26110 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:57.973629 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:57.976276 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:57.977583 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.978258 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.978425 22905 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:57.978705 22899 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:57.983229 22898 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:57.979123 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.983230 22904 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:57.979533 22902 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:57.986280 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.987495 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.988380 25715 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:07:57.988580 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.989084 26108 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:07:57.989207 22899 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:07:57.989712 26108 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:57.990576 26108 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:57.992168 26081 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.970553 26027 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:57.996099 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:57.998143 22904 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:58.000243 26075 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:57.973439 26113 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:58.003425 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.006335 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:58.007108 23450 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:58.013459 25952 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:57.955363 26054 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.016211 26054 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:07:58.016784 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:58.017069 26103 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.371s	user 0.002s	sys 0.000s
I20260709 13:07:58.018606 26077 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:58.019984 26100 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 13:07:57.981830 25963 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:58.022105 26012 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:58.022482 26012 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:58.023273 26012 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.024466 26012 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:58.025918 22903 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:58.025967 22755 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:58.028391 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.029066 22747 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:58.031100 26015 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:58.037379 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:57.980582 22900 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:07:57.973662 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.046911 22904 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:07:58.047485 22897 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:07:58.045985 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69785.2us
I20260709 13:07:58.047964 22747 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:58.048731 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.051404 22901 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:07:58.040063 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.058940 23448 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:07:58.060554 25176 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:58.061116 25176 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:58.061679 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.063457 25963 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:07:58.063817 25963 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:58.064173 25963 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:58.064007 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.066098 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.088747 23450 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
W20260709 13:07:58.096801 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 169394us
I20260709 13:07:58.099877 22899 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:58.056042 26113 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:58.111325 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.113723 25176 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:07:58.114333 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 89730us
I20260709 13:07:58.068207 26084 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:07:58.117354 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.090158 26077 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:07:58.133613 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 83957.8us
I20260709 13:07:58.134115 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.136346 26007 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:58.139091 22794 leader_election.cc:400] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:58.139431 22794 leader_election.cc:403] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:07:58.099588 25797 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:58.142731 25176 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:58.108809 25817 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:58.108559 26088 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:58.150285 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:58.158438 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.161475 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:58.147873 26084 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:58.168665 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:58.145071 25817 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:07:58.188275 23451 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:58.195081 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 44856.5us
I20260709 13:07:58.194903 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.196996 23452 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:58.197430 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.198351 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.198736 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.198854 22824 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:58.199038 22748 raft_consensus.cc:2393] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:58.199625 25984 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:58.199980 25984 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.200371 23446 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:58.187062 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:58.201367 25949 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:58.201748 23448 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:58.202246 26091 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:58.203037 25681 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:58.200380 25984 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:58.157860 25593 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:07:58.144820 25967 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.208876 25967 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:58.209600 22749 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:07:58.209978 25817 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:58.211598 25949 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:58.212098 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.212678 22900 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:58.129673 26110 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:07:58.213944 26110 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:58.214282 26110 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:58.214854 25593 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:58.215615 25593 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.215406 26110 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.216080 22900 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:07:58.216933 26110 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:58.216516 25593 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.152845 26019 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:07:58.218809 22897 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:58.235446 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.202317 25891 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:58.152541 26086 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:58.225281 22902 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:07:58.145664 25991 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:07:58.226459 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.230422 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:58.239478 22902 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:07:58.236083 22823 raft_consensus.cc:2376] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8. Re-sending same reply.
W20260709 13:07:58.239428 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 128175us
I20260709 13:07:58.230592 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.240482 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.243239 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.243610 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:58.149730 25797 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.245613 23451 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:07:58.235605 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:58.222064 26014 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:58.253587 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:58.244031 22903 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:58.254063 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:58.246667 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:58.254559 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:58.197932 26070 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7.
I20260709 13:07:58.249840 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:58.246557 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:58.252050 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:58.250909 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:58.263352 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:58.237293 25845 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:58.263193 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.262454 25891 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.267668 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:58.278790 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:58.280550 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.245827 26075 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:58.282492 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.290235 25176 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:58.179257 26033 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.246819 26084 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:58.295492 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.295950 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.299585 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.300531 22903 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:58.286295 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.287402 22748 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:58.287658 22755 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:58.287456 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.334800 22827 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:58.335187 22902 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:07:58.288681 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.289250 22826 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:58.225435 26083 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.293642 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:58.292874 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.277541 25967 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:07:58.324384 26070 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:58.247174 25984 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.290102 25593 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:58.325371 25891 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.351639 25960 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.352993 22904 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:58.354751 25984 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:07:58.361548 26116 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:07:58.362576 26033 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:58.333748 25176 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:58.290787 26081 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:58.336390 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:58.336629 22830 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:07:58.342338 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44818.5us
I20260709 13:07:58.343251 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.372092 22897 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 3: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:07:58.343864 26121 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:07:58.335276 26120 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:07:58.358632 26083 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:58.378515 26083 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:58.378861 26083 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:58.379534 25797 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:58.379645 26083 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.380967 26083 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:58.343954 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:58.344312 26117 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:07:58.393210 26073 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:07:58.345176 26114 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
I20260709 13:07:58.345664 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.346112 25715 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:58.328346 25845 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:07:58.268761 26027 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.347175 26090 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:58.349735 26115 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:07:58.364632 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.371425 22902 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:58.374147 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:58.418921 22897 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:58.387491 22830 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:58.366739 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.391570 25817 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:58.401418 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.425486 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:58.426353 23450 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:58.426432 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.403705 22824 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:58.428560 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.414664 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:58.430099 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.430751 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.431932 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.433027 22905 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:07:58.433429 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.433791 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.434319 22828 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:58.434605 22871 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.435667 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.436347 25176 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:58.436718 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.438652 25976 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:58.439613 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:58.441218 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 47946.4us
I20260709 13:07:58.439311 22897 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:58.443866 25176 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:58.443400 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:58.446108 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.447482 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:07:58.447922 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:58.448249 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.439878 26033 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:58.402906 26004 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:07:58.442713 26086 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:58.422063 26014 consensus_queue.cc:1059] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:07:58.442878 26087 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:58.442014 25715 raft_consensus.cc:2764] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:07:58.432624 26105 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:58.447134 26104 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:58.432940 26081 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:58.443030 26113 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:07:58.500149 25845 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:07:58.440515 26075 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:58.481794 26033 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.515882 22828 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:58.515034 26033 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.447746 25976 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:07:58.433497 25952 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:58.521000 26031 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:58.521427 26031 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:58.522382 26031 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:07:58.523625 26031 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:58.523706 26087 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:58.524930 26087 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.527585 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.528364 22904 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:58.530956 22900 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:07:58.534276 26129 raft_consensus.cc:3055] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:58.534647 26129 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:58.535477 26129 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:58.536645 26014 consensus_queue.cc:1175] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6a5191aaff76469bbf6c4735dc9e6509" responder_term: 4 status { last_received { term: 4 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 4 index: 7 } }
I20260709 13:07:58.537123 26129 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:58.537480 26014 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:58.538378 26014 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:58.538690 26014 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:58.532995 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.451942 22751 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:58.546505 22900 raft_consensus.cc:2410] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:07:58.450027 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.402560 25996 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:07:58.547210 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.450652 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.478366 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.547832 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.548091 22900 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:58.548713 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.449422 26111 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:07:58.553957 22825 raft_consensus.cc:2368] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:58.494711 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.517105 26033 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:58.496997 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 52096.1us
I20260709 13:07:58.497813 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.456733 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.562801 22748 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:07:58.455801 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.458246 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.450263 25176 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:58.564428 25176 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.442193 25850 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:58.406324 25963 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.446717 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:58.497028 25970 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:58.500227 22902 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:07:58.578553 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.503866 26004 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:58.507337 25960 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:58.507488 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:58.499594 25859 consensus_meta.cc:287] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.715s	user 0.014s	sys 0.001s
I20260709 13:07:58.511632 25969 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:58.594277 25969 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.595067 25969 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.596316 25969 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:58.599375 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.513275 25984 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:58.545706 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.601101 25984 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:58.548810 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.601523 25850 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:58.476658 26122 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:58.549630 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.602689 26122 raft_consensus.cc:3055] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:58.603049 26122 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:07:58.601434 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55809us
I20260709 13:07:58.603726 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.603797 26122 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.605120 26122 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:58.606215 26029 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:07:58.606581 26029 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:58.607339 26029 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.607966 25573 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:58.608459 26029 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:58.609899 25455 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:58.610244 25455 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.611215 26086 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:58.550421 22826 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:58.554863 26077 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:07:58.548038 22903 raft_consensus.cc:2376] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8. Re-sending same reply.
I20260709 13:07:58.613993 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.614523 22903 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:58.615285 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:58.616008 22897 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:58.616402 22903 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:58.550932 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.482889 26027 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:58.551457 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:58.553812 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.482168 25817 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:58.557226 22822 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
W20260709 13:07:58.493144 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49843.2us
I20260709 13:07:58.563630 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:58.625954 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 62429.2us
I20260709 13:07:58.627494 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:07:58.569793 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 123178us
I20260709 13:07:58.565529 25176 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.581036 22824 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:07:58.583956 22899 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:58.629022 25176 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:58.581113 26004 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.552457 26014 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.588961 25963 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:58.581602 26073 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:58.549738 25967 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:58.602039 25984 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.582707 26120 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:58.613175 26077 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:58.617642 22901 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:58.629984 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:58.651248 22899 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:58.529126 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.621802 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.550014 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:58.610903 25455 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.645963 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:58.646842 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.648365 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 2075) took 1027 ms. Trace:
I20260709 13:07:58.653373 22868 leader_election.cc:400] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:58.654358 22868 leader_election.cc:403] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:07:58.658290 22905 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:07:58.658638 22905 pending_rounds.cc:85] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 5
I20260709 13:07:58.658845 22905 pending_rounds.cc:107] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:07:58.659044 22905 raft_consensus.cc:2889] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:58.654897 22749 rpcz_store.cc:276] 0709 13:07:57.620695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.626205 (+  5510us) service_pool.cc:224] Handling call
0709 13:07:58.648161 (+1021956us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b540011a8d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:58.648303 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1339086592}
I20260709 13:07:58.662652 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.663743 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.664733 22896 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:58.665601 25455 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:58.632028 25845 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:58.698776 22903 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:58.698338 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:58.702214 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:58.706346 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:58.698506 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:58.698345 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.708109 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:58.708235 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:58.657040 26014 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:58.651217 25986 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:58.711589 22827 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:58.712460 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:58.650928 26077 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:07:58.657231 26027 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:58.656412 25963 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:07:58.660940 25817 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:58.699605 25721 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:58.643456 26091 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:07:58.709864 25970 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:07:58.708897 26073 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:58.705144 26041 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:58.701541 26015 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:07:58.645123 25681 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:07:58.713657 26019 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:58.661134 25949 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
W20260709 13:07:58.759822 26118 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:07:58.760104 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.724704 25991 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:58.716228 26111 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:58.669606 25850 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:07:58.770221 25573 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:07:58.708364 25990 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:07:58.765884 25891 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:07:58.774945 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.779027 23452 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:07:58.809715 26138 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:07:58.779538 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:07:58.816905 26133 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:07:58.756068 26004 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.821903 25990 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:58.822288 25990 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.823065 25990 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.824329 25990 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:58.791494 26073 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:58.760118 26090 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:07:58.791932 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 79557.4us
I20260709 13:07:58.796483 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:58.772739 25721 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:07:58.799924 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 91767.4us
I20260709 13:07:58.790900 25970 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:58.768946 25715 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:07:58.789647 23446 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
W20260709 13:07:58.827006 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 52137.1us
W20260709 13:07:58.828550 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.831847 22899 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:58.831861 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:58.833205 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.836004 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:58.837543 22753 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:58.838274 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.840687 25986 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:58.829754 26110 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:58.830997 26073 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:58.830045 25991 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:58.848474 25986 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.852568 25986 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.853384 25890 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:58.848820 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:58.833428 25721 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:58.841961 26007 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:58.850080 25845 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:58.839789 25976 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:07:58.842197 26015 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:58.854976 22822 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:58.835392 25970 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:58.839964 26090 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:58.861975 25891 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:07:58.868402 22829 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:58.869189 25969 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:58.852653 25991 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.858577 25986 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:58.892691 23446 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:58.893364 26146 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:58.893786 26146 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:58.894583 26146 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.895741 26142 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:58.876127 26031 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:58.901806 26090 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.875909 25969 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.910338 23446 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:07:58.910125 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:58.903764 26129 raft_consensus.cc:2764] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:07:58.927706 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:58.931875 23448 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:07:58.892282 26029 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:07:58.888984 25845 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:07:58.897703 26142 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:58.891824 25986 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:07:58.925865 26034 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.919883 25891 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:58.945573 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 113829us
I20260709 13:07:58.925963 26031 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:58.950659 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:07:58.953943 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.957558 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:58.923853 26038 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:07:58.951613 26054 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:07:58.964757 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44684.4us
I20260709 13:07:58.951941 26141 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:07:58.968854 26014 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:07:58.969444 26014 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:58.970352 26014 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.971583 25986 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:58.975417 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.980965 25721 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:58.974653 26031 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.981890 26142 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.983219 26142 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:58.983559 25721 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:58.985010 22903 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:58.987440 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.961956 25991 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:58.991070 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.992738 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.993507 26041 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:58.975409 26054 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:58.952229 26034 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.955114 25891 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:58.997691 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:58.994709 25991 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:59.013870 26149 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:58.998644 25949 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:59.015529 26148 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:07:59.020498 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.989411 25986 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 13:07:59.017144 26136 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.210s	user 0.002s	sys 0.000s
I20260709 13:07:58.994117 26111 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:07:58.967402 25969 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.040597 26054 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:59.022569 26129 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:59.038054 26152 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:07:59.104396 26151 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:07:59.083339 26148 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:07:59.090266 26143 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.202s	user 0.000s	sys 0.005s
I20260709 13:07:59.114972 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:59.091452 25997 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.109815 26091 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:59.109609 26073 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.117749 26034 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.122416 25969 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:59.132848 23453 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:59.142750 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:59.144760 22824 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:59.124029 25997 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.150055 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:59.140316 25741 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:59.151345 25741 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:59.144515 26041 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:59.152521 26041 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:59.153100 26148 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.154543 26041 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:59.123152 25991 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.144094 26015 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:59.157167 25991 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:59.159713 25997 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:59.142122 25891 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.143810 25970 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.164598 25970 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.147745 26034 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:07:59.168493 26073 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:59.171546 25681 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:07:59.171919 25681 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:59.172637 25681 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.173591 22902 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:59.173643 25715 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.175443 25984 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:59.177947 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:59.178946 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:59.212016 22830 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:07:59.211966 22754 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:07:59.186858 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:59.191570 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:59.213709 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:07:59.200594 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.185702 25573 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:59.192379 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:59.215453 23445 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:59.215544 26073 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:07:59.215993 26073 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.216341 26073 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:59.216897 26029 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:59.215408 25715 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.217252 26029 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:59.218192 26029 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:59.218869 25715 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.219239 26029 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:59.220204 25976 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:59.220297 25681 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:59.220737 25976 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:59.221621 25976 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.222870 25976 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:59.169116 25741 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.226162 25741 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:59.226200 22896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 754) took 1431 ms (client timeout 1791 ms). Trace:
W20260709 13:07:59.226724 22896 rpcz_store.cc:269] 0709 13:07:57.794401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.794577 (+   176us) service_pool.cc:224] Handling call
0709 13:07:59.226032 (+1431455us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b540031f954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:59.226134 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880163584}
I20260709 13:07:59.228052 22901 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:07:59.224952 26073 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.232296 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:07:59.232930 22754 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:59.236784 26041 raft_consensus.cc:3055] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:07:59.237178 26041 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:07:59.240643 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 47019.7us
I20260709 13:07:59.245388 26120 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:59.246114 26120 consensus_queue.cc:1243] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:07:59.246685 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:07:59.248016 26157 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:59.248531 26157 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:59.248968 26157 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:59.249388 26073 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:07:59.254885 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:59.272426 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:59.272825 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:59.273042 22751 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
W20260709 13:07:59.272634 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 723984us
I20260709 13:07:59.279347 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.283010 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:59.281292 25797 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:59.291266 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2020) took 1071 ms. Trace:
I20260709 13:07:59.291683 22898 rpcz_store.cc:276] 0709 13:07:58.219520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.219628 (+   108us) service_pool.cc:224] Handling call
0709 13:07:58.507955 (+288327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:58.637428 (+129473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:58.637471 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:58.644153 (+  6682us) log.cc:844] Serialized 26 byte log entry
0709 13:07:58.790828 (+146675us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:58.790859 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.790918 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:59.146892 (+355974us) raft_consensus.cc:1685] finished
0709 13:07:59.146907 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:59.291025 (+144118us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b540034b7d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:59.291210 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8841,"raft.queue_time_us":1961,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":8582,"spinlock_wait_cycles":662214400,"thread_start_us":210183,"threads_started":2,"wal-append.queue_time_us":180622}
I20260709 13:07:59.293462 26034 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:59.293845 25986 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:59.295058 25986 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:59.295414 25986 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:07:59.296497 25986 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:59.243288 25689 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:07:59.299681 22751 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:59.286069 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.302546 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.306205 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:59.307148 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 300717us
I20260709 13:07:59.307811 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.311025 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.312127 22754 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:59.313596 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:59.316130 22753 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:59.287560 26111 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:59.318184 22898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:07:59.319474 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:59.319868 22898 raft_consensus.cc:606] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.320264 22898 raft_consensus.cc:614] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:59.322383 23453 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:59.322991 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.323050 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:59.326033 26129 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:59.326611 26129 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:59.326670 25850 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:59.327447 26129 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.329398 26050 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:59.329761 25816 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:59.330348 25816 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.329986 26050 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.330749 25816 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:59.321686 25986 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:59.332437 26050 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.309230 26038 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:59.313410 26138 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:59.337082 26077 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:59.338255 26077 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:59.339360 26004 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:07:59.308761 26015 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:59.307468 26141 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:07:59.340672 22898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
W20260709 13:07:59.342684 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.286882 25890 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:07:59.345577 25890 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:59.321545 26090 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:07:59.345909 25890 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:59.335225 26138 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:59.357023 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48540.3us
I20260709 13:07:59.346728 25890 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.315979 26091 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:59.318344 25997 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:59.363732 25996 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:07:59.364830 26090 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:59.372589 22901 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
W20260709 13:07:59.368220 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45315.9us
I20260709 13:07:59.380239 22901 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:07:59.383988 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:59.385860 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:59.387401 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:59.388123 22896 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:07:59.395196 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:59.396061 22900 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:07:59.398109 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:59.420626 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49658.8us
I20260709 13:07:59.384729 22901 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:07:59.443259 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:07:59.454361 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 56373.8us
I20260709 13:07:59.456377 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.458004 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:59.458689 22830 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:07:59.463102 22871 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:07:59.468180 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:07:59.468508 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:59.469292 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:07:59.470798 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.474701 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:59.482496 22830 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:07:59.487486 26038 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:07:59.487653 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.489950 25890 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:59.469506 25689 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:07:59.460703 25960 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:59.501597 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1496) took 1125 ms. Trace:
I20260709 13:07:59.469775 26075 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:59.503132 26090 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.467990 26086 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:07:59.525424 22902 rpcz_store.cc:276] 0709 13:07:58.376085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.578332 (+202247us) service_pool.cc:224] Handling call
0709 13:07:59.501437 (+923105us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b5400275954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:07:59.501550 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1366564096}
I20260709 13:07:59.506804 26111 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:07:59.508193 22753 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:59.302218 26041 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.500990 26157 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:07:59.533661 22829 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:59.519025 22903 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:07:59.529639 22905 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:59.539628 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 215840us
I20260709 13:07:59.514899 26034 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:07:59.547920 23446 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:07:59.550930 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 174340us
I20260709 13:07:59.552482 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:59.539525 25960 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:07:59.564479 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:59.529417 25689 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.569531 26122 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:59.537220 25681 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:07:59.530877 25816 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.570319 26073 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:07:59.573272 25970 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:07:59.577600 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 109756us
I20260709 13:07:59.584632 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.565374 26034 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.564993 26050 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:07:59.595460 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:07:59.580132 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.587126 22871 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.601841 26075 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:07:59.565997 25997 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:07:59.604552 26019 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:59.604967 26019 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:59.605808 26019 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.607009 26019 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:59.613509 25970 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:07:59.588618 26178 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:59.592228 22823 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:07:59.614306 26073 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:07:59.580425 26091 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.614459 25970 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:07:59.588137 26165 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:07:59.614732 26073 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:07:59.556974 26038 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:07:59.618105 26178 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:59.623164 26073 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.624547 26073 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:59.616008 26034 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.627907 22753 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:59.628904 22828 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:07:59.640424 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:59.643991 22827 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:07:59.644142 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:59.632973 26038 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.645279 22901 raft_consensus.cc:2376] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6. Re-sending same reply.
I20260709 13:07:59.645537 26038 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.649187 26038 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.649632 22902 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:07:59.646294 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.651029 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:07:59.651867 26111 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:59.652426 26031 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:07:59.652595 25689 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.652668 26111 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:59.653188 26111 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.653535 26111 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:59.654189 25816 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.655051 26034 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:07:59.615612 25573 consensus_queue.cc:1175] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3b626c41600d4737a178ebab8a7ebfc2" responder_term: 10 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:59.658789 22826 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:07:59.660288 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:07:59.669724 22902 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:59.661808 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:59.661823 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:07:59.659679 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:07:59.631336 26091 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.665653 22899 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:59.666153 26033 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.649712 26038 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:07:59.672811 23451 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:07:59.675036 22898 raft_consensus.cc:606] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:07:59.671662 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:07:59.675452 22898 raft_consensus.cc:614] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:59.677019 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:07:59.677829 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:59.678032 22826 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:07:59.678387 23446 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:07:59.680135 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:59.681630 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:59.682605 26034 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:07:59.684976 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.694134 22904 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:07:59.695739 22900 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:07:59.697073 22896 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:59.674098 26031 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:07:59.721812 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.696972 26033 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.739343 26033 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.743779 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:07:59.709923 26031 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.767963 22901 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:07:59.732162 26147 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:07:59.743454 26108 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:59.683444 26111 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.767916 26031 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:07:59.797878 22754 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:59.780861 26108 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.782558 26147 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.802032 26108 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:59.847213 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:59.847993 22749 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:07:59.849375 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:59.813475 26141 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:07:59.838933 26170 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.860495 26141 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.862552 23446 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:07:59.854126 26149 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:59.898593 26111 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:07:59.870625 26170 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:07:59.928233 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:07:59.929001 22750 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:07:59.930490 23419 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:07:59.889130 26149 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.975750 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:07:59.976500 22896 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:59.976598 22900 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
I20260709 13:07:59.981570 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:07:59.893087 26091 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:07:59.998068 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.003472 22901 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:07:59.954126 26111 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.998685 26014 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:00.019326 25996 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:00.000504 26015 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.999003 26149 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:00.068317 22755 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:08:00.078269 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:00.001850 26164 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.081944 26110 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.084388 22755 pending_rounds.cc:85] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 7
I20260709 13:08:00.099674 22755 pending_rounds.cc:107] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:08:00.099977 22755 raft_consensus.cc:2889] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:00.082918 25593 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:00.104115 26164 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.104362 25593 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.000950 26029 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:08:00.106223 25593 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.082794 25996 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:00.000824 25986 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:08:00.107470 25996 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.108196 25986 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.108624 25986 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:00.100020 26015 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.111608 26015 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.001068 25991 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:08:00.112133 26015 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:00.112474 26015 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.112768 26110 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.112810 26015 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:00.114835 26110 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.034485 25984 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:08:00.116501 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.102205 26120 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:00.117574 26120 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:00.119491 26120 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:00.121307 26122 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.121803 26122 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.123662 26122 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:00.101711 25997 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.124956 26015 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.125439 25997 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.126921 26015 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.127900 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.128675 26141 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.106228 26164 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:00.135821 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.140179 22747 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:00.038888 26014 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.110469 25986 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:00.138589 26019 consensus_meta.cc:287] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.004s
I20260709 13:08:00.153616 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.154386 22747 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:00.156440 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.157459 22747 raft_consensus.cc:2410] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:08:00.035737 26073 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 13:08:00.158262 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:00.137210 26170 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.127377 25997 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:00.146178 26014 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:00.167749 26184 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:00.173231 26147 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:00.180074 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 101896us
I20260709 13:08:00.183717 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.185238 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:00.185829 22826 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:00.186031 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:00.186803 22897 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:00.196398 22897 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:00.183585 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:00.197808 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:00.199396 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.198393 22824 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:00.200408 22897 raft_consensus.cc:2393] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:00.201928 26038 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:00.203497 26038 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:00.204578 26038 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.177687 26184 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:00.085361 26111 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.221169 22755 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:00.227237 26184 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:00.264987 26034 consensus_meta.cc:287] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.566s	user 0.006s	sys 0.005s
I20260709 13:08:00.265803 26034 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:00.289191 22897 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:00.081845 22896 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:00.297755 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:00.300336 22829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.075s	sys 0.112s
W20260709 13:08:00.300741 22829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.076s	sys 0.112s
I20260709 13:08:00.302590 26034 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:08:00.307077 26194 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:00.307829 26194 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:00.308936 26194 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.312397 26146 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:00.299702 26120 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:00.313385 26019 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.314018 26120 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.314211 26019 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.316658 26120 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.316823 26019 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:00.287801 26189 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:08:00.142489 26129 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.161s
I20260709 13:08:00.320808 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1630) took 1466 ms. Trace:
I20260709 13:08:00.321288 22823 rpcz_store.cc:276] 0709 13:07:58.854307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.855285 (+   978us) service_pool.cc:224] Handling call
0709 13:07:58.855515 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.614670 (+759155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.320575 (+705905us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b5400381dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:00.320738 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1693557120}
I20260709 13:08:00.340871 26149 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.342903 26149 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.346254 26111 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.348346 26108 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.351640 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:00.352375 22823 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:00.352430 26184 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:00.354123 26108 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.356163 25996 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:00.356724 25996 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.357553 22904 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:00.358799 25996 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.359498 26184 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:00.360556 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:00.360721 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:00.362766 26111 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.363205 26111 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.366655 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.367565 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:00.367928 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:00.368129 22829 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:00.368476 22827 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:00.370157 26157 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.337085 26014 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.380339 26014 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.365126 26111 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:00.382961 26031 consensus_meta.cc:287] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 13:08:00.383375 26031 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.384313 22749 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:00.385987 26031 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.390143 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:00.396808 22747 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:08:00.394348 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:00.399202 22905 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:00.392637 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:00.336524 26192 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.400871 26192 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.402935 26192 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.391350 26086 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.413738 26086 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.415741 26086 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.418571 26086 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.418876 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.419070 26086 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.421021 26086 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:00.396348 26148 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:00.421670 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.421986 26148 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.424655 26148 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:00.425809 22901 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:00.426278 25850 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.426754 25850 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.434408 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:00.436528 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.437152 22755 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:00.438251 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:00.440277 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:00.440968 22828 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:00.444139 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.444873 22901 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:00.446452 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:00.542322 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 104132us
I20260709 13:08:00.652293 26111 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:00.653083 26111 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.655720 26129 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:00.656267 26129 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:00.656714 26129 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:00.660177 26091 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.660792 26091 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.661881 26086 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.662528 26086 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.663275 26091 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.664494 25991 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:00.665776 26077 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.666402 26077 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.668762 26077 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.670799 26157 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:00.671438 26157 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.678407 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.679879 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.438717 26012 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:00.680711 22756 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:00.682494 25944 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:00.683166 25944 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.665123 25991 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.685808 25944 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.686010 25991 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.690284 26012 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:00.693205 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:00.694831 26110 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:00.695299 26110 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.695360 26157 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.696064 26110 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:00.679965 25573 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.730226 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.730983 22756 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:00.733431 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:00.736583 26130 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:00.736959 25996 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:00.737686 25996 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.737236 26130 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.738181 25996 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:00.740070 26130 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.742041 25573 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.744661 25573 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:00.746318 26031 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:08:00.746225 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 299838us
I20260709 13:08:00.746848 26031 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.747215 26031 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:00.664976 26086 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.750463 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.751494 22901 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:00.751868 22752 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:00.752631 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:00.753086 26038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.200s	sys 0.321s
W20260709 13:08:00.753526 26038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.200s	sys 0.321s
I20260709 13:08:00.754245 26111 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.755198 22905 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:00.755705 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:00.758509 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:00.759289 22828 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:00.759624 22823 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:00.703426 26148 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:00.774729 26148 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.773563 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:00.768725 22831 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:00.757373 26077 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:00.782721 22747 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:00.784312 23419 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:00.775146 26148 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:00.693019 26012 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.788766 26130 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:00.806133 26130 raft_consensus.cc:697] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:00.806902 26130 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:00.812340 26191 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:00.782328 26184 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.788987 26077 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:00.785809 26031 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.826177 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.827595 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:00.829679 22897 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:08:00.833951 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:00.832916 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.836390 22901 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:00.837798 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.843046 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.849668 22905 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:08:00.852532 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:00.853284 22901 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:00.853426 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:00.819126 26014 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.847163 26149 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:08:00.852473 26197 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:08:00.870432 22752 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:00.811751 26129 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.871737 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:00.875762 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.876437 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58800 (request call id 23) took 1537 ms. Trace:
I20260709 13:08:00.877681 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:00.881659 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.846271 25573 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:00.870566 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 894) took 1302 ms. Trace:
I20260709 13:08:00.887676 22753 rpcz_store.cc:276] 0709 13:07:59.568408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.568572 (+   164us) service_pool.cc:224] Handling call
0709 13:07:59.568871 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.628205 (+ 59334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.870242 (+1242037us) spinlock_profiling.cc:243] Waited 58.7 ms on lock 0x7b54002aab54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:00.870441 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135111552}
I20260709 13:08:00.820326 26184 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.886554 22898 rpcz_store.cc:276] 0709 13:07:59.338759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.340557 (+  1798us) service_pool.cc:224] Handling call
0709 13:08:00.876197 (+1535640us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b5400184554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74df5853 00007fcb79dc360a 00007fcb733497d4 00007fcb73349762 00007fcb73349691 00007fcb7334960c 00007fcb733488c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:00.876335 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760178560}
I20260709 13:08:00.889147 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.886979 22755 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:00.886246 22748 raft_consensus.cc:2368] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10.
I20260709 13:08:00.893657 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.893726 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.895246 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:00.895473 22748 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:08:00.890085 22905 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:08:00.895983 22752 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:00.898463 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:00.836470 26148 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.897025 22753 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
W20260709 13:08:00.908113 22750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.021s	sys 0.024s
W20260709 13:08:00.909186 22750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.021s	sys 0.024s
W20260709 13:08:00.913947 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 165160us
I20260709 13:08:00.879714 26014 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:00.909907 25952 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:00.919610 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:00.890910 25573 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:00.923164 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:00.934262 22903 raft_consensus.cc:606] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:08:00.934659 22903 raft_consensus.cc:614] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:00.896836 26149 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:08:00.920583 25952 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.908986 26077 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:00.926050 26129 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:00.927731 26184 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:00.917474 25850 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:00.923038 26032 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:01.002779 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 143566us
W20260709 13:08:01.003789 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 95571.4us
W20260709 13:08:00.972148 26198 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:00.964195 25960 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:00.970288 25952 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:01.011478 25797 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:01.031874 26091 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
W20260709 13:08:01.050343 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 130795us
I20260709 13:08:01.051963 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:01.052728 22903 raft_consensus.cc:606] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:01.052309 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:01.032548 26130 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:01.053158 22903 raft_consensus.cc:614] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:01.040519 25797 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:01.039162 26149 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:01.058355 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 45290.1us
I20260709 13:08:01.041169 25960 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.051177 26031 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.030583 26032 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.050654 26212 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:01.069741 26212 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.070519 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:08:01.071847 26212 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.047278 25996 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.075368 22901 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:01.061335 25797 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:01.082387 26148 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:01.082517 26202 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:01.057667 26130 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.090957 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2524) took 1108 ms. Trace:
I20260709 13:08:01.092036 26130 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:01.093607 25960 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:01.081099 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:01.085638 26219 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:01.096066 26219 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.096493 26219 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:01.096982 25996 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.079221 26213 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:01.098479 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:01.086326 26221 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.087308 25573 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.087427 26110 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.101567 25797 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:01.104301 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2041) took 1918 ms. Trace:
I20260709 13:08:01.104619 26110 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.088053 26014 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.079393 26031 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:01.091442 22750 rpcz_store.cc:276] 0709 13:07:59.982629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.984393 (+  1764us) service_pool.cc:224] Handling call
0709 13:08:00.005479 (+ 21086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:00.319945 (+314466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:00.320332 (+   387us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:00.324269 (+  3937us) log.cc:844] Serialized 26 byte log entry
0709 13:08:00.910751 (+586482us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:00.910789 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.910850 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:01.090711 (+179861us) raft_consensus.cc:1685] finished
0709 13:08:01.090737 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:01.090885 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101329,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":870048,"threads_started":2,"wal-append.queue_time_us":722103}
I20260709 13:08:01.065804 26149 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.099756 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:01.101843 26215 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:01.102695 26216 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:08:01.105710 22897 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:01.104673 22899 rpcz_store.cc:276] 0709 13:07:59.185683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.187256 (+  1573us) service_pool.cc:224] Handling call
0709 13:07:59.187516 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.673264 (+485748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.104076 (+1430812us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400184554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:01.104247 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1098829696}
I20260709 13:08:01.099520 26193 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:01.106113 26130 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:01.110092 26149 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:01.113615 23452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 628) took 2202 ms (client timeout 1585 ms). Trace:
I20260709 13:08:01.100903 25996 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:01.116608 26032 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.097905 26213 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.117008 25996 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.119593 26213 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.112787 26193 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.120428 26193 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:01.122128 25681 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:01.122556 25681 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:01.123366 25681 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.124565 25681 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:01.114187 23452 rpcz_store.cc:269] 0709 13:07:58.909827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.909940 (+   113us) service_pool.cc:224] Handling call
0709 13:08:01.112328 (+2202388us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b5400750154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:01.112438 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1513671936}
I20260709 13:08:01.114708 26031 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:01.114346 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:01.126573 22797 leader_election.cc:400] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10.
I20260709 13:08:01.126797 22797 leader_election.cc:403] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:01.128600 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:01.130857 26031 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.131724 26054 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:01.127538 26130 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.135370 26130 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.135000 26219 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.110914 26215 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.138772 22797 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:01.141729 26215 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:01.143006 22747 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:08:01.144474 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.144763 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.145952 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:01.106606 26221 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.128196 26223 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:01.148229 26223 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.152197 26223 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:01.150298 26195 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:01.153080 26195 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.155326 26195 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.156054 26193 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:01.157986 26193 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:01.119592 25996 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.163129 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:01.151749 26224 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.147732 26130 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:01.159951 26213 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:08:01.164067 26212 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:01.164803 26130 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.165611 26221 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:01.165769 26190 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.102s	user 0.008s	sys 0.003s
I20260709 13:08:01.161229 25944 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:01.166509 25944 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:01.166877 25944 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:01.167534 26130 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:01.168066 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1639) took 1540 ms. Trace:
W20260709 13:08:01.168408 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54128.4us
I20260709 13:08:01.169117 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:01.170758 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:01.159543 25573 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:01.159376 26218 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:08:01.173975 25967 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:01.168114 26224 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.171482 25573 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.175251 25573 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:01.176105 26224 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.171384 26195 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:01.174711 26218 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:01.168594 22825 rpcz_store.cc:276] 0709 13:07:59.627916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.628476 (+   560us) service_pool.cc:224] Handling call
0709 13:07:59.628848 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.666250 (+ 37402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.666278 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:59.666639 (+   361us) log.cc:844] Serialized 26 byte log entry
0709 13:08:00.062897 (+396258us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:00.062952 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:00.063041 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:01.165110 (+1102069us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:01.167743 (+  2633us) raft_consensus.cc:1685] finished
0709 13:08:01.167767 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:01.168002 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34387,"raft.queue_time_us":35263,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":50532096,"thread_start_us":395953,"threads_started":1,"wal-append.queue_time_us":396333}
I20260709 13:08:01.172427 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:01.248080 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 75749.2us
I20260709 13:08:01.164043 26007 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:08:01.273720 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:01.285008 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.173557 26193 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:01.359691 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 768) took 1714 ms. Trace:
W20260709 13:08:01.375121 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 86878.3us
I20260709 13:08:01.287247 26007 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.376416 22902 rpcz_store.cc:276] 0709 13:07:59.687624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.688091 (+   467us) service_pool.cc:224] Handling call
0709 13:07:59.688475 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.688913 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:59.732344 (+ 43431us) log.cc:844] Serialized 14 byte log entry
0709 13:08:01.176672 (+1444328us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b1800055360. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b23a 00007fcb74e05a2a
0709 13:08:01.357854 (+181182us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:01.358368 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:08:01.358494 (+   126us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:01.358933 (+   439us) log.cc:844] Serialized 14 byte log entry
0709 13:08:01.359326 (+   393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.359391 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:01.359485 (+    94us) raft_consensus.cc:1685] finished
0709 13:08:01.359501 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:01.359629 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45048,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":73,"raft.queue_time_us":539,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":8168,"spinlock_wait_cycles":48135680,"thread_start_us":180710,"threads_started":1,"wal-append.queue_time_us":1625220}
W20260709 13:08:01.381760 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 153499us
W20260709 13:08:01.378069 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 127499us
I20260709 13:08:01.383611 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:01.254299 26195 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:01.247509 26130 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.384943 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.396420 23445 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:01.383800 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.385056 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:01.384202 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:01.385594 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:01.397819 23444 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:01.247331 26218 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:01.287890 26029 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:01.247061 26178 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:01.381348 26007 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:01.383165 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1511) took 1200 ms. Trace:
I20260709 13:08:01.374971 25967 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.378037 26219 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.405112 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.360508 26224 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.360539 26110 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:01.381498 26222 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:01.391657 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.414959 26218 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.415359 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 897) took 1622 ms. Trace:
I20260709 13:08:01.415904 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.393422 26090 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:01.453457 23445 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:01.387884 26148 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:01.457617 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.395730 26195 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:01.416630 26178 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.417004 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.474625 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.475358 23453 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:01.476408 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.418286 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.419183 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 899) took 1496 ms. Trace:
I20260709 13:08:01.478169 22749 rpcz_store.cc:276] 0709 13:07:59.922851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.923021 (+   170us) service_pool.cc:224] Handling call
0709 13:07:59.923366 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:00.384642 (+461276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.418988 (+1034346us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400046854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:01.419133 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1059664768}
I20260709 13:08:01.385793 25741 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.428256 23444 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:01.479969 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.480532 23444 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:01.481274 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.385807 26014 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.481695 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.482033 26014 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.482537 22896 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:01.483505 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.484210 22899 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:01.410035 22904 rpcz_store.cc:276] 0709 13:08:00.182832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.183439 (+   607us) service_pool.cc:224] Handling call
0709 13:08:01.382965 (+1199526us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b540034b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:01.383096 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398077312}
I20260709 13:08:01.484758 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.388131 26209 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.485327 22901 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:01.440825 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.485534 26209 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.486290 22902 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:01.450531 23451 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:01.453078 22754 rpcz_store.cc:276] 0709 13:07:59.793300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.793505 (+   205us) service_pool.cc:224] Handling call
0709 13:07:59.793835 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:59.798270 (+  4435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.415168 (+1616898us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b540016a654. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:01.415311 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8834304}
I20260709 13:08:01.396185 26193 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:01.487524 26209 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.448856 23452 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:01.484122 26014 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.487474 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:01.468709 23446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1086) took 1728 ms (client timeout 2242 ms). Trace:
I20260709 13:08:01.488530 23451 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:08:01.488785 23446 rpcz_store.cc:269] 0709 13:07:59.740799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.741094 (+   295us) service_pool.cc:224] Handling call
0709 13:08:01.468540 (+1727446us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400554454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:01.468661 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271624064}
I20260709 13:08:01.488842 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.444738 26032 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.469550 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:01.493376 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 106205us
I20260709 13:08:01.448446 26031 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.497915 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.488220 23452 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:01.453590 26229 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:01.475586 26077 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.409771 26130 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.495680 23453 raft_consensus.cc:2410] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:08:01.450421 25996 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:08:01.499776 23452 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.526698 23452 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:01.493505 26184 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.446624 25967 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.494669 25960 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.454416 26224 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:01.530845 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 111067us
I20260709 13:08:01.451609 26110 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.543310 23450 raft_consensus.cc:2410] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:01.534464 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:01.557853 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.479702 26148 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.507289 26212 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:01.563104 23453 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:01.507313 26218 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:01.495021 25741 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.563858 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:01.563927 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.524559 26029 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:01.488132 26193 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.554531 26091 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:01.533357 26120 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.498965 26032 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.491962 26223 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.458328 26222 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.588624 26222 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.568534 23453 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:01.590010 23447 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:01.590142 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.590873 23453 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:01.591979 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:01.592403 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.592832 23453 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:01.593932 26029 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.594658 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.595270 23445 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:08:01.596268 23445 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:08:01.474407 26090 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.574095 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:01.555258 26014 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:01.504528 26031 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.559092 26209 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.561228 26111 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.582819 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.585722 26007 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.535887 26077 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.604002 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.604774 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:01.638530 22754 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:01.606765 23451 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:01.603741 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:01.543123 26184 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.555867 25960 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.613445 26014 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.614861 26223 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.615448 25741 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.618122 25967 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.619755 26110 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:01.621480 26041 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:01.622433 26187 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.551s	user 0.006s	sys 0.020s
I20260709 13:08:01.625152 25996 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:01.626030 26149 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:01.626106 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.584558 26091 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.627746 25573 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:01.631256 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.522068 26229 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.634007 26234 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:01.634382 26031 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.600049 26120 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.578049 25850 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.634855 26217 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:01.635946 23449 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:01.644304 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:01.642921 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:01.648775 23452 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:01.652410 23444 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:08:01.632795 25944 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.597792 26236 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.567898 26212 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:01.632795 25952 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.000s
I20260709 13:08:01.707572 25952 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.708274 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.710094 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.676355 26149 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:01.668442 26238 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.647054 26209 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:01.668978 26226 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:08:01.646952 26111 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.669870 26077 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.640209 26235 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.670817 26237 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.674551 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:01.652123 25991 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:01.676762 22897 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:08:01.674006 26012 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:01.674791 25797 consensus_meta.cc:287] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.000s
I20260709 13:08:01.695940 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:01.743542 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 69057.1us
I20260709 13:08:01.697031 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:01.697804 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.744961 22904 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:08:01.699375 22896 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:08:01.692653 23446 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 3.
I20260709 13:08:01.704988 23450 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:01.746565 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.705523 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.747709 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.710806 22831 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:01.748759 22827 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:01.712271 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.749876 22830 raft_consensus.cc:2368] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10.
I20260709 13:08:01.610003 25986 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:01.685734 25996 consensus_queue.cc:1243] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:08:01.715181 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.692159 26234 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.723582 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:01.760011 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:01.724709 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.721216 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:01.760958 22899 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:01.729477 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.761713 26007 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.683012 25681 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:08:01.705914 26217 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:01.707026 26231 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.736403 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.710019 25952 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:01.739707 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:01.740447 26077 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.745630 22754 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:01.747789 22901 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:08:01.712247 26054 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:01.713809 26239 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:01.754793 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53971.1us
I20260709 13:08:01.721282 26192 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.716024 26230 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:01.757859 22905 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:01.714807 22903 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:08:01.772956 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.774186 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.758936 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:01.781841 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.774469 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:01.729097 26178 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.710480 25850 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:01.706740 26129 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:08:01.763772 22747 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:01.764989 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.794854 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:01.795383 22823 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:01.709040 22825 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:01.753479 26237 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.766722 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:01.766844 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.769241 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:01.770324 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.739137 25991 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:01.784456 22831 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:01.836438 22828 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:01.838310 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:01.754446 26233 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:01.754591 26130 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.787896 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:01.772750 26217 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:01.792364 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:01.844035 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51735.7us
W20260709 13:08:01.845832 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 84853.6us
I20260709 13:08:01.846144 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.846894 22896 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:01.847131 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.847390 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:01.847714 22898 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:01.848054 22904 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:01.848562 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.839368 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.791380 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.849360 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.849754 22829 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:01.850468 26157 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:01.850512 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.851008 26157 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.851382 26157 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:01.851666 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.852648 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.853693 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:01.854404 22747 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:01.757337 26216 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:08:01.855597 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.875352 22896 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:08:01.713943 26236 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.872092 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.791443 26007 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.791530 26110 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:01.854524 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.876385 22901 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:01.886314 22902 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:01.791479 26244 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:01.892203 22899 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:01.789443 26148 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:01.729933 26238 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.881225 26130 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:08:01.884187 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 287601us
I20260709 13:08:01.873816 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:01.928057 22755 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:01.879958 22823 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:01.929097 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.929646 22823 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:01.930358 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:01.930912 22823 raft_consensus.cc:2368] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:01.931515 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.932020 22823 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:01.932709 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.742141 26235 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.887786 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.938732 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.840420 25991 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:01.894424 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 258223us
I20260709 13:08:01.855227 26178 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.948573 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:01.908385 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.901684 26242 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.912577 22754 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:08:01.914134 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.915969 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.963573 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.965022 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.906211 22750 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:01.873769 26254 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.906638 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.780797 26149 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.922089 25960 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:01.883381 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:01.791086 26054 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.874173 26233 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:01.787627 26231 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.789103 26077 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.790876 26192 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.942760 22896 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:01.945914 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:01.951200 22900 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:01.952178 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.991214 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:01.907639 26244 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.909484 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:01.993417 26192 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.994689 26238 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.952961 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.997347 22901 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:01.997505 26014 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.954038 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.998999 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.954602 22831 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:08:01.999399 26209 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.000432 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:02.001224 22831 raft_consensus.cc:2410] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:08:02.001260 26235 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.002382 22831 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:08:01.987133 26246 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:02.002938 26246 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:02.003002 26244 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.003443 22831 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:02.003700 26244 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.953022 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.835251 26230 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.004283 26244 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.005113 26234 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.006167 26230 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.959723 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.793668 26239 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.961752 25573 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.008244 26239 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.969561 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:01.990058 25960 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.970395 22829 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:08:01.882786 26216 consensus_queue.cc:1243] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:08:02.009191 25960 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.926142 26193 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:02.010457 26193 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.011166 25960 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.011662 25960 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:02.012118 26215 consensus_meta.cc:287] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.870s	user 0.009s	sys 0.000s
I20260709 13:08:02.012456 26193 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.924625 22830 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:02.009090 22903 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:08:02.012709 26215 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.927764 26130 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:01.982115 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:02.011718 26090 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.020408 22825 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:01.982137 22827 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:01.904402 26007 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.953868 22899 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 3.
I20260709 13:08:02.048744 22755 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:08:02.003834 26246 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.008296 25944 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.926740 26091 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:01.972980 22753 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:02.008244 25850 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:02.013449 26120 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.014112 26244 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.016106 22826 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
W20260709 13:08:02.016501 26213 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.851s	user 0.007s	sys 0.001s
I20260709 13:08:01.981168 26254 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.907351 26110 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:01.980158 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:01.957834 26232 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 13:08:02.021327 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 75487.1us
I20260709 13:08:02.070590 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.071806 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:01.999228 26014 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:02.053907 22824 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:02.054375 22822 raft_consensus.cc:2376] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10. Re-sending same reply.
I20260709 13:08:01.975466 26241 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:02.055941 26216 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:02.057533 22898 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:02.012163 25960 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.062450 26111 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:01.990883 26229 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.020133 26243 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:08:02.065092 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 228115us
I20260709 13:08:02.066288 22823 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:08:02.067517 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.051424 26192 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:02.048633 26212 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.070107 22748 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:02.019843 26236 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:01.999250 26148 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:08:02.059113 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.094111 22753 raft_consensus.cc:2410] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:08:01.971611 26242 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:01.961459 26178 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.067191 26213 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10.
I20260709 13:08:02.083298 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:02.074992 26014 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.077821 26244 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.078603 26031 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:02.074494 26157 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.079860 25850 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:02.117013 25850 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.091536 26233 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.118135 26157 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.076735 26193 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:08:02.109159 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:02.091148 26192 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.112578 26007 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.113577 26264 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:02.114001 26031 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.114409 26054 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.115047 26077 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:02.115909 26254 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.092015 26130 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.080332 26241 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.081490 26216 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.072925 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:02.107136 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 319281us
I20260709 13:08:02.102376 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:02.119915 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 171397us
I20260709 13:08:02.119628 25850 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:02.298424 25850 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:02.122030 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 153690us
I20260709 13:08:02.301815 26031 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:02.303195 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:02.110569 26014 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:02.306087 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 233095us
I20260709 13:08:02.310328 25967 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:02.311079 25967 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:02.311717 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:02.315007 26110 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.316708 26077 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:02.317294 26077 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.318279 26077 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:02.318310 26178 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:02.313244 22899 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:02.322927 25960 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:02.323055 26007 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:02.115620 25991 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.317847 26111 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
W20260709 13:08:02.325991 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 223605us
I20260709 13:08:02.326601 26111 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:02.327054 26111 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:02.328122 25991 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.328461 26111 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.330498 26111 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:02.292517 25986 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:02.333895 26054 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:02.334558 26054 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:02.334115 25986 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.335923 26054 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.317675 26130 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:02.313017 25967 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.339026 26129 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:02.339697 26129 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:02.288516 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:02.202872 26195 consensus_meta.cc:287] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.704s	user 0.005s	sys 0.005s
I20260709 13:08:02.346002 26192 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:02.346539 23450 pending_rounds.cc:85] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 8
I20260709 13:08:02.346968 23450 pending_rounds.cc:107] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:08:02.347339 23450 raft_consensus.cc:2889] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:02.346935 26195 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.351686 26195 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:02.300179 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 190881us
I20260709 13:08:02.353145 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:02.354970 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:02.355736 26110 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:02.356227 26110 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.356591 26110 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:02.283221 22752 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:02.337097 25986 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.362754 26120 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:02.363687 26178 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:08:02.372954 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 52517.5us
I20260709 13:08:02.377027 23444 pending_rounds.cc:85] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 7
I20260709 13:08:02.377383 23444 pending_rounds.cc:107] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:08:02.377656 23444 raft_consensus.cc:2889] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:02.377730 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 72409.8us
I20260709 13:08:02.378608 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:02.378891 26241 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:02.380295 26271 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:02.235761 26217 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.380726 26271 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.381150 26271 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:02.381675 22749 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:02.202244 26215 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.300153 25850 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.385979 26032 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:02.396315 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.340934 26129 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.409823 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:02.412845 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:02.119782 26231 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.413798 22899 raft_consensus.cc:2410] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:08:02.517897 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:02.532971 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.383636 26264 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.386726 26216 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.383085 26217 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.607261 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:02.607617 22899 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:02.420825 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:02.608363 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:02.417136 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:02.613608 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:02.613986 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:02.615181 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:02.616838 26041 consensus_meta.cc:287] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.929s	user 0.010s	sys 0.014s
I20260709 13:08:02.621783 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:02.622910 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38478 (request call id 2537) took 1019 ms. Trace:
I20260709 13:08:02.627756 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.426935 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:02.612223 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:02.421893 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.419004 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:02.530663 22828 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:02.123598 26237 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.540518 26184 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:02.655197 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43533us
I20260709 13:08:02.664665 23446 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:02.210165 26242 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.656972 22900 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:08:02.545437 26194 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:02.667193 23449 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:08:02.669951 22897 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:02.672259 22904 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:02.121896 26244 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.674422 26192 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.675065 26007 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:02.675746 26007 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.646948 22749 rpcz_store.cc:276] 0709 13:08:01.603715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.603812 (+    97us) service_pool.cc:224] Handling call
0709 13:08:02.622620 (+1018808us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54000780d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:02.622815 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712841088}
I20260709 13:08:02.678488 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.678946 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.679308 22754 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:02.679721 22749 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:02.682154 26272 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:02.660522 25850 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:02.680320 22751 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:02.688339 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:02.689049 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 2135) took 1544 ms. Trace:
I20260709 13:08:02.689425 22751 rpcz_store.cc:276] 0709 13:08:01.144399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.144541 (+   142us) service_pool.cc:224] Handling call
0709 13:08:02.688862 (+1544321us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b540011b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:02.688977 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3474567936}
I20260709 13:08:02.680392 22755 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:02.679790 26194 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.646000 26264 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:02.739521 22904 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:08:02.780076 22903 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:02.738757 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 57955.1us
W20260709 13:08:02.782043 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 379514us
I20260709 13:08:02.682880 26272 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.683648 26261 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.718801 26192 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.719832 26038 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.792136 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.738830 26224 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.800560 22752 raft_consensus.cc:2410] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:08:02.800874 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 406452us
W20260709 13:08:02.804028 26012 consensus_meta.cc:287] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.047s	user 0.011s	sys 0.011s
I20260709 13:08:02.804936 26231 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.805625 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.806007 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.766772 26271 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.765482 26238 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:08:02.814968 22747 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:02.815335 26249 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.815825 26234 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.816028 26249 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:02.816339 26234 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:02.809514 26031 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.007s
I20260709 13:08:02.813715 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:02.826049 26247 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:02.826500 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:02.826835 26247 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.828193 26251 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:02.828997 26251 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:02.827315 22823 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:02.825174 22755 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:02.828217 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.829173 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:02.835745 22903 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:08:02.833019 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 604) took 1070 ms. Trace:
W20260709 13:08:02.830060 26077 consensus_meta.cc:287] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.006s
I20260709 13:08:02.836869 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.826368 26219 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:02.840333 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:02.838383 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:02.841574 22754 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:02.845559 22831 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:08:02.851034 26184 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:02.851459 26184 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:02.854094 26157 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:02.854821 26157 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.855732 22822 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:08:02.857089 26157 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.858652 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:02.859536 25991 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:02.859966 25991 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.860347 25991 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:02.836758 22830 rpcz_store.cc:276] 0709 13:08:01.750997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.751140 (+   143us) service_pool.cc:224] Handling call
0709 13:08:02.798557 (+1047417us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400090bd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:02.821390 (+ 22833us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369964928}
I20260709 13:08:02.862138 26194 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.865694 26184 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.866900 26184 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:02.869431 26218 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.840432 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1656) took 1002 ms. Trace:
I20260709 13:08:02.870069 26218 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.870471 26244 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.872069 26218 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.866963 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1649) took 1076 ms. Trace:
I20260709 13:08:02.870077 22829 rpcz_store.cc:276] 0709 13:08:01.794986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.852454 (+ 57468us) service_pool.cc:224] Handling call
0709 13:08:02.769276 (+916822us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54000ad2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:02.797461 (+ 28185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269591936}
I20260709 13:08:02.873436 22828 rpcz_store.cc:276] 0709 13:08:01.790590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.909196 (+118606us) service_pool.cc:224] Handling call
0709 13:08:02.866800 (+957604us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54001fa9d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:02.866914 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":681782016}
I20260709 13:08:02.811199 26038 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.873967 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:02.874567 22830 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:02.876387 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:02.876976 22831 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:02.877374 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:02.880542 23445 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:02.820526 26231 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.881804 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:02.880483 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:02.883140 23444 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:02.883195 26231 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:02.883672 26240 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.111s	user 0.002s	sys 0.000s
I20260709 13:08:02.873112 22903 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
W20260709 13:08:02.885478 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.888566 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:02.889978 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:02.891403 26129 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:02.891917 26129 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.892355 26129 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:02.895928 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.822561 26031 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.608816 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:02.843122 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:02.905859 22828 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:02.910681 26231 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:02.911155 26231 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.911537 26231 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:02.837509 26261 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.912873 22825 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
W20260709 13:08:02.913949 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.842342 26007 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.916235 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:02.916600 26007 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:02.917706 22755 raft_consensus.cc:2368] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:02.918641 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.919478 22755 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:02.920336 26247 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.920485 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:02.920938 26247 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:02.923528 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.926057 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 611) took 1152 ms. Trace:
I20260709 13:08:02.840296 26219 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:02.926379 26261 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.926471 22826 rpcz_store.cc:276] 0709 13:08:01.773878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.773977 (+    99us) service_pool.cc:224] Handling call
0709 13:08:02.925716 (+1151739us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54002c9054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:02.925891 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538371840}
I20260709 13:08:02.928633 26219 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.929271 26219 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:02.929666 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:02.930410 22897 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:02.916301 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 104666us
I20260709 13:08:02.935451 25850 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:02.935765 23448 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:02.935914 25850 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:02.929239 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:02.937819 26244 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:02.938294 26244 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.938678 26244 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:02.937705 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:02.925034 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.941669 22752 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 4.
I20260709 13:08:02.925257 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:02.943869 22830 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:02.945189 23444 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 4.
I20260709 13:08:02.946727 22822 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:02.947721 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:02.948392 22831 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:08:02.947818 22753 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:02.920364 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.839514 26272 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.947659 26234 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.838312 26077 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.954667 22753 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:02.956024 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.956754 26077 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:02.959896 26014 consensus_meta.cc:287] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.010s
I20260709 13:08:02.960049 22751 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:02.960309 26014 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.962090 26264 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:02.962621 26031 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:02.963973 26038 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.966984 22898 raft_consensus.cc:2368] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:02.849040 26110 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:02.968070 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66262.7us
I20260709 13:08:02.963984 26264 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:02.973603 22871 leader_election.cc:400] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:02.973867 22871 leader_election.cc:403] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:02.974800 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49631.9us
I20260709 13:08:02.981387 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:02.972333 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:02.986279 22752 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:02.988678 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:02.990877 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 74790.7us
I20260709 13:08:02.937039 25850 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:02.997078 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:03.002600 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.003038 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.003432 22824 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:03.004065 22750 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:03.005462 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 659) took 1363 ms. Trace:
I20260709 13:08:03.005806 23451 rpcz_store.cc:276] 0709 13:08:01.641662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.641816 (+   154us) service_pool.cc:224] Handling call
0709 13:08:01.642110 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:01.681642 (+ 39532us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:01.791877 (+110235us) log.cc:844] Serialized 14 byte log entry
0709 13:08:02.109147 (+317270us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:02.109200 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:02.109244 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.937596 (+828352us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b18002a9a60. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419
0709 13:08:02.985627 (+ 48031us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b54000b2cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:02.985681 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:03.005255 (+ 19574us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":288466,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":5517,"raft.queue_time_us":81,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":3189,"spinlock_wait_cycles":115886080,"thread_start_us":426886,"threads_started":2,"wal-append.queue_time_us":415845}
I20260709 13:08:03.007962 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.008137 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.008639 23452 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:08:03.003446 26149 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:03.009949 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:02.977095 26234 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:03.013520 26234 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.013850 26234 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.023315 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.024550 23452 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:03.023315 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:03.026181 22900 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:03.027757 26110 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.030462 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.013047 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.032977 23446 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:03.033443 22903 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:03.035586 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:03.036339 22755 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:03.039650 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 921) took 1168 ms. Trace:
I20260709 13:08:03.040133 22747 rpcz_store.cc:276] 0709 13:08:01.871394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.871724 (+   330us) service_pool.cc:224] Handling call
0709 13:08:03.039421 (+1167697us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54003381d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:03.039588 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2122736384}
I20260709 13:08:03.041709 26249 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.053900 25850 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.057569 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:03.058547 23452 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:02.986061 26247 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:03.063974 26231 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:03.000182 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 171137us
I20260709 13:08:03.078482 22755 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:03.085188 22752 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:03.085455 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 95233us
I20260709 13:08:03.085799 22756 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:08:03.089494 23419 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:02.999733 26272 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:03.090906 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:03.085496 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 105136us
I20260709 13:08:03.090332 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.093869 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.094784 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.097415 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.092326 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:03.100152 23444 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:03.100250 22900 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:08:03.103796 26271 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.104730 22797 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:03.105347 26289 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:08:03.106004 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:03.107141 26261 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:03.107558 26261 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.107904 26261 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.108058 22868 leader_election.cc:400] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:03.109504 26110 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:03.113560 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.117067 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:03.000216 26129 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.111171 22871 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:03.120949 23450 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:08:03.122532 22897 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:08:03.115195 26249 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:03.099469 26272 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:03.126194 23448 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:03.089166 26264 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:08:03.128455 22898 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:03.125294 26249 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.134034 26249 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:03.133971 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:03.135424 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:03.136312 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:03.137501 26271 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:03.138281 26271 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:03.138639 26271 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.096268 26247 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:08:03.133049 26038 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:03.143793 23446 raft_consensus.cc:2368] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:03.138986 26271 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:03.147056 26231 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.147996 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:03.143894 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:03.149595 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 56676.2us
I20260709 13:08:03.128674 26272 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.142931 26209 raft_consensus.cc:2764] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:08:03.156072 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:03.156533 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 73456.3us
I20260709 13:08:03.152554 26014 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.152354 26077 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:03.164819 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.165714 23450 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:03.166296 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:03.165275 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:03.167529 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.167222 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:03.172366 22824 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:03.162750 26077 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:03.180050 26077 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:03.180869 26256 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:03.188913 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:03.189510 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:03.160962 26038 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:03.166391 26014 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.168787 26148 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:03.195429 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.195628 22896 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:03.168383 26251 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:03.167867 26149 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:08:03.207566 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 59672.3us
I20260709 13:08:03.196106 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.197233 25991 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.174487 26264 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:03.206111 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.202553 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.212383 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:03.206435 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.214849 26038 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.215553 25991 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:03.216411 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.160266 26244 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.207469 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.197748 22831 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:03.209698 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:03.221182 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 83837.7us
I20260709 13:08:03.210907 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.208650 26256 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:03.221244 23445 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:03.225684 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.202322 26148 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:03.226542 23449 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:03.228400 22650 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 338) took 10370 ms. Trace:
I20260709 13:08:03.231137 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.236464 22822 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:03.238135 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.238713 23445 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:03.232944 22719 leader_election.cc:400] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:03.239504 22719 leader_election.cc:403] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:08:03.220293 26054 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:03.211303 26014 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.244222 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.226161 26231 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.247723 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1650) took 1456 ms. Trace:
I20260709 13:08:03.248404 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.243212 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:03.232744 26148 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:03.253283 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.255087 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.229487 25991 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:03.256804 22749 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:03.261898 22828 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:03.284233 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:03.252491 26110 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:03.289422 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.292507 22831 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:03.285243 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:03.293470 22824 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:03.291245 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.291567 22822 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:03.252964 22827 rpcz_store.cc:276] 0709 13:08:01.790799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:02.051176 (+260377us) service_pool.cc:224] Handling call
0709 13:08:02.933449 (+882273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:02.933819 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:02.934144 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 13:08:03.176893 (+242749us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:03.177053 (+   160us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:03.177854 (+   801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:03.177941 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:03.246950 (+ 69009us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b180047c380. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4
0709 13:08:03.247443 (+   493us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b54002c9050. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:03.247539 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4394,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":135,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"spinlock_wait_cycles":2075570816,"thread_start_us":236946,"threads_started":1,"wal-append.queue_time_us":296855}
I20260709 13:08:03.299606 22896 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:08:03.285061 22794 connection.cc:256] RPC call timeout handler was delayed by -1.1760866329999544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
I20260709 13:08:03.265483 26129 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.284176 26111 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:03.306437 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.307282 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:03.308570 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.310554 22901 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:03.291241 25850 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:03.292718 26110 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:03.314143 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.301210 26251 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:03.319329 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.302366 26233 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:03.323348 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:03.325286 22897 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:03.327214 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:08:03.328246 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 103506us
I20260709 13:08:03.332470 22752 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:03.306089 25991 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:03.334522 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.336404 22822 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:03.336555 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.338269 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.316172 26219 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:03.297624 26231 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.329205 26237 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.344846 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:03.324791 26233 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.293609 26054 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.345860 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:03.346405 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:03.349017 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:03.339661 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.531s (SENT) [suppressed 32 similar messages]
W20260709 13:08:03.351110 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.347165 26032 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:03.352808 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.352455 26032 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.352973 22903 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:03.355469 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:03.366521 22749 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:08:03.355026 26184 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:08:03.368068 26295 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:08:03.369038 26032 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.368949 26243 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:03.372845 26247 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:03.378626 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.390403 22822 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:08:03.231798 22650 rpcz_store.cc:276] 0709 13:07:52.745667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.773271 (+ 27604us) service_pool.cc:224] Handling call
0709 13:08:03.101391 (+10328120us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:57.926774 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:57.927302 (+   528us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:57.927333 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:07:57.927353 (+    20us) tablet.cc:662] Decoding operations
0709 13:07:57.928618 (+  1265us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:57.928697 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:57.928724 (+    27us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:07:57.931070 (+  2346us) tablet.cc:701] Row locks acquired
0709 13:07:57.931091 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:07:57.931219 (+   128us) write_op.cc:270] Start()
0709 13:07:57.931319 (+   100us) write_op.cc:276] Timestamp: P: 1783602477931176 usec, L: 0
0709 13:07:57.931338 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:07:57.932002 (+   664us) log.cc:844] Serialized 5865 byte log entry
0709 13:07:58.553952 (+621950us) op_driver.cc:464] REPLICATION: finished
0709 13:07:58.883303 (+329351us) write_op.cc:301] APPLY: starting
0709 13:07:58.883375 (+    72us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:58.883672 (+   297us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:58.883776 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:58.968372 (+ 84596us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:58.968669 (+   297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:07:58.968734 (+    65us) write_op.cc:312] APPLY: finished
0709 13:07:58.969067 (+   333us) log.cc:844] Serialized 107 byte log entry
0709 13:07:58.969224 (+   157us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:58.969401 (+   177us) write_op.cc:454] Released schema lock
0709 13:07:58.969455 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:58.969986 (+   531us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:58.970019 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":66470,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":329116,"apply.run_cpu_time_us":4410,"apply.run_wall_time_us":86604,"num_ops":15,"prepare.queue_time_us":203401,"prepare.run_cpu_time_us":53080,"prepare.run_wall_time_us":127649,"raft.queue_time_us":106800,"raft.run_cpu_time_us":33040,"raft.run_wall_time_us":330192,"replication_time_us":622463,"thread_start_us":524188,"threads_started":4,"wal-append.queue_time_us":221985,"wal-append.run_cpu_time_us":11798,"wal-append.run_wall_time_us":1816745}]]}
I20260709 13:08:03.376027 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.391144 22830 raft_consensus.cc:2368] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:03.376659 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.382699 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.392084 22899 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:03.383286 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.383728 25991 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.379274 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.378621 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.387740 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.329632 26256 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.395823 22825 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:03.396260 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.399403 22751 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:03.401772 22824 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
W20260709 13:08:03.401921 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.388940 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:03.402894 26014 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:03.374477 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 47372.1us
I20260709 13:08:03.348242 26219 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:03.403522 22753 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:03.403795 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:03.332930 26272 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.392953 22831 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:03.350651 26233 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:03.349236 26077 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.390851 26244 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:03.390012 26243 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:03.412554 26014 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.421859 22827 raft_consensus.cc:2410] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:08:03.416893 26291 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:08:03.427347 22903 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:03.417631 26157 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:03.416214 26007 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:03.454582 26007 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:03.454986 26007 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:03.455250 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 51541.3us
I20260709 13:08:03.456720 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:03.453641 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.457728 22901 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:03.454032 26077 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:03.360630 26237 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.404153 26219 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.487926 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.489662 26077 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:03.490451 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.490584 22901 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:03.497007 26231 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.452207 26243 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.526870 26007 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.498449 22749 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:03.504029 22905 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:03.516981 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:03.517776 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.518260 26237 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.518929 26077 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:03.535516 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 117915us
I20260709 13:08:03.404912 26110 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.531476 26184 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:03.529096 26007 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.468158 26014 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:03.539707 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 206158us
I20260709 13:08:03.574998 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.535341 26231 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:08:03.581200 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:03.581274 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 236474us
I20260709 13:08:03.560817 26264 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:03.600330 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.601140 22749 raft_consensus.cc:2410] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:03.532204 26148 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.621620 22900 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:03.622781 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:08:03.624629 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 221924us
I20260709 13:08:03.596614 26110 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:03.630542 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:03.618853 26234 consensus_meta.cc:287] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.001s
I20260709 13:08:03.596450 26244 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:03.637143 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:03.638641 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.625192 26054 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.624446 26209 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:03.643641 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68717.6us
W20260709 13:08:03.650625 22653 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 336) took 23410 ms (client timeout 15000 ms). Trace:
W20260709 13:08:03.651026 22653 rpcz_store.cc:269] 0709 13:07:40.240112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.241490 (+  1378us) service_pool.cc:224] Handling call
0709 13:08:03.646409 (+23404919us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":37}
I20260709 13:08:03.650882 22652 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 5 to 9, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 9 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.637097 26312 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:08:03.656821 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.657933 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:03.659034 26247 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:03.658922 26271 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.659842 26247 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:03.660224 26247 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.660559 26247 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:03.661666 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 205069us
W20260709 13:08:03.647104 26302 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:08:03.664819 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:03.665886 26300 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:08:03.667623 26313 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:03.668045 26313 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:03.668396 26313 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:03.653612 26249 consensus_meta.cc:287] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 13:08:03.669701 26249 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.671582 26249 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.639421 26184 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.635573 26250 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:03.672626 26299 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.147s	user 0.001s	sys 0.002s
I20260709 13:08:03.677089 25850 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:03.677453 26077 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.645594 26157 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:03.678159 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:03.640719 26234 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:03.645763 26261 consensus_meta.cc:287] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.006s
I20260709 13:08:03.695726 26261 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.673978 23452 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:03.697122 22828 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:03.697397 22652 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 7 to 12, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 12 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:03.686931 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 49648us
I20260709 13:08:03.642176 26272 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.693768 26313 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.694499 26148 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.695099 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.675037 26014 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.683923 26192 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:03.703353 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:03.704654 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:03.705242 26234 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.705348 26313 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.705741 26261 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.706167 26077 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:03.720036 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:03.706193 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 90017.8us
I20260709 13:08:03.697752 26157 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:03.688750 26271 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.688431 25850 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:03.706748 26178 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:03.707105 26317 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:08:03.698011 26250 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.723816 26077 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:03.724712 26077 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.725574 26250 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.725576 26247 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.728113 26247 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.714622 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:03.731642 22868 leader_election.cc:400] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:03.732429 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:03.733182 26311 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:03.733685 26311 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.733987 26313 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:03.734151 26311 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:03.734403 26313 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:03.734822 26313 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:03.735476 26243 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.682979 26209 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.733592 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:03.721735 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:03.714648 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 75261.4us
I20260709 13:08:03.742408 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:03.742390 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:03.722560 26110 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:03.748236 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:03.748785 22748 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.749222 26209 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:03.750221 26007 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:03.750124 25850 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.759958 22748 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:03.748780 22652 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 6 to 7, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 7 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.762208 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:03.762593 26178 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:03.763062 26271 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:03.763468 26271 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.763794 26271 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:03.766031 25850 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:03.767033 26251 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:03.747347 26234 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:03.770056 22652 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 4 to 8, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 8 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:03.771261 26157 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.757692 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:03.772445 26271 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.773496 26307 raft_consensus.cc:2764] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:08:03.773921 23416 leader_election.cc:400] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10.
I20260709 13:08:03.798238 23416 leader_election.cc:403] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:03.787211 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53734.4us
I20260709 13:08:03.799206 22652 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 1 to 5, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 5 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.759825 26110 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:03.802291 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:03.806222 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.809499 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.807634 22652 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 4 to 7, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 7 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:03.812362 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:03.813939 22652 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.815910 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:03.804240 26243 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.808902 26244 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:03.821169 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.821347 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:03.837765 22747 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:03.821481 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:03.820843 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:03.840241 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.841373 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:03.841924 22831 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:03.818037 26324 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:03.825582 26007 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:03.825827 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:03.808344 26261 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:03.826738 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.825791 22828 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:03.848549 23452 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 4: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:03.811811 26184 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:08:03.831034 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57352.8us
I20260709 13:08:03.850710 26243 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:03.852303 26243 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.845723 26324 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:03.853992 26324 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.854377 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.838181 26110 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.855552 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.832549 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:03.833459 26247 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.853457 22831 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:08:03.834084 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:03.857035 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.857651 22825 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:03.833775 26311 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.856956 26129 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:03.858585 26322 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:03.859881 26007 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:03.860553 26007 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.861717 23452 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:08:03.863144 26271 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:03.846609 22756 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:03.810232 26251 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10.
I20260709 13:08:03.826124 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:03.846042 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:03.817932 26178 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:03.830418 22652 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 4 to 8, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 8 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:03.849901 26250 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:03.869228 26234 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:03.869801 26234 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:03.870685 26234 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:03.872620 22754 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:03.859925 26311 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.833775 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:03.873749 22830 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:03.838179 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:03.867918 22822 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:03.852849 26184 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:03.868381 26244 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:03.868471 22825 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
W20260709 13:08:03.888773 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50665.1us
W20260709 13:08:03.868739 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56948.8us
I20260709 13:08:03.869514 26250 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.874742 22652 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 3 to 7, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 7 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:03.889492 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:03.876271 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.852674 26243 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:03.892722 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:03.890743 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 58255.7us
I20260709 13:08:03.895896 22830 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:03.896649 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.899739 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:03.891198 26244 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:03.900617 23446 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:03.901790 26250 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.887548 26184 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.887858 25850 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:08:03.894550 26328 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:03.906987 26328 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.908233 23448 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:08:03.909449 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.910627 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.911262 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.909894 26157 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.897388 26261 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:03.912179 26077 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:03.912792 26007 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.916162 22822 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:03.922428 23449 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:03.925100 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.925709 22822 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
W20260709 13:08:03.926400 26233 consensus_meta.cc:287] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.507s	user 0.004s	sys 0.004s
I20260709 13:08:03.926905 26233 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.928187 22750 raft_consensus.cc:2368] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:03.929872 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:03.933516 26077 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:03.934020 26077 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.934469 26077 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:03.937085 25850 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:03.909075 26328 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:03.937501 25850 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.937856 25850 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:03.938887 26324 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.940938 26324 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:03.941502 22797 leader_election.cc:400] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:03.941891 22797 leader_election.cc:403] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:08:03.943284 26261 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:03.943987 26244 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.944940 25850 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.946074 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.947062 22825 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:03.898715 26054 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:03.949075 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:03.949159 26054 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.949954 23453 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:03.898440 26271 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:03.954596 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:03.956406 23449 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:03.956709 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.957634 22822 raft_consensus.cc:2368] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8.
W20260709 13:08:03.961229 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 104222us
I20260709 13:08:03.963428 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.965111 22830 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:03.965301 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:03.966219 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.966440 22831 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:03.968533 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:03.969864 26301 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:03.971019 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:03.971740 26301 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:03.972106 26301 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:03.972494 26301 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:03.951208 26054 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:03.977712 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:03.979522 26243 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:03.972321 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:03.982319 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:03.983160 23453 raft_consensus.cc:2410] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:08:03.984329 26313 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:03.989507 26192 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:03.990038 22824 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:03.990078 26192 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:04.002574 22830 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
W20260709 13:08:04.003860 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.014361 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.016739 22830 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:04.018103 22831 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:04.017767 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:04.020932 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.021374 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:04.021987 22754 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:04.023316 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.023955 23449 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:08:04.023417 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:03.995235 26311 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:04.026656 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:04.029397 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:04.030143 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.030395 26308 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:04.033079 22830 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:04.037534 26233 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:04.042327 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:04.046079 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.044927 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:04.045078 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:04.051231 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:04.051578 23449 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:04.051916 22828 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:04.052685 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.053145 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.053256 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.053761 22822 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:04.054147 23450 raft_consensus.cc:2368] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:04.054260 26308 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:04.054979 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:04.055433 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.055182 26308 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.056133 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:04.056875 26251 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:04.057410 26250 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:04.057868 23452 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:04.058017 26256 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:04.059477 26209 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:04.071028 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.071388 26209 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:04.071727 22903 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:04.024053 26264 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:08:04.072386 23452 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:04.060686 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.074733 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.075631 22898 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:04.054999 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.010320 26157 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:08:04.063908 26301 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.078003 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.078887 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.079000 26301 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:04.064611 26244 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:04.058808 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:04.055212 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:04.079761 26184 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.080480 23445 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:08:04.068986 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.082528 22897 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:04.047317 23453 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:04.083758 22901 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:04.024587 26110 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:04.053256 22899 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:04.085767 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:04.085780 22905 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 5: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:04.058787 26250 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:04.086416 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.087248 26271 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:04.052795 26178 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:04.070777 26243 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:04.088121 26178 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:03.991197 26192 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.089027 26178 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.072636 26264 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:04.068601 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:04.053341 26304 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:04.071915 26209 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:04.060508 26129 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:04.060389 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.098405 26313 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:04.098555 22902 raft_consensus.cc:2368] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:04.098960 23453 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:04.059841 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:04.100199 23451 raft_consensus.cc:2410] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:04.060389 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:04.079458 23452 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:04.085899 22825 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:08:04.072667 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.071472 26311 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:04.087644 26271 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:04.102267 26311 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:04.102384 26271 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:04.103133 26311 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.103721 26261 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:04.061533 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.057214 26251 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:04.097746 26304 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:04.105509 22822 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:04.109558 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.108196 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:04.115248 22831 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 9: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:04.077304 22900 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:04.116588 22754 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:04.117431 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.118445 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.118523 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.117776 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.119923 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.120519 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.121752 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.122138 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.124017 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.124995 22905 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:08:04.125442 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.125710 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.128407 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.129675 22831 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:04.109403 25850 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:04.113257 26251 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:04.163163 26054 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.104388 26264 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:04.133800 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.165807 22899 raft_consensus.cc:2368] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7.
I20260709 13:08:04.134042 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.166091 26264 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.167376 26264 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:04.133442 23419 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:04.170367 23419 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:04.171756 26322 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:04.172137 26322 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.172480 26322 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:04.136101 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.139077 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.174567 22822 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:04.140005 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.135380 26321 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:04.141085 23448 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:04.118878 26077 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.118577 26129 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:04.141873 26317 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:04.142283 26148 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:04.142443 26304 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.145696 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:04.156801 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.157282 26243 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:04.180259 22898 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:04.181710 22831 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:08:04.183960 23444 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:04.206820 22871 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:04.187865 23452 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:04.189190 22825 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:04.191859 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:04.193089 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:04.194200 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:04.194967 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.197574 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:04.202162 22822 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:04.189109 26250 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:08:04.210013 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 64379.8us
W20260709 13:08:04.232327 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 175351us
I20260709 13:08:04.211395 22748 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:04.233052 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:04.232355 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:04.236956 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.237883 22898 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:04.193738 26301 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.210445 22899 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:04.239225 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:04.245214 22755 raft_consensus.cc:2368] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
W20260709 13:08:04.245327 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 88594.9us
I20260709 13:08:04.211885 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.240780 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.247380 22831 raft_consensus.cc:2368] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:04.198659 26184 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:04.241597 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.250145 22825 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:04.243798 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.246418 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.252246 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.252480 22830 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:04.253573 22822 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:04.236481 26307 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:04.236315 26328 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:04.256040 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.243297 26178 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:04.201192 26271 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.238732 26247 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:04.238540 26231 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:08:04.262563 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50763.1us
I20260709 13:08:04.268970 22825 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:04.250100 26054 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:04.269209 26328 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:04.239892 26233 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:08:04.282174 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.289125 22903 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:04.284471 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:04.290722 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48133us
I20260709 13:08:04.290823 22898 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:04.290984 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.291606 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.264039 26317 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:04.265626 26313 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:04.299867 22822 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:04.300989 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.266758 25991 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:04.302465 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.250314 26301 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:04.270071 26307 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:04.306883 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.307370 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.309792 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.273766 26178 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:04.307871 22898 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:04.312495 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.299312 26261 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:04.318898 26348 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:04.320009 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.320403 22900 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:04.298880 26233 consensus_queue.cc:1243] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:08:04.299060 26219 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:04.301899 26250 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:04.353041 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:04.354997 22871 leader_election.cc:400] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
W20260709 13:08:04.353812 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 62267.3us
I20260709 13:08:04.315610 26184 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:04.433802 26336 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:08:04.425014 26347 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:08:04.435492 26307 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.435202 26250 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.436853 26209 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.309737 26244 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:04.437794 26244 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:04.438215 26244 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:04.401510 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 96463.3us
I20260709 13:08:04.439239 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:04.439535 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 134462us
I20260709 13:08:04.312526 26309 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:04.309791 26247 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:04.428390 22898 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:04.428519 22871 leader_election.cc:403] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:08:04.429014 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.452776 23416 leader_election.cc:400] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7.
I20260709 13:08:04.453596 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:04.403091 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.451375 22749 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:08:04.456143 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 101308us
W20260709 13:08:04.456737 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 153869us
W20260709 13:08:04.457386 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.457600 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.463348 22719 leader_election.cc:400] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:04.351730 26313 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 13:08:04.302100 26129 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:04.471590 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.351521 26338 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 13:08:04.480139 22904 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
W20260709 13:08:04.481552 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 169518us
I20260709 13:08:04.460685 26054 raft_consensus.cc:2764] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:08:04.460634 26077 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:04.488288 22831 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:08:04.484783 26209 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:04.489780 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.490163 22825 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:08:04.491555 26342 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.129s	user 0.006s	sys 0.003s
I20260709 13:08:04.491899 26356 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:08:04.492449 26340 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:08:04.493062 26344 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:04.526772 22831 pending_rounds.cc:85] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 4
I20260709 13:08:04.527866 22831 pending_rounds.cc:107] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.5
I20260709 13:08:04.530045 22831 raft_consensus.cc:2917] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Aborting config change with OpId 7.5: Aborted: Op aborted by new leader
I20260709 13:08:04.540354 22898 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:04.477634 26261 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:04.473379 26247 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:04.545864 22896 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
W20260709 13:08:04.547556 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 94022.8us
I20260709 13:08:04.546617 25991 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:08:04.549054 26354 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:04.552233 26353 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:04.553881 26077 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.554337 26077 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:04.451776 26328 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:04.557917 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 90269us
I20260709 13:08:04.561551 26311 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:04.561619 26271 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:04.543159 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 69328.8us
I20260709 13:08:04.456073 26309 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:04.558121 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.566937 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:04.571170 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 113640us
I20260709 13:08:04.571774 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.571756 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:04.584666 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:04.587399 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.588510 22868 leader_election.cc:400] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8.
I20260709 13:08:04.589888 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:04.591013 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:04.589100 22825 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:08:04.588948 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.592669 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:04.600052 22754 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:08:04.600291 22830 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:04.600517 22754 pending_rounds.cc:85] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 6
I20260709 13:08:04.601073 22754 pending_rounds.cc:107] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:08:04.601380 22754 raft_consensus.cc:2889] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:04.606756 22828 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:04.588251 26357 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:04.485965 26178 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.614187 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:04.585460 26251 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:04.591997 26261 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:08:04.592438 26350 raft_consensus.cc:2764] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:08:04.643169 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.619438 22830 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
W20260709 13:08:04.645381 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55186.6us
I20260709 13:08:04.837937 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:04.829501 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 271470us
I20260709 13:08:04.656603 23447 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:04.829953 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:04.866816 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:04.691941 22829 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:08:05.000947 22794 leader_election.cc:400] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:04.833066 22826 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:05.001492 22794 leader_election.cc:403] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:08:05.002754 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 136038us
I20260709 13:08:05.004734 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:04.592731 26233 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:05.005398 22826 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
W20260709 13:08:05.005921 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 362831us
I20260709 13:08:05.014969 23444 raft_consensus.cc:1217] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[5.7-8.9]   Dedup: 5.7->[6.8-8.9]
I20260709 13:08:05.024318 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:05.015897 23444 pending_rounds.cc:85] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 7
I20260709 13:08:05.030341 23444 pending_rounds.cc:107] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:08:05.030637 23444 raft_consensus.cc:2889] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:04.653704 25850 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:05.008759 26359 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:05.006392 26110 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:05.065598 22826 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:04.603569 26301 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:04.656459 26338 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:05.018613 26233 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:05.074723 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.043953 26234 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:05.067516 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:05.017355 26304 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:08:04.592944 26355 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.452s
I20260709 13:08:05.105124 23449 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:04.671775 26309 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:05.059656 26351 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.063444 26157 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:05.079869 26315 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:05.121927 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:05.137269 26346 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.524s	user 0.003s	sys 0.000s
I20260709 13:08:05.100787 26243 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
W20260709 13:08:05.139653 26362 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:08:05.134739 26350 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:08:05.145288 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 812) took 1021 ms. Trace:
I20260709 13:08:05.145704 22896 rpcz_store.cc:276] 0709 13:08:04.124340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.124473 (+   133us) service_pool.cc:224] Handling call
0709 13:08:05.145124 (+1020651us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54001b6f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:05.145233 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854048000}
I20260709 13:08:05.148413 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:05.153559 25991 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:05.154088 25991 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:05.154311 26361 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:05.154891 26361 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:05.155411 26361 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:05.155007 25991 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.059881 26356 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:08:05.159855 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:05.163612 23449 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:05.107407 26233 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:05.167243 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:05.110484 26301 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:05.110430 26338 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:05.174996 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:05.177608 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 310052us
I20260709 13:08:05.181015 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:05.168907 26234 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:05.190651 22756 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:05.193054 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:05.167635 26157 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:05.181173 26350 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:05.174242 26271 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:05.249012 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:05.191085 26184 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.187456 26358 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:05.185258 26301 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:05.249940 23451 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:05.249980 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:05.267303 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:05.254482 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:05.267982 22829 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11.
I20260709 13:08:05.268643 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:05.255584 22898 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:08:05.257138 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 249959us
I20260709 13:08:05.260553 22899 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:05.261219 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:05.271464 22830 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:05.215468 26272 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:05.277987 22826 raft_consensus.cc:2368] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7.
I20260709 13:08:05.257598 26315 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:05.165405 26351 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:05.279834 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 104921us
I20260709 13:08:05.258460 26304 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.258409 26256 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:05.261396 26264 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:05.287251 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:05.289577 22824 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:05.290254 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:05.290779 22826 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:05.266606 26243 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.292891 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:05.294593 22827 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:05.277530 26358 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:05.299299 26250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.003s	sys 0.015s
I20260709 13:08:05.282086 26272 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:05.267010 26184 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.294186 26355 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:05.358486 23416 leader_election.cc:400] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8.
I20260709 13:08:05.373204 22827 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:08:05.330191 26250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.004s	sys 0.015s
I20260709 13:08:05.381408 22896 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:08:05.357566 26348 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.007s	sys 0.016s
I20260709 13:08:05.311086 26353 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:05.293080 26304 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.295377 26256 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.387029 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:05.387158 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:05.393421 22794 leader_election.cc:400] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:05.394485 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:05.395907 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:05.397338 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:05.311378 26264 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.401002 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:05.386775 26348 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.007s	sys 0.016s
I20260709 13:08:05.334465 26243 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.409451 22827 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:05.386431 26368 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.398423 26250 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:08:05.419209 26367 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:08:05.419471 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:05.419876 26372 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.386262 26358 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.421092 26272 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.398980 26251 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:05.425416 26371 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:08:05.426524 26256 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:05.374112 26355 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.420497 26372 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.311873 26157 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.404955 26315 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:05.439754 22826 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:05.357416 26359 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:05.386430 26301 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:05.445834 26341 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.021s	user 0.013s	sys 0.000s
I20260709 13:08:05.446349 26380 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:08:05.428041 26244 consensus_meta.cc:287] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.960s	user 0.006s	sys 0.004s
W20260709 13:08:05.426561 26369 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:08:05.398862 26353 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.294894 26338 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.417891 26365 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:08:05.428148 26368 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.427757 26321 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:08:05.553490 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:05.547890 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 132224us
I20260709 13:08:05.507082 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1113) took 1429 ms. Trace:
I20260709 13:08:05.459637 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2180) took 1283 ms. Trace:
I20260709 13:08:05.558609 22752 rpcz_store.cc:276] 0709 13:08:04.175780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.176006 (+   226us) service_pool.cc:224] Handling call
0709 13:08:04.198043 (+ 22037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:04.602877 (+404834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:04.602909 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:04.602948 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:04.602976 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.459370 (+856394us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54003cfad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:05.459425 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:05.459570 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910496640}
W20260709 13:08:05.557418 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 160185us
I20260709 13:08:05.563828 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:05.466287 22899 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
W20260709 13:08:05.539026 26337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.046s	sys 0.045s
I20260709 13:08:05.531601 26348 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:05.461714 26322 consensus_meta.cc:287] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.289s	user 0.008s	sys 0.005s
I20260709 13:08:05.567330 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:05.458093 26304 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:05.460539 26377 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:08:05.556614 26358 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:05.575589 22827 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:08:05.558239 23450 rpcz_store.cc:276] 0709 13:08:04.077409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.080679 (+  3270us) service_pool.cc:224] Handling call
0709 13:08:04.080970 (+   291us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:04.081370 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:04.081677 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 13:08:04.188937 (+107260us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:04.197628 (+  8691us) log.cc:844] Serialized 52 byte log entry
0709 13:08:04.197808 (+   180us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:04.203761 (+  5953us) log.cc:844] Serialized 14 byte log entry
0709 13:08:04.304434 (+100673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:04.304500 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:05.445197 (+1140697us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:05.506877 (+ 61680us) raft_consensus.cc:1685] finished
0709 13:08:05.506900 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:05.507028 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7268,"prepare.queue_time_us":1763,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":24440,"raft.queue_time_us":34642,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":9245,"spinlock_wait_cycles":9773312,"thread_start_us":153113,"threads_started":2,"wal-append.queue_time_us":196556}
W20260709 13:08:05.570055 26337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.048s	sys 0.047s
I20260709 13:08:05.461866 26244 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.579555 26353 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:05.578255 26322 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.581135 26372 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:05.561681 26351 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:05.568614 26380 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:05.561259 26250 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:05.600267 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:05.602305 22827 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:05.612880 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:05.613415 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:05.616055 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 61741.7us
I20260709 13:08:05.618257 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:05.600291 26007 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:05.602411 26178 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.606899 26184 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:05.510625 26359 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.575892 26348 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.593878 26358 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.612597 26250 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:05.617408 26243 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:05.668998 26311 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.107s	user 0.007s	sys 0.007s
I20260709 13:08:05.671705 23448 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:05.669976 26368 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:05.615489 26312 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:08:05.680827 26394 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 13:08:05.681469 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 45640.5us
W20260709 13:08:05.673236 26386 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.119s	user 0.001s	sys 0.001s
I20260709 13:08:05.617240 26264 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:05.681541 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1052) took 1381 ms. Trace:
I20260709 13:08:05.684453 23448 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:05.678453 26324 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:05.681541 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 685) took 1084 ms. Trace:
I20260709 13:08:05.687500 23450 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:05.620584 26355 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:05.690618 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:05.691258 23451 rpcz_store.cc:276] 0709 13:08:04.300429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.300562 (+   133us) service_pool.cc:224] Handling call
0709 13:08:04.309966 (+  9404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:05.267552 (+957586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.680075 (+412523us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b54006329d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:05.681478 (+  1403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156855680}
I20260709 13:08:05.696374 23416 leader_election.cc:400] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11.
I20260709 13:08:05.692493 23448 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.679659 26178 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.706673 23416 leader_election.cc:400] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7.
I20260709 13:08:05.694015 23445 rpcz_store.cc:276] 0709 13:08:04.596730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.596854 (+   124us) service_pool.cc:224] Handling call
0709 13:08:04.830640 (+233786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:05.010603 (+179963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:05.010650 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:05.027064 (+ 16414us) log.cc:844] Serialized 26 byte log entry
0709 13:08:05.164998 (+137934us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:05.165044 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.165119 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:05.681338 (+516219us) raft_consensus.cc:1685] finished
0709 13:08:05.681366 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:05.681487 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7399,"raft.queue_time_us":26853,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":96,"spinlock_wait_cycles":166016,"thread_start_us":132829,"threads_started":1,"wal-append.queue_time_us":299604}
I20260709 13:08:05.716559 23448 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:05.713598 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:05.746268 23416 leader_election.cc:403] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:05.714772 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:05.700464 26351 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:05.738688 22897 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:05.753188 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:05.755695 22827 raft_consensus.cc:2368] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11.
I20260709 13:08:05.756155 23451 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:05.698519 26264 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:05.691511 26312 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:05.742870 22829 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:05.763597 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:05.714203 26355 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:05.768422 26234 consensus_meta.cc:287] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.000s
I20260709 13:08:05.768253 26355 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.768798 26234 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:05.777766 26360 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7.
I20260709 13:08:05.772572 26264 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:05.779529 26233 consensus_meta.cc:287] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.594s	user 0.008s	sys 0.003s
I20260709 13:08:05.779968 26233 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:05.786815 26382 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:08:05.787312 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1557) took 1499 ms. Trace:
I20260709 13:08:05.787775 22900 rpcz_store.cc:276] 0709 13:08:04.288240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.288365 (+   125us) service_pool.cc:224] Handling call
0709 13:08:04.307893 (+ 19528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:04.350076 (+ 42183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.787079 (+1437003us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b54001b7454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:05.787231 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13188352}
I20260709 13:08:05.754644 26250 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.789263 26348 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:05.830720 26234 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:05.827534 26404 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:08:05.848996 26301 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.874549 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:05.829558 26344 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:08:05.835216 22903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1551) took 1764 ms (client timeout 1509 ms). Trace:
I20260709 13:08:05.850212 23448 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:08:05.842295 22831 raft_consensus.cc:2889] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:05.856280 22828 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:05.802068 22822 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:08:05.713040 26378 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:05.799180 23445 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:05.849118 26350 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:05.875427 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 168900us
I20260709 13:08:05.879236 22829 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:05.879954 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57174.1us
I20260709 13:08:05.790019 26244 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:05.880036 22903 rpcz_store.cc:269] 0709 13:08:04.070711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.072473 (+  1762us) service_pool.cc:224] Handling call
0709 13:08:05.835031 (+1762558us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400275454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:05.835126 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429919104}
I20260709 13:08:05.883487 26350 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.884629 26410 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:05.885233 26350 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:05.869287 26272 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.869055 26355 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:05.886262 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:05.792464 25850 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.873008 26250 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:05.875183 26309 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:05.877640 26348 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.875005 26307 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:05.916172 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 49798.4us
W20260709 13:08:05.887652 26361 consensus_meta.cc:287] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.003s
I20260709 13:08:05.880813 26358 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:05.921669 23446 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:05.879019 26301 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.922372 26361 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.923228 26264 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:05.913785 26272 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.924322 26301 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:05.915408 26380 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:05.925117 26380 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:05.885275 26410 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.926060 26380 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.919237 26348 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.929790 26348 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:05.931015 26361 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:05.879741 26344 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:05.922554 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:47094 (request call id 16) took 7136 ms. Trace:
I20260709 13:08:05.876581 26234 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.916059 26233 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:05.932266 22831 rpcz_store.cc:276] 0709 13:07:58.783989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.784104 (+   115us) service_pool.cc:224] Handling call
0709 13:07:59.354084 (+569980us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54001011a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d78520 00007fcb74e01b42 00007fcb74e00800 00007fcb79dc1d29 00007fcb733401d4 00007fcb73340162 00007fcb73340091 00007fcb7334000c 00007fcb7333f2c2 00007fcb7313648c
0709 13:08:00.727967 (+1373883us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b54001011a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d784b1 00007fcb74e00863 00007fcb79dc1d29 00007fcb733401d4 00007fcb73340162 00007fcb73340091 00007fcb7334000c 00007fcb7333f2c2 00007fcb7313648c 00007fcb731358c6
0709 13:08:00.851423 (+123456us) log.cc:844] Serialized 499 byte log entry
0709 13:08:02.111364 (+1259941us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b54001011a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d70dbe 00007fcb74d7b930 00007fcb74d7b8d8 00007fcb74d7b858 00007fcb74d7b813 00007fcb74d7aa9c 00007fcb7ac66626 00007fcb7ac60dd8 00007fcb74db77cb 00007fcb74dba2c0
0709 13:08:05.917417 (+3806053us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10525,"raft.run_cpu_time_us":1578,"raft.run_wall_time_us":1577,"spinlock_wait_cycles":1308421248,"thread_start_us":131822,"threads_started":2,"wal-append.queue_time_us":205601,"wal-append.run_cpu_time_us":16328,"wal-append.run_wall_time_us":2043260}
I20260709 13:08:05.942525 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2380) took 1455 ms. Trace:
W20260709 13:08:05.942669 23452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1051) took 1642 ms (client timeout 1885 ms). Trace:
I20260709 13:08:05.942842 22831 rpcz_store.cc:276] 0709 13:08:04.487390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.487553 (+   163us) service_pool.cc:224] Handling call
0709 13:08:04.487897 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:05.842192 (+1354295us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b4c0044d680. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb730c7a20 00007fcb730d8a29 00007fcb7309e644 00007fcb730c5f2d 00007fcb730c5b93 00007fcb74df618b 00007fcb74df6cb5 00007fcb74e05fde 00007fcb74e057bf 00007fcb74e1781f 00007fcb74e177b8
0709 13:08:05.942307 (+100115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.942476 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6072704}
W20260709 13:08:05.943188 23452 rpcz_store.cc:269] 0709 13:08:04.300046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.300153 (+   107us) service_pool.cc:224] Handling call
0709 13:08:05.942492 (+1642339us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540056ac54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:05.942616 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2465482624}
I20260709 13:08:05.944079 26054 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:05.940367 23449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1048) took 1842 ms (client timeout 1896 ms). Trace:
W20260709 13:08:05.945283 23449 rpcz_store.cc:269] 0709 13:08:04.097737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.109348 (+ 11611us) service_pool.cc:224] Handling call
0709 13:08:05.940172 (+1830824us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b5400672d54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:05.940287 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2273356672}
I20260709 13:08:05.919627 26307 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.917799 26309 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:05.949067 26370 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.711s	user 0.002s	sys 0.002s
I20260709 13:08:05.960848 26309 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:05.969719 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 55398.9us
I20260709 13:08:05.921988 26359 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.918190 26312 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:05.970752 26359 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:05.975961 26234 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:05.936151 25850 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.095973 26338 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.096585 26338 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.098212 26344 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.155489 26361 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:06.147957 26324 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:06.163153 25850 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:06.098661 26338 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:06.167454 26244 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:06.192246 26344 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:06.204358 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 957) took 1846 ms. Trace:
I20260709 13:08:06.195483 26359 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:06.204905 22755 rpcz_store.cc:276] 0709 13:08:04.357754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.357868 (+   114us) service_pool.cc:224] Handling call
0709 13:08:04.429270 (+ 71402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:04.440436 (+ 11166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:04.440469 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:04.440521 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:04.440564 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.204062 (+1763498us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b5400045454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:06.204127 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.204287 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25198976}
I20260709 13:08:06.195679 26361 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:06.221047 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 336484us
I20260709 13:08:06.197790 26324 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.208992 26403 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:06.202275 26244 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.223948 26307 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:06.232345 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:06.237883 26360 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:06.238692 26360 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:06.239009 26360 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:06.239835 26360 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.240974 26360 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:06.242576 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:06.246050 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:06.244638 26351 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:06.228206 26403 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.250011 26307 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:06.250761 26361 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.250550 26307 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.251451 26403 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.251957 26403 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.252375 26403 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.252923 26307 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.254536 26403 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:06.257969 26403 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.258419 26403 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.259068 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 684) took 1662 ms. Trace:
I20260709 13:08:06.259433 23444 rpcz_store.cc:276] 0709 13:08:04.596412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.596721 (+   309us) service_pool.cc:224] Handling call
0709 13:08:04.830585 (+233864us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:08:05.064362 (+233777us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:05.064404 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:05.972193 (+907789us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b2c0013bd48. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:08:05.972806 (+   613us) log.cc:844] Serialized 52 byte log entry
0709 13:08:06.220146 (+247340us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:06.220183 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.220227 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:06.258769 (+ 38542us) raft_consensus.cc:1685] finished
0709 13:08:06.258789 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.259013 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5397,"raft.queue_time_us":46912,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":10530,"spinlock_wait_cycles":5072768,"thread_start_us":246250,"threads_started":1,"wal-append.queue_time_us":283264}
I20260709 13:08:06.260551 26403 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:06.260888 26352 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:06.230069 26244 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.244897 26354 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:06.275035 22756 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:08:06.278167 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.278395 26272 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:06.278882 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.279238 26410 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.268990 26351 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:06.276413 26271 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:06.281927 26271 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:06.282094 26351 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.276526 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.283845 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.284804 22756 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:06.283721 22752 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:06.291558 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.292150 22756 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:06.293432 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.295681 22903 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:06.295658 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.298071 22901 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:06.297950 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.298841 22752 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:06.300197 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.302575 22901 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:06.302479 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:06.305390 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 690) took 1017 ms. Trace:
I20260709 13:08:06.306238 22900 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:06.306919 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.307471 22904 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 4.
I20260709 13:08:06.308171 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.311012 26324 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.299628 26338 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.312388 26338 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.314385 26338 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:06.315266 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.315809 22756 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:08:06.315929 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.316824 22752 raft_consensus.cc:2410] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:08:06.309007 23448 rpcz_store.cc:276] 0709 13:08:05.287910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.886402 (+598492us) service_pool.cc:224] Handling call
0709 13:08:05.886722 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:06.223092 (+336370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:06.223120 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:06.223476 (+   356us) log.cc:844] Serialized 26 byte log entry
0709 13:08:06.290970 (+ 67494us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:06.291011 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.291077 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:06.305151 (+ 14074us) raft_consensus.cc:1685] finished
0709 13:08:06.305188 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.305294 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":345694,"raft.queue_time_us":336937,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":2291,"thread_start_us":43722,"threads_started":1,"wal-append.queue_time_us":71718}
I20260709 13:08:06.322819 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.327713 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.328066 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.600498 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:06.597188 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:06.601648 26356 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.344s	user 0.037s	sys 0.069s
I20260709 13:08:06.355441 26324 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.363014 25850 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.259s
I20260709 13:08:06.408401 26192 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.620850 26192 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.607478 26307 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:06.613072 26356 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.360s	user 0.038s	sys 0.070s
I20260709 13:08:06.623117 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1690) took 1249 ms. Trace:
I20260709 13:08:06.623519 26307 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.625456 26307 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.623760 22826 rpcz_store.cc:276] 0709 13:08:05.373774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.362412 (+988638us) service_pool.cc:224] Handling call
0709 13:08:06.362785 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:06.363220 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:06.363270 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:06.363766 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:08:06.486620 (+122854us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:06.486678 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.486807 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:06.622844 (+136037us) raft_consensus.cc:1685] finished
0709 13:08:06.622871 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.623047 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":9807,"thread_start_us":99673,"threads_started":1,"wal-append.queue_time_us":233775}
I20260709 13:08:06.626422 26324 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.623147 26192 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.633873 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:06.639297 26271 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:06.645929 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 336945us
I20260709 13:08:06.613734 26313 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.647892 26313 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.658380 26313 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.650785 26324 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:06.652313 26271 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:06.627259 26307 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:06.679260 26424 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:08:06.664763 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.667249 22904 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:06.667856 26054 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.677943 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:06.682118 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:06.679572 26307 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.684319 26054 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.691020 26430 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:06.691107 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:06.691622 26430 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:06.692005 26430 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:06.689939 26054 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.699787 26424 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:06.700160 26424 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:06.700474 26424 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:06.706715 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 636) took 1313 ms. Trace:
I20260709 13:08:06.707109 22825 rpcz_store.cc:276] 0709 13:08:05.392949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.705914 (+1312965us) service_pool.cc:224] Handling call
0709 13:08:06.706201 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:06.706387 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:06.706413 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:06.706454 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:06.706489 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.706543 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.706661 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.708137 26328 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:06.708655 26328 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.710902 26307 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.684818 26313 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:06.739836 26430 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:06.764096 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.766681 26425 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:06.770975 26425 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:06.789398 23450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1059) took 1328 ms (client timeout 1709 ms). Trace:
W20260709 13:08:06.789876 23450 rpcz_store.cc:269] 0709 13:08:05.461424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.607468 (+146044us) service_pool.cc:224] Handling call
0709 13:08:06.789164 (+1181696us) spinlock_profiling.cc:243] Waited 68.6 ms on lock 0x7b5400567cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:06.789301 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157732096}
I20260709 13:08:06.782882 26357 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:06.725224 26313 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:06.821640 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:06.869640 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 178610us
I20260709 13:08:06.870705 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:06.872484 26328 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:06.872579 26425 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:06.874650 26178 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.875149 26178 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.838353 26313 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:06.878733 26244 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:06.879156 26244 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:06.879566 26244 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:06.881929 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.883469 22871 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:06.814025 26357 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:06.875448 26178 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.885618 26357 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.886641 26357 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:06.886850 26178 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:06.887403 26178 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:06.888558 22825 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:08:06.887733 26178 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.817278 26424 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.889657 26178 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:06.891144 26424 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:06.868954 26312 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:06.893148 26365 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:06.876976 26430 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:06.903326 22897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2090) took 2790 ms (client timeout 1885 ms). Trace:
I20260709 13:08:06.902113 26365 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:06.903919 22897 rpcz_store.cc:269] 0709 13:08:04.112103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.112226 (+   123us) service_pool.cc:224] Handling call
0709 13:08:06.902229 (+2790003us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54003bf6d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:06.902323 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3598715136}
I20260709 13:08:06.905452 26404 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.905548 26315 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.906008 26404 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.906121 26315 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:06.907673 26398 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.235s	user 0.001s	sys 0.010s
I20260709 13:08:06.908093 26404 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.908258 26315 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.911409 22904 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:06.911528 22905 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:08:06.912536 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.912987 22898 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:08:06.913367 22898 pending_rounds.cc:85] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 5
I20260709 13:08:06.913245 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.913666 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.913681 22898 pending_rounds.cc:107] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:08:06.914887 22898 raft_consensus.cc:2889] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:06.916157 22897 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:08:06.916751 26374 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:08:06.920989 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.922037 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.922587 22754 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:08:06.923067 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.923804 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:06.924124 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.924621 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:06.925462 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.926031 22752 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:06.926256 26271 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:08:06.927008 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 847c8a2cc0bb4233a1d55ca04643418d: Aborted: Op aborted by new leader
I20260709 13:08:06.927067 26271 consensus_queue.cc:1243] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:08:06.927736 26271 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:06.928377 26271 consensus_queue.cc:1243] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:08:06.929502 26375 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:06.929768 26054 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:06.929914 26375 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:06.930297 26375 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:06.930261 26054 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.932369 26054 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:06.925132 22747 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:06.934118 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:06.921255 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:06.935986 22751 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:06.936589 22824 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:06.933348 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:06.937651 22901 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:08:06.937110 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1691) took 1549 ms. Trace:
I20260709 13:08:06.925113 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:06.938517 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 632) took 1608 ms. Trace:
I20260709 13:08:06.935398 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:06.940097 23447 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:06.940515 22748 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:06.938207 22824 rpcz_store.cc:276] 0709 13:08:05.387348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.925720 (+1538372us) service_pool.cc:224] Handling call
0709 13:08:06.925982 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:06.936919 (+ 10937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.937067 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:06.942476 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.943140 22748 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:06.944769 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.945382 22902 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:06.894937 26077 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:06.946429 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.947364 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.947762 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.947927 22897 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:06.947240 22902 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:08:06.949021 23416 leader_election.cc:400] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11.
I20260709 13:08:06.949357 23416 leader_election.cc:403] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:08:06.949401 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:06.950007 22901 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:06.951207 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:06.951095 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.951824 22900 raft_consensus.cc:2376] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8. Re-sending same reply.
I20260709 13:08:06.952126 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.952365 22826 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:06.953109 22748 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:08:06.953380 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.954026 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.955843 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:06.956787 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.957144 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.957255 22898 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:06.959132 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.959556 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.959857 22829 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:08:06.960124 26250 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11.
I20260709 13:08:06.955019 26247 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.960763 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.961447 26247 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.959055 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.957813 22902 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:06.955531 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.960773 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.959260 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.964902 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.965561 22822 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:06.966109 22902 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:06.966565 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.939987 26321 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:06.939028 22823 rpcz_store.cc:276] 0709 13:08:05.329903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.330282 (+   379us) service_pool.cc:224] Handling call
0709 13:08:05.378868 (+ 48586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:05.390258 (+ 11390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:05.390281 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:05.412177 (+ 21896us) log.cc:844] Serialized 26 byte log entry
0709 13:08:05.582759 (+170582us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:05.611383 (+ 28624us) log.cc:844] Serialized 14 byte log entry
0709 13:08:05.690760 (+ 79377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.690837 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:05.760046 (+ 69209us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b4800130158. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb79d24aa1 00007fcb79d1bb37 00007fcb79d1b9ca 00007fcb79e1326f 00007fcb79e12f37 00007fcb79e2755c 00007fcb79e27508 00007fcb79e27488 00007fcb79e27443 00007fcb79e266ec 00007fcb78d50a96
0709 13:08:06.907407 (+1147361us) spinlock_profiling.cc:243] Waited 8.35 ms on lock 0x7b2400295898. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71b949c4 00007fcb71b92acb 00007fcb71d4ddc9 00007fcb71d4da55 00007fcb71d4d09d 00007fcb7497f10e 00007fcb7499688c 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221
0709 13:08:06.938331 (+ 30924us) raft_consensus.cc:1685] finished
0709 13:08:06.938351 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.938453 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22158,"prepare.queue_time_us":71212,"prepare.run_cpu_time_us":1064,"prepare.run_wall_time_us":14845,"raft.queue_time_us":22318,"raft.run_cpu_time_us":3569,"raft.run_wall_time_us":122834,"spinlock_wait_cycles":67100928,"thread_start_us":203078,"threads_started":3,"wal-append.queue_time_us":211997}
I20260709 13:08:06.968654 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.974697 22825 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:06.944917 26374 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:06.970156 22826 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:06.966969 22904 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:08:06.973268 22747 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:06.972705 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:06.977240 22752 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:06.977540 23444 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:08:06.977270 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:06.969406 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.971419 22871 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:06.950903 26328 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:06.978564 23447 raft_consensus.cc:2410] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:08:06.973989 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.979449 22871 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:06.980700 22747 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:08:06.951759 26404 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.980700 22830 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:08:06.985574 22748 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 4.
I20260709 13:08:06.951084 26315 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:06.900655 26312 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:06.976297 26247 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.974414 26321 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:06.976586 26157 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:06.996264 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.989688 26315 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.989985 26322 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:07.000313 26157 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.000926 26351 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:07.002400 26157 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:06.996749 26349 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:07.003849 26349 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.005033 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2091) took 1787 ms. Trace:
I20260709 13:08:07.005414 22896 rpcz_store.cc:276] 0709 13:08:05.217545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.217688 (+   143us) service_pool.cc:224] Handling call
0709 13:08:05.252824 (+ 35136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:05.381925 (+129101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.004844 (+1622919us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400184a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:07.004984 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294758784}
I20260709 13:08:07.005940 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.006114 26324 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:07.006997 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.990123 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.007697 22753 raft_consensus.cc:2410] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:08:06.990548 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.976964 26250 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:06.990918 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.985970 26328 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:06.991947 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:07.018787 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 4b7d2a5f73cc4afd98b29c55cc0eb72b: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:07.020787 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:06.992570 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.993371 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:07.014883 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.017813 22751 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:06.992087 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:06.965869 26365 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.024382 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.997288 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:06.998126 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:06.955312 26209 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:08:06.999522 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:07.010040 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:06.992488 26077 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:07.015729 22828 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:06.994498 26271 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:08:07.021463 22823 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:07.024804 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:06.993021 26431 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:07.023874 22898 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:07.062907 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:07.064076 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.064158 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:07.064910 22751 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:07.022714 23447 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:07.066982 22901 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:07.067422 26077 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.067957 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.026153 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:07.059116 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 61026.8us
I20260709 13:08:07.069621 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:07.039937 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.059685 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.060320 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:07.070796 22900 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:07.071681 22904 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:08:07.061013 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.061168 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.060683 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.073900 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.074054 22748 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:07.074995 22904 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:08:07.070796 22754 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:06.993393 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.077178 22747 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:07.077679 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:07.078138 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.077680 22747 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:07.079448 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.080034 22752 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:07.076606 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:07.081570 22901 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:08:07.063836 26317 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:07.076452 26157 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:07.082401 26317 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.025523 26321 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.082826 26317 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:07.082778 26157 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:07.068114 26374 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:07.083989 26374 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.086350 26374 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.087037 26054 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:07.087448 26054 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.087818 26054 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:07.087549 26328 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.088994 26328 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:07.070065 26077 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:07.028151 26349 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:07.029877 26428 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:08:07.027019 26315 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.024302 26364 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:07.063570 26307 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:07.077441 26324 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:06.886447 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:07.007856 26404 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:06.997524 26430 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:08:07.093569 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:07.094316 22751 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:07.064334 26431 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:07.094666 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:07.093603 26387 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.069888 26250 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.097183 22898 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:07.096326 26243 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:08:07.085045 26157 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.083078 26321 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:07.132629 22824 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:07.101184 26364 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:07.093653 26360 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:08:07.105914 26307 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:07.106158 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:07.105948 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.107069 22830 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:08:07.113811 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.119601 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:07.118250 26429 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:08:07.123096 22827 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 5.
I20260709 13:08:07.125525 26390 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:07.129740 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:07.131834 22750 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:07.102329 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:07.140348 22828 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:07.098412 26349 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:07.038172 26313 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.108644 26324 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:07.127032 25991 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:07.108559 26219 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:08:07.137089 22747 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.136855 26261 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:07.128183 26251 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:08:07.179720 22899 raft_consensus.cc:2368] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11.
I20260709 13:08:07.143389 26192 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:07.131198 26244 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.132665 26250 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.127637 26271 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:07.149544 22650 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:07.177618 26364 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:07.198716 26250 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.193186 26244 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:07.141708 23419 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:07.198045 22650 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 4 to 6, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 6 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.188665 26312 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:07.176100 26374 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:07.200681 26157 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:07.200336 22650 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:07.201104 26157 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:07.200946 26374 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.201922 26157 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.202961 26374 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:07.204596 26313 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:07.181567 26261 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:07.158967 26243 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:08:07.207273 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.209966 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:07.210711 22754 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:07.210172 22747 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:07.127300 26431 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:07.196448 26317 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:07.241429 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.142189 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:07.201923 22650 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 5 to 8, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 8 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:07.263718 22652 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33010 (request call id 224) took 9677 ms. Trace:
W20260709 13:08:07.263989 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 121871us
I20260709 13:08:07.156749 26365 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:07.264307 22650 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 6 to 10, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 10 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:08:07.286721 26007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.528s	user 0.024s	sys 0.016s
I20260709 13:08:07.131201 26387 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.178927 26054 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:07.288160 26404 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.288829 26387 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:07.289822 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:07.177728 26315 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:08:07.287171 26007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.529s	user 0.024s	sys 0.016s
I20260709 13:08:07.249543 26243 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.276583 22650 catalog_manager.cc:5697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 8, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 8 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:07.206722 26157 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:07.264073 22652 rpcz_store.cc:276] 0709 13:07:57.528990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:57.533469 (+  4479us) service_pool.cc:224] Handling call
0709 13:08:07.206483 (+9673014us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:04.262022 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:04.558237 (+296215us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:04.568028 (+  9791us) write_op.cc:435] Acquired schema lock
0709 13:08:04.570954 (+  2926us) tablet.cc:662] Decoding operations
0709 13:08:04.672145 (+101191us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:04.672235 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:04.672268 (+    33us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:08:04.673175 (+   907us) tablet.cc:701] Row locks acquired
0709 13:08:04.673244 (+    69us) write_op.cc:260] PREPARE: finished
0709 13:08:04.673447 (+   203us) write_op.cc:270] Start()
0709 13:08:04.673625 (+   178us) write_op.cc:276] Timestamp: P: 1783602484673378 usec, L: 0
0709 13:08:04.673648 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:08:04.674399 (+   751us) log.cc:844] Serialized 3574 byte log entry
0709 13:08:05.681171 (+1006772us) op_driver.cc:464] REPLICATION: finished
0709 13:08:05.930591 (+249420us) write_op.cc:301] APPLY: starting
0709 13:08:05.930690 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:05.930922 (+   232us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:05.931066 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:05.935787 (+  4721us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:05.964402 (+ 28615us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:08:05.964450 (+    48us) write_op.cc:312] APPLY: finished
0709 13:08:05.964809 (+   359us) log.cc:844] Serialized 71 byte log entry
0709 13:08:05.964994 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:05.965117 (+   123us) write_op.cc:454] Released schema lock
0709 13:08:05.965180 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:05.965598 (+   418us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:05.965625 (+    27us) write_op.cc:454] Released schema lock
0709 13:08:06.658955 (+693330us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b54000ab4d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74df832a 00007fcb74d7df14 00007fcb74d7dec3 00007fcb74d7de38 00007fcb74d7ddf3 00007fcb74d7d0dc 00007fcb74da6e16 00007fcb74d8ecbb 00007fcb74d7939d 00007fcb74d7cd75
Metrics: {"mutex_wait_us":89146,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":247080,"apply.run_cpu_time_us":3627,"apply.run_wall_time_us":34969,"num_ops":9,"prepare.queue_time_us":221992,"prepare.run_cpu_time_us":100225,"prepare.run_wall_time_us":1364595,"raft.queue_time_us":140506,"raft.run_cpu_time_us":82294,"raft.run_wall_time_us":1088548,"replication_time_us":939782,"spinlock_wait_cycles":70178176,"thread_start_us":1297981,"threads_started":4,"wal-append.queue_time_us":716756,"wal-append.run_cpu_time_us":8807,"wal-append.run_wall_time_us":1569052}]]}
I20260709 13:08:07.292929 26375 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.299773 26347 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:07.301209 26271 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:07.157598 26349 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:07.176180 25991 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.301542 26178 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:07.249805 26192 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:07.275966 26365 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:07.301847 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 160212us
I20260709 13:08:07.339198 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:07.302436 26251 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:07.304069 26440 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.305115 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.306505 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:07.307242 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.346552 23450 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:07.307364 22754 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:07.308151 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:07.348392 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.322054 26347 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:07.350558 22905 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 4. Current term is 6. Ops: []
W20260709 13:08:07.312186 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 217570us
I20260709 13:08:07.310184 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:07.289063 26054 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.311600 22650 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:07.320670 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 375375us
I20260709 13:08:07.328238 26361 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:07.293059 26315 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:07.324826 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.359635 23449 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:07.328133 23452 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:08:07.361454 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.318259 26243 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:07.289153 26324 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.307899 26219 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:08:07.344846 26360 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.368137 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.350479 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:07.312693 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 312955us
I20260709 13:08:07.369422 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.353111 22650 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 9 to 11, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:07.370190 23452 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:07.354192 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:07.323021 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.370615 22650 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 8 to 10, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 10 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:07.354444 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:07.355346 26439 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:08:07.355837 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:07.358894 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48793.1us
W20260709 13:08:07.364605 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 99757.6us
I20260709 13:08:07.375663 23450 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:07.346258 26365 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:07.308339 22747 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:08:07.379786 22747 pending_rounds.cc:85] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:08:07.380095 22747 pending_rounds.cc:107] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:08:07.380330 22747 raft_consensus.cc:2889] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:07.326579 26271 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.372318 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:07.372038 22650 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 5 to 7, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.389665 22650 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 13, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 13 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.391212 22650 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 9 to 11, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:07.374430 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.392485 22903 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:08:07.306082 26307 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:07.378731 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:07.381461 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:07.379815 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.372318 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.394201 23449 raft_consensus.cc:2376] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11. Re-sending same reply.
I20260709 13:08:07.383476 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.384291 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.379197 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:07.385239 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.386089 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.395032 22899 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:08:07.395661 23447 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:07.359347 26251 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:07.366384 26443 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.395768 22896 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:08:07.396677 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:07.396950 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:07.407680 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:07.398326 23452 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:07.408435 23446 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:07.379245 26338 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:07.409497 23416 leader_election.cc:400] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11.
I20260709 13:08:07.409895 23416 leader_election.cc:403] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:07.399822 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.400295 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:07.400928 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.401813 23448 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:07.412801 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.413079 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.402012 23450 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:07.391341 26380 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:07.417991 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.398941 22650 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 12, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 12 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.418980 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.367334 26430 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:08:07.389052 26384 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:07.391546 26129 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:07.389756 26261 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.422876 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.420435 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:07.436499 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:07.420197 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.432410 26007 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:07.437502 26007 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:07.438570 26007 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.439953 26007 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:07.432502 26384 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.442095 26384 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:07.432686 26338 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:07.444181 26338 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.424822 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:07.446771 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:07.425922 22650 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 4 to 8, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 8 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:07.448566 26456 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:07.376526 26244 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:08:07.449798 26351 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:07.375635 26404 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:08:07.378108 26366 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:08:07.433905 23450 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:08:07.379158 26357 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:08:07.434798 22904 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:08:07.448344 22650 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 5 to 7, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.457048 22650 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 5 to 7, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.432019 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.460979 22899 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:07.407078 26264 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:07.389492 26256 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:08:07.419389 26322 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:07.466347 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:07.388078 26360 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.419740 26328 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:08:07.486881 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 89295.6us
I20260709 13:08:07.482200 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:07.467080 22748 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:07.469681 22650 catalog_manager.cc:5697] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 9, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 9 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:07.496300 22650 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 8, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 8 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.498891 22650 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 5 to 7, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.501696 22650 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 10, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 10 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:07.429961 26313 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:07.430140 26445 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:08:07.419553 26250 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:08:07.461258 26458 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.481169 26322 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:07.552510 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 173399us
I20260709 13:08:07.552861 23445 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:08:07.517918 26264 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:07.415405 26192 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.563530 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:07.480734 26357 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:07.599351 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:07.518088 26424 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:07.603221 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:07.603756 22748 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:07.480028 26456 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:07.465430 26446 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:07.575410 23445 pending_rounds.cc:85] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 7
I20260709 13:08:07.470770 26359 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:07.420128 26444 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:08:07.408452 26315 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:07.480302 26261 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.631364 22829 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:07.630652 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:07.605728 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:07.606819 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:07.605108 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:07.610266 23445 pending_rounds.cc:107] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 6.8
W20260709 13:08:07.595631 26433 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.392s	user 0.001s	sys 0.000s
I20260709 13:08:07.633481 23445 raft_consensus.cc:2889] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:07.627194 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 139615us
I20260709 13:08:07.628887 26375 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.630759 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:07.593925 26322 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:07.576977 22797 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:07.430405 23447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:08:07.596278 26264 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:07.527966 26430 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:08:07.672894 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.676033 22748 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:07.640726 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:07.667207 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:07.676875 22902 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:07.667289 22830 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:07.670035 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:07.670672 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:07.554889 26317 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.678213 23449 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:07.465620 26129 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.569689 26425 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:08:07.684938 23453 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:08:07.677479 22871 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:07.609567 26357 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:07.610198 26446 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.608109 25850 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:07.663830 25991 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:07.709950 26435 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:08:07.706038 26357 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:07.634613 26256 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:07.610045 26456 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:07.664785 26178 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:07.685469 26264 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:07.721191 23452 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:07.728507 23447 raft_consensus.cc:606] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:07.728969 23447 raft_consensus.cc:614] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:07.620855 26359 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:07.593818 26447 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
I20260709 13:08:07.737855 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:07.704159 26349 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.705376 26375 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:07.752470 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.715176 25850 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.573134 26251 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.666292 26261 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.713722 26446 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:07.747040 26364 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:07.717332 26256 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:07.467643 26347 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.720710 26328 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:07.778728 26450 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:08:07.778808 26328 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.349982 26440 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.779625 26129 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:07.773056 26453 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:08:07.783212 26440 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:07.784137 26077 consensus_meta.cc:287] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.000s
I20260709 13:08:07.784535 26077 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:07.785631 22897 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:07.786087 26312 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:07.786933 26454 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:07.787482 26454 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.787891 26454 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:07.793576 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.794277 22902 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 5.
I20260709 13:08:07.738549 26244 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:07.601142 26458 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.415846 26443 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.749852 26361 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:07.722050 26321 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:07.794883 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.795184 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:07.812711 22748 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
W20260709 13:08:07.774114 26451 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:08:07.833094 22752 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:08:07.797206 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 101952us
I20260709 13:08:07.717585 26247 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
W20260709 13:08:07.760426 26442 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.344s	user 0.001s	sys 0.001s
W20260709 13:08:07.760252 26441 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
W20260709 13:08:07.845916 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 240124us
I20260709 13:08:07.847477 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:07.748117 26359 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:07.760622 25850 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:07.777756 26256 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:07.756410 26271 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.809252 26360 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:07.834579 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:07.837707 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 149862us
I20260709 13:08:07.878499 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:07.832306 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:07.842362 22823 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:07.849282 23448 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:08:07.880399 22902 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:07.880757 22899 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:07.880702 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.762423 26375 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:07.811450 26361 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:07.807147 26364 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:07.866886 23446 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
W20260709 13:08:07.869602 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 292667us
I20260709 13:08:07.881640 22901 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:07.881891 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:07.897042 22904 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:07.891983 22902 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:08:07.893014 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60788.4us
I20260709 13:08:07.889204 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:07.900473 22899 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:07.854938 26470 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:07.925889 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:07.926112 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:07.879813 26243 consensus_meta.cc:287] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.500s	user 0.009s	sys 0.000s
W20260709 13:08:07.943254 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49420.2us
I20260709 13:08:07.868360 26460 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:07.944970 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:07.959501 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 125000us
I20260709 13:08:07.960475 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:08:07.962985 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet e4cc99e6a1f14071a587af061e82fee0: Illegal state: Replica 7aad11e70ca94fce8f672db05f1c28c3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:07.927327 26271 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:07.970830 22899 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:07.934113 26470 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:07.976468 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 291660us
I20260709 13:08:07.977509 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:07.925709 26313 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:08:07.946493 26444 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:07.930143 26360 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:08:07.926435 26317 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:08:07.946651 26471 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:07.979192 26470 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:07.984369 26007 consensus_meta.cc:287] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.004s
I20260709 13:08:07.985219 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:07.981510 25991 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:07.956877 26243 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:08.009238 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:07.992105 26380 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.002s
I20260709 13:08:08.030900 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:07.997543 26465 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:08:07.990278 26440 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:07.994189 26458 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:07.987720 26390 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:08.027665 26360 consensus_queue.cc:1243] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:08.028705 26321 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.024763 26447 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:07.989806 26424 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.036664 26444 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:08:08.036769 26380 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:08.036702 26440 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:08.030383 26007 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:08:08.057441 26469 consensus_queue.cc:1059] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:08:08.058981 26479 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:08:08.060755 26431 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.768s	user 0.005s	sys 0.002s
I20260709 13:08:08.064082 26473 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:08.065366 26374 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:08:07.961580 26460 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:08.029559 25991 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:08.032207 26077 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:08.108852 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 74163.4us
I20260709 13:08:08.042847 26261 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:08.045039 26219 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:08.056217 26463 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:08:08.042191 26364 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:08.033898 26178 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 13:08:08.109268 26445 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11.
I20260709 13:08:08.108599 26447 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:08.117800 26424 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:08.135123 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 145537us
I20260709 13:08:08.116784 26390 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.121243 26440 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:08.126940 26479 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.130195 26473 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:08.116894 26322 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:08.132862 26431 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:08.165450 26219 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:08.118366 26321 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:08.169814 26365 consensus_meta.cc:287] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.776s	user 0.009s	sys 0.001s
I20260709 13:08:08.169616 26491 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:08.333570 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.143824 25991 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:08.301102 26324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.024s	sys 0.046s
I20260709 13:08:08.247058 26261 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:08.183164 26443 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:08.246016 26375 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:08.147770 26454 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:08.294203 26309 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 13:08:08.437811 23451 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:08.342692 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 157704us
W20260709 13:08:08.342677 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 387363us
W20260709 13:08:08.390383 26488 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 13:08:08.844242 26359 consensus_meta.cc:287] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.972s	user 0.003s	sys 0.005s
I20260709 13:08:08.437016 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1061) took 1098 ms. Trace:
I20260709 13:08:08.294536 26447 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:08.934633 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:08.935400 22829 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:08.936748 22901 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
W20260709 13:08:08.868840 26264 consensus_meta.cc:287] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.116s	user 0.005s	sys 0.005s
I20260709 13:08:08.417385 23449 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
W20260709 13:08:08.921392 22868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.008s	sys 0.006s
W20260709 13:08:08.860416 25850 consensus_meta.cc:287] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.990s	user 0.006s	sys 0.002s
W20260709 13:08:08.923787 22797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.006s	sys 0.009s
I20260709 13:08:08.827276 26495 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:08.911897 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:08.909169 26324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.117s	user 0.037s	sys 0.074s
W20260709 13:08:08.864354 26256 consensus_meta.cc:287] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.989s	user 0.004s	sys 0.004s
W20260709 13:08:08.839332 26493 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.462s	user 0.000s	sys 0.001s
W20260709 13:08:08.837641 26470 consensus_meta.cc:287] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.845s	user 0.007s	sys 0.003s
I20260709 13:08:08.926090 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:08.928124 23446 rpcz_store.cc:276] 0709 13:08:07.338866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.338989 (+   123us) service_pool.cc:224] Handling call
0709 13:08:08.436798 (+1097809us) spinlock_profiling.cc:243] Waited 68.7 ms on lock 0x7b54000b2cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:08.436949 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157936384}
W20260709 13:08:08.773967 26357 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.063s	user 0.008s	sys 0.003s
W20260709 13:08:08.847303 26361 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.917s	user 0.009s	sys 0.000s
W20260709 13:08:08.390676 26481 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
W20260709 13:08:08.828802 26384 consensus_meta.cc:287] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.386s	user 0.010s	sys 0.000s
I20260709 13:08:08.933876 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:08.409277 22823 consensus_meta.cc:287] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.002s
W20260709 13:08:08.352349 26456 consensus_meta.cc:287] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.594s	user 0.008s	sys 0.001s
W20260709 13:08:08.571419 26157 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.250s	user 0.009s	sys 0.000s
I20260709 13:08:08.934525 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:40298 (request call id 7) took 1534 ms. Trace:
I20260709 13:08:08.435566 26467 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.774s
I20260709 13:08:08.940335 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:08.941061 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:08.933449 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:08.830518 26424 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:08.391693 26490 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
W20260709 13:08:08.436354 26486 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.293s	user 0.000s	sys 0.000s
W20260709 13:08:08.841720 26446 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.039s	user 0.008s	sys 0.001s
W20260709 13:08:08.587502 26312 consensus_meta.cc:287] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.801s	user 0.008s	sys 0.000s
W20260709 13:08:08.949473 22797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.006s	sys 0.009s
I20260709 13:08:08.958897 26503 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:08.944339 22903 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:08.341461 26445 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.909729 26479 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:08.910315 26473 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:08.920742 26219 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:08.573285 26243 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:08.550079 26497 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:08.923102 26375 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:08.957115 22902 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:08.957285 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 45484.9us
I20260709 13:08:08.966284 22823 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:08.965348 22830 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:08.973042 22831 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:08.973178 22824 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:08.926599 26261 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:08.971524 23447 rpcz_store.cc:276] 0709 13:08:07.395972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.396247 (+   275us) service_pool.cc:224] Handling call
0709 13:08:08.922979 (+1526732us) spinlock_profiling.cc:243] Waited 60.5 ms on lock 0x7b54006dea54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74df5853 00007fcb79dc360a 00007fcb733497d4 00007fcb73349762 00007fcb73349691 00007fcb7334960c 00007fcb733488c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:08.923105 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139054464}
I20260709 13:08:08.377727 26390 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:08.984462 22754 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:08.942408 26359 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:08.390475 26478 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.787s
I20260709 13:08:08.929960 26365 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:08.951203 26264 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:08.995826 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 1.03026e+06us
I20260709 13:08:09.036870 22902 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:09.037596 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:08.950121 22868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.009s	sys 0.006s
I20260709 13:08:08.951787 25850 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:08.983031 26311 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.019s	sys 0.013s
W20260709 13:08:08.981142 26482 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.011s	sys 0.014s
I20260709 13:08:08.979389 26456 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:08.957270 26256 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.060691 22898 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:08.970278 26357 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:08.970213 26361 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:09.029795 26271 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.000s	user 0.008s	sys 0.063s
I20260709 13:08:08.965338 26470 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:09.038295 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 1.05338e+06us
I20260709 13:08:09.039016 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:09.006363 26506 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.955317 26495 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.010450 26309 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:08.972770 26384 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:08.933490 26491 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.050554 26505 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:09.052074 26264 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:09.056493 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1073) took 1646 ms. Trace:
I20260709 13:08:09.078915 23452 rpcz_store.cc:276] 0709 13:08:07.409718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.419957 (+ 10239us) service_pool.cc:224] Handling call
0709 13:08:09.056348 (+1636391us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54006dea54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:09.056442 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":652667264}
I20260709 13:08:09.057081 26390 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.005733 26312 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:09.048111 26157 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:08:09.066682 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:09.089551 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:09.020886 26445 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.092279 22900 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:09.039592 26460 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:08.995797 26446 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:09.071605 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:09.094707 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:09.076812 22826 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:09.077651 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:09.077001 22748 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:09.063627 26271 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.008s	sys 0.063s
W20260709 13:08:09.076167 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 107104us
I20260709 13:08:09.079903 22896 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:09.063465 26512 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.097611 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:09.098291 22902 raft_consensus.cc:2368] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9.
I20260709 13:08:09.074173 26467 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.061621 26498 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.030179 26375 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.074206 26338 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 13:08:09.047016 26311 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.020s	sys 0.013s
I20260709 13:08:09.076499 26349 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:08:09.058171 26482 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.011s	sys 0.014s
W20260709 13:08:09.090934 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 62702.1us
I20260709 13:08:09.021733 26503 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:09.093387 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 55860.2us
I20260709 13:08:09.111336 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:09.096208 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:09.081027 26322 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.103526 26519 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:09.088524 26264 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:09.092757 26431 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:09.035043 26497 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:09.112799 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:09.097295 26271 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:09.101945 26424 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:09.099604 26390 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:09.113546 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:09.099618 26515 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:08:09.123588 26499 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:08:09.104996 26454 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.126376 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:09.129463 22749 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:09.126703 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:09.126996 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:09.106457 26349 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:09.075698 26506 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:09.138685 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:09.101867 26309 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:08:09.147068 22824 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:08:09.107316 26364 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 13:08:09.150250 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 56568.4us
I20260709 13:08:09.150462 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.104295 26467 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.124548 26424 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:09.152962 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:09.129418 26322 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.137267 26349 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:09.162586 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:09.166846 22831 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:09.167375 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:09.131724 26520 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:09.173785 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:09.168464 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:09.189617 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:09.163223 26424 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:09.190174 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:09.102190 26512 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:09.154273 26321 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.148496 26510 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:09.144023 26478 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:09.110109 26498 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.148660 26454 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:09.234068 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:09.204399 22719 connection.cc:256] RPC call timeout handler was delayed by -1.8660435819999748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
I20260709 13:08:09.238333 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:09.238615 22905 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:08:09.238866 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 70473.3us
I20260709 13:08:09.244410 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:09.242502 23453 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:09.247289 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:09.245419 22902 raft_consensus.cc:2410] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:08:09.248157 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 121551us
I20260709 13:08:09.188993 26338 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:09.164456 26460 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:09.191089 26482 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:08:09.188436 26261 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.191192 26264 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:09.235651 26359 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.257782 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:09.266383 23452 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:09.239027 25850 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.190969 26390 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:08:09.269825 22905 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:08:09.190346 26505 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:08:09.271545 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.276387 22824 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:09.273403 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.277516 22822 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:09.251801 26256 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.274515 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.274994 22831 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:08:09.250710 26321 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:09.252610 26515 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.264956 26264 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.293778 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:09.188722 26322 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.273553 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.249974 26361 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.290764 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2118) took 1402 ms. Trace:
I20260709 13:08:09.290977 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.268582 26456 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.123175 26519 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.249723 26431 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:08:09.252964 26308 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:08:09.259595 26460 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:09.272217 26454 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:08:09.269622 26482 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:08:09.256217 26508 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.238s
I20260709 13:08:09.246431 26271 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.452943 22829 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
W20260709 13:08:09.379873 22900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2115) took 1979 ms (client timeout 1987 ms). Trace:
I20260709 13:08:09.375846 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 844) took 1242 ms. Trace:
I20260709 13:08:09.452940 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:09.454205 22900 rpcz_store.cc:269] 0709 13:08:07.399976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.400086 (+   110us) service_pool.cc:224] Handling call
0709 13:08:09.379709 (+1979623us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540034c1d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:09.379823 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3862477056}
I20260709 13:08:09.438470 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:09.390275 22794 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.987s (SENT) [suppressed 10 similar messages]
I20260709 13:08:09.379010 23452 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:08:09.382427 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 651) took 1777 ms. Trace:
I20260709 13:08:09.458521 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:09.458781 22826 rpcz_store.cc:276] 0709 13:08:07.604811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.605219 (+   408us) service_pool.cc:224] Handling call
0709 13:08:07.605468 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.095356 (+1489888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.382219 (+286863us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540008b454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:09.382376 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3383265152}
I20260709 13:08:09.459205 22752 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:09.274397 26338 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:08:09.461112 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.454399 22898 rpcz_store.cc:276] 0709 13:08:08.133635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.133789 (+   154us) service_pool.cc:224] Handling call
0709 13:08:08.147168 (+ 13379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.061062 (+913894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.375549 (+314487us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400206a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:09.375776 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2100433280}
I20260709 13:08:09.409623 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:09.380204 22901 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:08:09.276232 26384 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:09.471966 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.472234 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.467492 22900 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:08:09.309661 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.468508 26517 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:09.464032 22897 rpcz_store.cc:276] 0709 13:08:07.888522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.888720 (+   198us) service_pool.cc:224] Handling call
0709 13:08:07.888981 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:07.889263 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:07.889289 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:07.889724 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:08:08.295091 (+405367us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:08.376191 (+ 81100us) log.cc:844] Serialized 14 byte log entry
0709 13:08:08.378593 (+  2402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.378690 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.290496 (+911806us) raft_consensus.cc:1685] finished
0709 13:08:09.290524 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.290693 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2148,"prepare.queue_time_us":83370,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":98152,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":27834,"spinlock_wait_cycles":9556224,"thread_start_us":336048,"threads_started":1,"wal-append.queue_time_us":737979}
I20260709 13:08:09.475785 22826 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:09.467322 26349 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:09.477015 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.477125 22747 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:09.478609 26349 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:09.334506 26516 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.232s	user 0.001s	sys 0.001s
I20260709 13:08:09.481721 22826 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:09.487517 22826 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:09.488149 26452 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.488641 26452 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:09.489187 22826 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:09.490779 26452 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.291026 26321 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:09.491757 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:09.493813 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.494190 22829 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:09.492710 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 245476us
I20260709 13:08:09.296018 26264 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:09.353605 26464 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:09.407483 26512 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:09.353242 26506 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.368754 26497 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.333777 26446 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:09.496477 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 357835us
I20260709 13:08:09.496922 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:09.449328 26467 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.377647 26491 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:09.505823 22826 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:08:09.506045 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:09.511210 22752 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:09.513592 22824 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:09.340095 26523 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:09.308657 26503 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.292271 26495 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.467598 26445 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.520519 22747 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:09.312089 26515 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:09.491760 26251 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.433s	user 0.021s	sys 0.019s
I20260709 13:08:09.530607 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:09.353530 26525 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:09.498517 26321 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:09.493755 26338 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:09.537112 26479 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.000s
W20260709 13:08:09.549489 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 159218us
W20260709 13:08:09.548570 22868 connection.cc:256] RPC call timeout handler was delayed by -1.7188215669999636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
I20260709 13:08:09.545563 26384 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:09.555747 26491 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.558612 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:09.549213 26512 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:09.557657 26251 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.499s	user 0.021s	sys 0.019s
W20260709 13:08:09.570930 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 111604us
W20260709 13:08:09.576195 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.570142 26497 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.584600 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:09.570302 26445 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.591122 26338 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.574772 26479 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.570273 26444 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:09.593771 26440 consensus_meta.cc:287] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.667s	user 0.005s	sys 0.005s
I20260709 13:08:09.573652 26495 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:09.593919 26467 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:09.568686 26312 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:09.591053 26498 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.573953 26446 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.609705 26322 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.610179 26361 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:08:09.588868 25991 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.000s
I20260709 13:08:09.574721 26384 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:09.612092 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:09.584203 26456 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:09.576148 26464 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:09.514016 26517 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.620761 26459 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:09.597775 26454 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:09.625475 23447 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:09.620002 26380 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.621618 26495 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.585475 26512 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.588552 26491 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:09.673916 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 61876.7us
I20260709 13:08:09.674178 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:09.616051 26251 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:08:09.513909 26473 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:09.616765 26445 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.679844 26519 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.715201 26473 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.619436 26444 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:09.682037 26219 consensus_meta.cc:287] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.001s
I20260709 13:08:09.617036 26482 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:08:09.669109 26384 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:09.572121 26523 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.680670 26359 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:08:09.721642 26264 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.723284 22897 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:08:09.617674 26497 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.723629 26264 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.621485 26440 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.671923 26456 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.691721 26460 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.680037 26452 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:09.715052 26445 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.626847 26312 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.628229 26446 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:09.739671 26483 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.681s	user 0.013s	sys 0.026s
I20260709 13:08:09.719794 26219 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.689388 26454 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.704552 26495 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:09.692888 26459 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:09.688692 26515 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.728776 23452 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:09.728109 26512 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.730818 26440 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.727866 26456 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:09.731165 26491 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:09.760675 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.761333 22825 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:09.762248 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:09.731490 26347 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:09.734283 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 837) took 2096 ms. Trace:
I20260709 13:08:09.763281 22896 rpcz_store.cc:276] 0709 13:08:07.638005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.638557 (+   552us) service_pool.cc:224] Handling call
0709 13:08:07.638871 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.096442 (+1457571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.734022 (+637580us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400206f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:09.734214 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3309373184}
I20260709 13:08:09.735208 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2624) took 1913 ms. Trace:
W20260709 13:08:09.735522 26447 consensus_meta.cc:287] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.781s	user 0.007s	sys 0.002s
I20260709 13:08:09.725212 26497 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.733045 26523 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:09.739291 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:09.768332 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:09.741142 26446 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.761152 26251 consensus_queue.cc:1243] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:09.741194 26535 raft_consensus.cc:2764] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:08:09.668826 25991 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:09.747761 22868 leader_election.cc:336] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.938s (SENT)
I20260709 13:08:09.774380 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:09.728765 26452 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:09.705302 26380 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.739701 26424 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.764519 22748 rpcz_store.cc:276] 0709 13:08:07.822142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.822306 (+   164us) service_pool.cc:224] Handling call
0709 13:08:07.822598 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.095976 (+1273378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.734977 (+639001us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002a9754. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:09.735147 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2884256128}
I20260709 13:08:09.769910 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:09.753026 26454 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:09.750728 22829 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:09.755609 22899 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:08:09.794354 26537 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:08:09.794997 26463 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:09.796525 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:09.772243 26505 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:09.808257 22755 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:09.773972 26482 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:09.811717 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:09.812384 22829 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:09.795392 26463 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.773736 26479 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:09.789739 26445 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.790181 26264 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:09.796388 26452 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:09.818971 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:09.769659 26447 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.834950 26444 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.760584 26459 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:09.835345 26384 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.749492 26464 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:09.835930 26459 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:09.809990 26349 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:08:09.817063 26387 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:09.844408 22829 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:09.840481 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:09.817605 26463 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:09.807523 26361 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:08:09.866247 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1076) took 2231 ms. Trace:
I20260709 13:08:09.868750 22751 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:09.868580 23448 rpcz_store.cc:276] 0709 13:08:07.608662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.609592 (+   930us) service_pool.cc:224] Handling call
0709 13:08:07.609907 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.879787 (+269880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.840304 (+1960517us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54006833d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:09.840503 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532487936}
I20260709 13:08:09.822511 26482 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:09.871496 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:09.872164 22829 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:09.818614 26390 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:08:09.874275 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:09.876243 23448 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:09.871517 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:09.877381 23452 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:09.822038 26474 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:08:09.883349 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:09.884702 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:09.885386 23452 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:09.821234 26445 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:09.888700 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:09.890606 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:09.894701 26517 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:09.896740 23452 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:09.902352 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:09.879563 26505 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:09.910287 22829 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:09.919857 23448 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:09.922885 26349 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:09.882360 26387 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:08:09.942710 26472 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:09.945381 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:09.991411 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:09.998741 26501 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.106s	user 0.004s	sys 0.005s
I20260709 13:08:09.916134 26505 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:09.919139 26219 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:10.057960 25991 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.919004 26312 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.918540 26482 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:09.920156 26384 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:09.923286 26349 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:10.120087 26447 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:10.112212 26321 consensus_meta.cc:287] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.520s	user 0.004s	sys 0.006s
W20260709 13:08:10.132850 26541 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
W20260709 13:08:10.137795 23453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.011s	sys 0.036s
W20260709 13:08:10.142242 23453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.011s	sys 0.036s
W20260709 13:08:10.143361 22905 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.014s	sys 0.042s
W20260709 13:08:10.143800 22905 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.014s	sys 0.042s
I20260709 13:08:10.144956 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:10.111068 26375 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:10.112474 26472 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:10.182660 22826 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.035s	sys 0.055s
W20260709 13:08:10.183099 22826 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.036s	sys 0.055s
W20260709 13:08:10.185918 22756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.003s	sys 0.045s
W20260709 13:08:10.186254 22756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.003s	sys 0.045s
W20260709 13:08:10.229079 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:10.232290 22749 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.031s	sys 0.059s
W20260709 13:08:10.233103 22749 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.031s	sys 0.059s
I20260709 13:08:10.131120 26424 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:10.251219 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:10.256862 23448 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:10.252311 26351 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.134528 26463 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.147291 26321 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.270365 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1591) took 1011 ms. Trace:
I20260709 13:08:10.273523 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:10.332196 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 58756.3us
I20260709 13:08:10.277001 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1585) took 1076 ms. Trace:
I20260709 13:08:10.277999 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:10.334720 22829 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:10.243597 26380 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:10.284144 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:10.326558 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:10.267154 26462 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:10.252480 26505 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:10.333760 22901 rpcz_store.cc:276] 0709 13:08:09.200821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.243959 (+ 43138us) service_pool.cc:224] Handling call
0709 13:08:09.379993 (+136034us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.465869 (+ 85876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.276784 (+810915us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b540034c1d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:10.276935 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285646208}
I20260709 13:08:10.331029 22904 rpcz_store.cc:276] 0709 13:08:09.259095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.381053 (+121958us) service_pool.cc:224] Handling call
0709 13:08:09.381351 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.381677 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.381709 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.460864 (+ 79155us) log.cc:844] Serialized 26 byte log entry
0709 13:08:09.556435 (+ 95571us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:09.556553 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.556701 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.270118 (+713417us) raft_consensus.cc:1685] finished
0709 13:08:10.270146 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.270301 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1030,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":3088,"thread_start_us":61119,"threads_started":1,"wal-append.queue_time_us":196995}
I20260709 13:08:10.365761 23445 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:10.247335 26349 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:10.357391 26521 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:10.249405 26472 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.336474 26351 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.373805 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:10.369118 22829 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:10.357733 26462 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:10.375213 26454 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.581s	user 0.007s	sys 0.003s
I20260709 13:08:10.362831 26445 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:10.378451 26464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.002s	sys 0.012s
I20260709 13:08:10.377964 26505 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:10.259871 26482 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:10.404753 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:10.405206 26349 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:10.412403 22868 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.040s	sys 0.122s
W20260709 13:08:10.412788 22868 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.040s	sys 0.122s
W20260709 13:08:10.406911 26544 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 13:08:10.413066 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 637396us
I20260709 13:08:10.402112 26521 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:10.406903 26464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.002s	sys 0.012s
W20260709 13:08:10.417997 22750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.023s	sys 0.046s
I20260709 13:08:10.428815 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:10.429491 22829 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:10.428635 22750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.025s	sys 0.047s
I20260709 13:08:10.430915 22822 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:10.431286 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2629) took 1023 ms. Trace:
W20260709 13:08:10.434887 22719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.009s	sys 0.013s
W20260709 13:08:10.420987 26459 consensus_meta.cc:287] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.583s	user 0.005s	sys 0.007s
W20260709 13:08:10.443780 26542 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:08:10.436106 22719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.010s	sys 0.013s
W20260709 13:08:10.456339 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 611880us
W20260709 13:08:10.457171 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:10.434165 26271 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.029s	sys 0.030s
I20260709 13:08:10.405660 26454 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.454916 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2630) took 1046 ms. Trace:
I20260709 13:08:10.448477 26521 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:10.464919 22753 rpcz_store.cc:276] 0709 13:08:09.408032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.413133 (+  5101us) service_pool.cc:224] Handling call
0709 13:08:09.413486 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.466230 (+ 52744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.466271 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.467426 (+  1155us) log.cc:844] Serialized 26 byte log entry
0709 13:08:09.907341 (+439915us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:09.907394 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.907477 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.449921 (+542444us) raft_consensus.cc:1685] finished
0709 13:08:10.449950 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.454842 (+  4892us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1490,"raft.queue_time_us":59166,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":9627,"thread_start_us":392961,"threads_started":1,"wal-append.queue_time_us":788077}
W20260709 13:08:10.482007 22754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 977) took 3069 ms (client timeout 2169 ms). Trace:
W20260709 13:08:10.482714 22754 rpcz_store.cc:269] 0709 13:08:07.412752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.412884 (+   132us) service_pool.cc:224] Handling call
0709 13:08:10.481670 (+3068786us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400045e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:10.481860 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3612461440}
I20260709 13:08:10.486409 22825 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:08:10.472968 26495 consensus_meta.cc:287] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.000s
W20260709 13:08:10.471753 26543 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:08:10.463776 26361 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:10.476994 26550 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:08:10.477689 26547 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:08:10.433503 26462 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:10.546882 22824 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.037s	sys 0.096s
W20260709 13:08:10.510643 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:10.589148 22824 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.065s	sys 0.108s
W20260709 13:08:10.501724 23452 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.011s	sys 0.021s
I20260709 13:08:10.593266 22829 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:10.594336 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:10.462108 26271 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.029s	sys 0.031s
I20260709 13:08:10.476863 26447 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:10.591077 23452 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.022s	sys 0.027s
W20260709 13:08:10.613924 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.632877 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:10.635620 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:10.637291 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1588) took 1393 ms. Trace:
I20260709 13:08:10.637488 22898 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:10.486117 26219 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:10.645562 26219 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.647403 26448 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:10.642912 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:10.648104 26448 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:10.650223 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.650411 26445 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.650522 26250 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.650802 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:10.650924 26445 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.651485 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2633) took 1242 ms. Trace:
I20260709 13:08:10.651841 22755 rpcz_store.cc:276] 0709 13:08:09.408875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.551433 (+142558us) service_pool.cc:224] Handling call
0709 13:08:09.551730 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.811483 (+259753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.651288 (+839805us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54000780d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:10.651435 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589472128}
I20260709 13:08:10.652537 23416 leader_election.cc:400] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9.
I20260709 13:08:10.652858 23416 leader_election.cc:403] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:08:10.652941 26445 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:10.653493 26347 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:10.651059 26250 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.489936 26321 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:10.583540 26357 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:10.490217 26463 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:10.685474 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:10.637684 22902 rpcz_store.cc:276] 0709 13:08:09.243597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.254009 (+ 10412us) service_pool.cc:224] Handling call
0709 13:08:09.254455 (+   446us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:09.486454 (+231999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.486485 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:09.486978 (+   493us) log.cc:844] Serialized 52 byte log entry
0709 13:08:09.902422 (+415444us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:10.312853 (+410431us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.313289 (+   436us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.330488 (+ 17199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.331826 (+  1338us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.637107 (+305281us) raft_consensus.cc:1685] finished
0709 13:08:10.637126 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.637242 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31624,"prepare.queue_time_us":840761,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":1463,"raft.queue_time_us":39336,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":32201,"spinlock_wait_cycles":13000960,"thread_start_us":978305,"threads_started":3,"wal-append.queue_time_us":775091}
I20260709 13:08:10.688442 23446 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:10.692124 22898 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:10.653914 26209 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9.
I20260709 13:08:10.697773 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:10.684490 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1700) took 1424 ms. Trace:
W20260709 13:08:10.718485 23453 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.022s	sys 0.058s
W20260709 13:08:10.734191 23453 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.026s	sys 0.058s
I20260709 13:08:10.647895 26219 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:10.699565 22822 rpcz_store.cc:276] 0709 13:08:09.260385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.488353 (+227968us) service_pool.cc:224] Handling call
0709 13:08:09.488642 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.431431 (+942789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.654124 (+222693us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b5400231ed4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:10.684427 (+ 30303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2166469376}
I20260709 13:08:10.738680 26462 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:10.740337 26554 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:08:10.754843 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 710) took 1557 ms. Trace:
I20260709 13:08:10.495355 26495 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.683769 26459 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:10.754694 22898 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:08:10.760985 22749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.031s	sys 0.068s
W20260709 13:08:10.768293 22749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.031s	sys 0.068s
I20260709 13:08:10.768817 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:10.769848 22829 raft_consensus.cc:2368] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7.
I20260709 13:08:10.758630 23453 rpcz_store.cc:276] 0709 13:08:09.197431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.292983 (+ 95552us) service_pool.cc:224] Handling call
0709 13:08:09.309531 (+ 16548us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:09.317407 (+  7876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.317442 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:09.468108 (+150666us) log.cc:844] Serialized 52 byte log entry
0709 13:08:10.143112 (+675004us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:10.754305 (+611193us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.754564 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.754630 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.754664 (+    34us) raft_consensus.cc:1685] finished
0709 13:08:10.754674 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.754794 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15082,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":15202,"thread_start_us":1209779,"threads_started":2,"wal-append.queue_time_us":923843}
I20260709 13:08:10.760563 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:10.769166 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2632) took 1360 ms. Trace:
I20260709 13:08:10.640118 26361 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.775975 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:10.776945 22822 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:08:10.777565 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1721) took 1001 ms. Trace:
I20260709 13:08:10.777930 22822 rpcz_store.cc:276] 0709 13:08:09.776630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.755685 (+979055us) service_pool.cc:224] Handling call
0709 13:08:10.777496 (+ 21811us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.654269 26447 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:10.792830 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1720) took 1016 ms. Trace:
W20260709 13:08:10.767280 26338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.007s	sys 0.037s
I20260709 13:08:10.795668 22829 rpcz_store.cc:276] 0709 13:08:09.776319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.696742 (+920423us) service_pool.cc:224] Handling call
0709 13:08:10.792754 (+ 96012us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:10.768309 26545 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 13:08:10.690845 26448 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:10.857340 22823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 645) took 3327 ms (client timeout 2236 ms). Trace:
W20260709 13:08:10.865305 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.832971 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:10.951452 22823 rpcz_store.cc:269] 0709 13:08:07.549783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.574980 (+ 25197us) service_pool.cc:224] Handling call
0709 13:08:10.856974 (+3281994us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54002c8b54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:10.857192 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480951168}
I20260709 13:08:10.866818 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 715) took 1380 ms. Trace:
I20260709 13:08:10.953256 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:10.953349 23445 rpcz_store.cc:276] 0709 13:08:09.486280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.276314 (+790034us) service_pool.cc:224] Handling call
0709 13:08:10.276639 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.366968 (+ 90329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.866548 (+499580us) spinlock_profiling.cc:243] Waited 88.9 ms on lock 0x7b5400554454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:10.866752 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204400384}
W20260709 13:08:10.853754 23419 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.273s	user 0.021s	sys 0.033s
I20260709 13:08:10.861424 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1138) took 1333 ms. Trace:
W20260709 13:08:10.955595 23419 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.375s	user 0.021s	sys 0.041s
I20260709 13:08:10.830344 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:10.955817 23448 rpcz_store.cc:276] 0709 13:08:09.528157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.890417 (+362260us) service_pool.cc:224] Handling call
0709 13:08:10.861235 (+970818us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b5400672fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:10.861365 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66026368}
I20260709 13:08:10.829906 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1711) took 1566 ms. Trace:
W20260709 13:08:10.957101 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 1.37677e+06us
I20260709 13:08:10.959307 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 993) took 1652 ms. Trace:
I20260709 13:08:10.961831 22826 rpcz_store.cc:276] 0709 13:08:09.263658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.551485 (+287827us) service_pool.cc:224] Handling call
0709 13:08:09.566831 (+ 15346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.577383 (+ 10552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.577410 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.586013 (+  8603us) log.cc:844] Serialized 26 byte log entry
0709 13:08:10.183632 (+597619us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:10.183688 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.183782 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.779058 (+595276us) raft_consensus.cc:1685] finished
0709 13:08:10.779086 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.829820 (+ 50734us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13628,"raft.queue_time_us":13404,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":5553,"thread_start_us":597252,"threads_started":1,"wal-append.queue_time_us":750383}
I20260709 13:08:10.959307 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:10.830684 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:10.967257 22823 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:10.965197 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:10.962090 22756 rpcz_store.cc:276] 0709 13:08:09.307024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.307159 (+   135us) service_pool.cc:224] Handling call
0709 13:08:09.307540 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.307789 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.307825 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.471545 (+163720us) log.cc:844] Serialized 26 byte log entry
0709 13:08:10.186628 (+715083us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:10.186680 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.186776 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.959117 (+772341us) raft_consensus.cc:1685] finished
0709 13:08:10.959135 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.959255 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":670040,"threads_started":1,"wal-append.queue_time_us":834114}
I20260709 13:08:10.970276 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:10.970852 22823 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
W20260709 13:08:10.777724 26551 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 13:08:10.972433 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:10.973547 23448 raft_consensus.cc:2410] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:08:10.974408 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:10.975035 23448 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:10.971477 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 665) took 1359 ms. Trace:
I20260709 13:08:10.976432 22823 rpcz_store.cc:276] 0709 13:08:09.611844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.970072 (+1358228us) service_pool.cc:224] Handling call
0709 13:08:10.971429 (+  1357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:10.977205 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:10.977710 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:10.978740 22756 raft_consensus.cc:2410] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:08:10.794752 26557 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.330s	user 0.001s	sys 0.001s
I20260709 13:08:10.977717 22823 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:10.834245 26549 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:08:10.797482 26338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.007s	sys 0.037s
W20260709 13:08:10.797283 26556 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.221s	user 0.001s	sys 0.002s
I20260709 13:08:11.013765 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:11.015086 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:11.016956 23450 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:10.796485 26209 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:10.860260 26315 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:10.831341 26271 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:10.858572 26458 raft_consensus.cc:3055] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:11.020614 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:10.969419 26456 consensus_meta.cc:287] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.201s	user 0.009s	sys 0.003s
I20260709 13:08:11.037159 26456 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.040154 26456 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:11.042809 26495 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:11.014124 26470 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:11.043511 26496 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 2.207s	user 0.007s	sys 0.023s
I20260709 13:08:11.044461 22830 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
W20260709 13:08:11.046064 22830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1696) took 1787 ms (client timeout 2040 ms). Trace:
W20260709 13:08:11.046439 22830 rpcz_store.cc:269] 0709 13:08:09.259126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.259242 (+   116us) service_pool.cc:224] Handling call
0709 13:08:11.045891 (+1786649us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400432454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:11.046005 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4036206464}
I20260709 13:08:11.046304 26271 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.047644 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:11.048676 22830 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:08:11.049655 26349 consensus_meta.cc:287] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.001s
W20260709 13:08:11.050187 26452 consensus_meta.cc:287] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.212s	user 0.010s	sys 0.001s
I20260709 13:08:11.050704 26452 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:11.052539 26452 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:10.833938 26250 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:11.053689 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.054607 22754 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:08:11.054667 26505 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.001s
I20260709 13:08:11.023463 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:10.842756 26447 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:11.056413 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:11.059374 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.059733 22900 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:11.060164 22747 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:10.834895 26361 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:10.767659 26347 consensus_queue.cc:1059] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.218s
I20260709 13:08:11.025113 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:11.078148 23453 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:11.079442 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:11.080542 23445 raft_consensus.cc:2410] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:08:11.024055 26454 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:10.865744 26477 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:11.094305 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:10.865934 26244 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:11.049223 26271 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:11.101480 26477 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.102399 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 655) took 2188 ms. Trace:
I20260709 13:08:11.102748 22827 rpcz_store.cc:276] 0709 13:08:08.914011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.914127 (+   116us) service_pool.cc:224] Handling call
0709 13:08:08.933552 (+ 19425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.048775 (+115223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.048805 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:09.048852 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:09.048880 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.102183 (+2053303us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54001f9fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:11.102225 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.102328 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264558464}
I20260709 13:08:11.103785 26477 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:11.103953 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:11.027271 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:11.029381 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.028013 26458 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:11.027258 26448 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:11.112089 23449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1078) took 3499 ms (client timeout 1979 ms). Trace:
W20260709 13:08:11.112835 23449 rpcz_store.cc:269] 0709 13:08:07.609326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.717521 (+108195us) service_pool.cc:224] Handling call
0709 13:08:11.108417 (+3390896us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b5400567cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:11.108529 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1336160640}
I20260709 13:08:11.056924 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.114385 22753 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:11.022070 22902 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:11.052615 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:11.054804 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.058494 22829 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:11.063386 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.069150 22823 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:11.119203 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2415) took 1565 ms. Trace:
I20260709 13:08:11.119556 22823 rpcz_store.cc:276] 0709 13:08:09.553991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.977016 (+1423025us) service_pool.cc:224] Handling call
0709 13:08:11.119141 (+142125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.120551 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:11.070592 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.071700 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1712) took 1801 ms. Trace:
I20260709 13:08:11.121635 22754 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:10.775825 26459 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.122849 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.123728 22754 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:11.121150 22823 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:10.959775 26543 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 13:08:11.090777 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1590) took 1837 ms. Trace:
I20260709 13:08:11.026053 26209 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:11.092504 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:11.092865 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 713) took 1846 ms. Trace:
I20260709 13:08:11.138552 22898 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:11.003486 26357 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:11.112056 26271 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:11.059120 26443 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.099268 26244 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:11.100425 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.143743 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2428) took 1465 ms. Trace:
I20260709 13:08:11.143894 22752 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:11.129884 22905 rpcz_store.cc:276] 0709 13:08:09.253181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.341256 (+ 88075us) service_pool.cc:224] Handling call
0709 13:08:09.341548 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.341938 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:09.341976 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:09.471113 (+129137us) log.cc:844] Serialized 26 byte log entry
0709 13:08:10.144212 (+673099us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:10.518279 (+374067us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.865506 (+347227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.865583 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.090600 (+225017us) raft_consensus.cc:1685] finished
0709 13:08:11.090618 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.090727 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":346976,"prepare.queue_time_us":407276,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":50,"raft.queue_time_us":354462,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":13274,"spinlock_wait_cycles":896896,"thread_start_us":980338,"threads_started":2,"wal-append.queue_time_us":913896}
I20260709 13:08:11.144219 22829 rpcz_store.cc:276] 0709 13:08:09.678753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.971091 (+1292338us) service_pool.cc:224] Handling call
0709 13:08:11.143685 (+172594us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.144874 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.145591 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:11.145922 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.025625 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:11.107769 23447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 704) took 1937 ms (client timeout 2224 ms). Trace:
I20260709 13:08:11.104564 22827 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:11.112056 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:11.147066 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.147569 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 666) took 1530 ms. Trace:
I20260709 13:08:11.147890 22827 rpcz_store.cc:276] 0709 13:08:09.617173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.103723 (+1486550us) service_pool.cc:224] Handling call
0709 13:08:11.147518 (+ 43795us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.148005 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.105494 22826 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:08:11.146807 23447 rpcz_store.cc:269] 0709 13:08:09.170009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.170134 (+   125us) service_pool.cc:224] Handling call
0709 13:08:11.107547 (+1937413us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54000b2054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:11.107701 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":873958656}
W20260709 13:08:11.149879 22828 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.089s	sys 0.085s
W20260709 13:08:11.150230 22828 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.089s	sys 0.085s
I20260709 13:08:11.150974 22831 raft_consensus.cc:606] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:11.151333 22831 raft_consensus.cc:614] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:11.135352 22756 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:11.118659 22755 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:11.134683 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.121579 22824 rpcz_store.cc:276] 0709 13:08:09.270498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.551874 (+281376us) service_pool.cc:224] Handling call
0709 13:08:09.552117 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.552283 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:10.590004 (+1037721us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.071322 (+481318us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.071384 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:11.071440 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.071518 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.071640 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38124,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":2244,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":3523,"thread_start_us":1494972,"threads_started":2}
I20260709 13:08:11.138365 23452 rpcz_store.cc:276] 0709 13:08:09.246819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.915370 (+668551us) service_pool.cc:224] Handling call
0709 13:08:09.916845 (+  1475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.917060 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:10.636756 (+719696us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.072647 (+435891us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.073362 (+   715us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.092564 (+ 19202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.092607 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:11.092654 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.092712 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.092810 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10605,"prepare.queue_time_us":866792,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":15288,"raft.queue_time_us":32782,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":9195,"spinlock_wait_cycles":51684480,"thread_start_us":1146448,"threads_started":2}
I20260709 13:08:11.140619 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:11.142570 23419 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:11.146222 22829 raft_consensus.cc:2376] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10. Re-sending same reply.
I20260709 13:08:11.117874 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.082597 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:11.162689 22830 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:11.162897 22748 raft_consensus.cc:2410] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:08:11.137688 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.173768 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:11.175711 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:11.182780 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.184576 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.185842 22748 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:08:11.139590 26357 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.146078 26361 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.187047 22829 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:08:11.329886 26495 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:11.235442 26365 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:11.140311 26209 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:11.242424 26459 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:11.268460 26480 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.433s	user 0.014s	sys 0.008s
I20260709 13:08:11.296557 22823 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
W20260709 13:08:11.323704 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.271100 22900 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
W20260709 13:08:11.191628 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.326462 23447 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:08:11.236130 22826 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:11.211489 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:11.205771 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:11.326565 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:11.323369 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:11.333240 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1716) took 2015 ms. Trace:
I20260709 13:08:11.340044 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 701) took 3990 ms. Trace:
I20260709 13:08:11.340742 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1714) took 2050 ms. Trace:
I20260709 13:08:11.344738 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1080) took 1798 ms. Trace:
I20260709 13:08:11.344419 22899 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:11.344633 23451 rpcz_store.cc:276] 0709 13:08:07.349254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.395456 (+ 46202us) service_pool.cc:224] Handling call
0709 13:08:08.437601 (+1042145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.926770 (+489169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.339837 (+2413067us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000b2cd0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:11.339979 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2367280896}
I20260709 13:08:11.332752 26361 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:11.144680 26271 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.345615 23445 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:11.689172 22896 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:08:11.343616 22829 rpcz_store.cc:276] 0709 13:08:09.317840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.185062 (+1867222us) service_pool.cc:224] Handling call
0709 13:08:11.186689 (+  1627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.332985 (+146296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.333163 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.419247 23453 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:11.690671 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1141) took 1092 ms. Trace:
I20260709 13:08:11.690972 23453 rpcz_store.cc:276] 0709 13:08:10.628272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.987756 (+359484us) service_pool.cc:224] Handling call
0709 13:08:11.690623 (+702867us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.345383 23447 rpcz_store.cc:276] 0709 13:08:09.546808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.183927 (+1637119us) service_pool.cc:224] Handling call
0709 13:08:11.326249 (+142322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.344508 (+ 18259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.344690 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.396555 22822 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:11.212484 22827 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:11.342715 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:11.464449 26209 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.698176 22829 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:08:11.449532 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 273877us
I20260709 13:08:11.344990 22826 rpcz_store.cc:276] 0709 13:08:09.290080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.184415 (+1894335us) service_pool.cc:224] Handling call
0709 13:08:11.235934 (+ 51519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.340471 (+104537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.340682 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.349355 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1715) took 2057 ms. Trace:
I20260709 13:08:11.335556 23448 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:08:11.723217 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:11.336439 26480 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.501s	user 0.017s	sys 0.009s
I20260709 13:08:11.275903 22830 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:11.686573 22905 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:11.723367 22824 rpcz_store.cc:276] 0709 13:08:09.292370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.182492 (+1890122us) service_pool.cc:224] Handling call
0709 13:08:11.182804 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.183011 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:11.208839 (+ 25828us) log.cc:844] Serialized 14 byte log entry
0709 13:08:11.348959 (+140120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.349023 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:11.349078 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.349155 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.349279 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6074,"prepare.queue_time_us":28137,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":6728,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":37567,"thread_start_us":133639,"threads_started":1,"wal-append.queue_time_us":133933}
I20260709 13:08:11.687270 26271 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:11.333091 26495 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.687659 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:11.393342 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:11.512921 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.726930 22900 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:11.727036 22902 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:11.188388 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.727229 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2410) took 2174 ms. Trace:
I20260709 13:08:11.727912 22756 raft_consensus.cc:2410] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:08:11.706018 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:11.696126 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 182152us
W20260709 13:08:11.695559 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 352909us
I20260709 13:08:11.697868 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:11.478104 26567 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:08:11.732262 22753 raft_consensus.cc:2393] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:08:11.407912 26574 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.081s	user 0.000s	sys 0.004s
I20260709 13:08:11.151786 26468 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:11.682435 26244 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:11.588500 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:11.724515 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 717) took 2008 ms. Trace:
I20260709 13:08:11.744553 23448 rpcz_store.cc:276] 0709 13:08:09.716191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.334925 (+1618734us) service_pool.cc:224] Handling call
0709 13:08:11.335288 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.724251 (+388963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.724449 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.348685 26576 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:11.189095 26445 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:11.184648 26250 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:08:11.724761 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 404151us
I20260709 13:08:11.725190 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 671) took 1025 ms. Trace:
I20260709 13:08:11.729851 26495 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:11.748278 22830 rpcz_store.cc:276] 0709 13:08:10.699912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.049272 (+349360us) service_pool.cc:224] Handling call
0709 13:08:11.725137 (+675865us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.748890 23448 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
W20260709 13:08:11.215137 26543 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:11.625234 26357 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:11.598436 26578 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.265s	user 0.000s	sys 0.004s
I20260709 13:08:11.693109 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.486817 26458 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:11.413089 26569 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:08:11.727574 22829 rpcz_store.cc:276] 0709 13:08:09.552431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.697716 (+2145285us) service_pool.cc:224] Handling call
0709 13:08:11.697980 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.726983 (+ 29003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.727179 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.733768 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:11.770366 23445 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:11.705477 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1713) took 2421 ms. Trace:
I20260709 13:08:11.771654 22827 rpcz_store.cc:276] 0709 13:08:09.284423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.177984 (+1893561us) service_pool.cc:224] Handling call
0709 13:08:11.212241 (+ 34257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.705111 (+492870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.705397 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.483258 26522 raft_consensus.cc:3055] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:11.749763 26250 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:11.151750 26443 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:11.731221 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:11.778895 23447 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:11.733901 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:11.739012 22824 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:11.591037 26579 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:11.733947 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:11.781028 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47136.7us
I20260709 13:08:11.782979 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:11.749445 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1143) took 1109 ms. Trace:
I20260709 13:08:11.751530 22753 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
W20260709 13:08:11.477240 26565 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:08:11.747785 26244 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.774569 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.774194 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2417) took 2216 ms. Trace:
I20260709 13:08:11.788954 22829 rpcz_store.cc:276] 0709 13:08:09.558242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.773351 (+2215109us) service_pool.cc:224] Handling call
0709 13:08:11.773696 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.773864 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.773891 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.773937 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:11.773973 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.774054 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.774145 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.746969 26445 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:11.448055 26477 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:11.777756 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 54614us
I20260709 13:08:11.775367 26522 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:11.770390 26357 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.784041 23448 rpcz_store.cc:276] 0709 13:08:10.639778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.748497 (+1108719us) service_pool.cc:224] Handling call
0709 13:08:11.748743 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.749232 (+   489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.749400 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.753190 26576 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:11.749835 26431 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:11.790437 22829 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:08:11.790751 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:11.791985 22755 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 6: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:11.794241 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.785419 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2413) took 2232 ms. Trace:
I20260709 13:08:11.746982 26468 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.807725 26468 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:11.808434 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1084) took 2260 ms. Trace:
I20260709 13:08:11.808771 23449 rpcz_store.cc:276] 0709 13:08:09.547874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.183676 (+1635802us) service_pool.cc:224] Handling call
0709 13:08:11.184036 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.656101 (+472065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.656144 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:11.656713 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.734481 (+ 77768us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:11.734529 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.734619 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.808235 (+ 73616us) raft_consensus.cc:1685] finished
0709 13:08:11.808253 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.808381 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3010,"raft.queue_time_us":145716,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":403813,"threads_started":2,"wal-append.queue_time_us":135126}
I20260709 13:08:11.809995 26576 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.812206 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:11.798691 26477 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.813452 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:11.815292 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57774 (request call id 19) took 2648 ms. Trace:
I20260709 13:08:11.815527 26477 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:11.815778 22831 rpcz_store.cc:276] 0709 13:08:09.190581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.233883 (+ 43302us) service_pool.cc:224] Handling call
0709 13:08:11.815015 (+2581132us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b540043a8d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74df5853 00007fcb79dc360a 00007fcb733497d4 00007fcb73349762 00007fcb73349691 00007fcb7334960c 00007fcb733488c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:11.815165 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4280207744}
I20260709 13:08:11.778442 26250 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:11.792760 26584 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:11.817170 26584 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:11.817677 26584 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:11.818114 26209 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:11.819134 26361 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:11.819558 22896 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: [7.4-9.5]
W20260709 13:08:11.820145 26571 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.137s	user 0.002s	sys 0.000s
I20260709 13:08:11.820524 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:11.821609 26357 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.822820 22898 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:08:11.800235 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2418) took 2228 ms. Trace:
I20260709 13:08:11.823792 22829 rpcz_store.cc:276] 0709 13:08:09.571353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.789964 (+2218611us) service_pool.cc:224] Handling call
0709 13:08:11.790273 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.799468 (+  9195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.800181 (+   713us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.780408 26509 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:11.800587 22824 rpcz_store.cc:276] 0709 13:08:09.553468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.726549 (+2173081us) service_pool.cc:224] Handling call
0709 13:08:11.735800 (+  9251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.785003 (+ 49203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.785300 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.803730 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 647) took 4200 ms. Trace:
I20260709 13:08:11.803488 26522 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.810433 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:11.830327 23449 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:11.816150 26477 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.797158 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:11.825438 22828 rpcz_store.cc:276] 0709 13:08:07.603607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.604169 (+   562us) service_pool.cc:224] Handling call
0709 13:08:07.604529 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:10.429139 (+2824610us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b180039ee60. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb749a34bc 00007fcb71cf148b 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0b455 00007fcb74df3060 00007fcb74df3bfb 00007fcb74dfe64b
0709 13:08:10.429264 (+   125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:10.429289 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:10.438086 (+  8797us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.150658 (+712572us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:11.150715 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.150950 (+   235us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.803374 (+652424us) raft_consensus.cc:1685] finished
0709 13:08:11.803398 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.803645 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23639,"raft.queue_time_us":112772,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":27637,"spinlock_wait_cycles":23946496,"thread_start_us":801989,"threads_started":2,"wal-append.queue_time_us":830509}
I20260709 13:08:11.831538 23448 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:11.802055 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:11.832944 23453 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:11.793516 26445 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:11.842231 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:11.846843 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.847054 22824 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:08:11.793735 26328 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:11.846026 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.847681 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2421) took 2263 ms. Trace:
I20260709 13:08:11.843030 22828 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:11.795152 26244 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.788223 26579 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:08:11.865582 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.936450 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.949705 22754 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:08:11.881009 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.848466 22824 rpcz_store.cc:276] 0709 13:08:09.584767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.827106 (+2242339us) service_pool.cc:224] Handling call
0709 13:08:11.841677 (+ 14571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.847420 (+  5743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.847619 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.951170 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.950237 22831 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:11.951721 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.926410 23449 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:11.941447 22829 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:08:11.952178 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 667) took 2332 ms. Trace:
I20260709 13:08:11.952724 22831 rpcz_store.cc:276] 0709 13:08:09.619522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.900203 (+2280681us) service_pool.cc:224] Handling call
0709 13:08:11.950069 (+ 49866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.951963 (+  1894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.952132 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.953058 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:11.877508 22898 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:11.953801 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:11.935645 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:11.936069 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.903591 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:11.955356 23449 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:11.926715 23448 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:08:11.952976 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.961201 22754 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:11.848912 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 663) took 2258 ms. Trace:
I20260709 13:08:11.963663 22754 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:08:11.964946 22828 rpcz_store.cc:276] 0709 13:08:09.590529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.831617 (+2241088us) service_pool.cc:224] Handling call
0709 13:08:11.831932 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.848617 (+ 16685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.848850 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.965636 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.967345 26594 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.963680 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:11.968305 26594 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.952840 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2419) took 2379 ms. Trace:
I20260709 13:08:11.969302 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2414) took 2415 ms. Trace:
I20260709 13:08:11.970638 22754 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:11.971031 22754 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:11.970677 22829 rpcz_store.cc:276] 0709 13:08:09.573590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.824484 (+2250894us) service_pool.cc:224] Handling call
0709 13:08:11.941129 (+116645us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.952636 (+ 11507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.952786 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.971828 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1145) took 1302 ms. Trace:
I20260709 13:08:11.969789 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.976266 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:11.980440 22755 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:11.981297 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:11.842010 26271 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:11.884690 26584 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.842233 26431 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:11.829649 26509 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.935590 26357 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:11.979841 22754 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:11.860028 26328 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.972421 23449 rpcz_store.cc:276] 0709 13:08:10.669256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.970799 (+1301543us) service_pool.cc:224] Handling call
0709 13:08:11.971176 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.971420 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.971456 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.971516 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:11.971562 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.971632 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.971757 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:11.987459 22755 raft_consensus.cc:2410] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:08:11.859724 26157 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.877382 26522 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:12.019037 26157 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:12.021018 26157 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:11.994385 22756 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:11.986088 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 673) took 1215 ms. Trace:
I20260709 13:08:11.971647 22830 rpcz_store.cc:276] 0709 13:08:09.553755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.748989 (+2195234us) service_pool.cc:224] Handling call
0709 13:08:11.749245 (+   256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.792638 (+ 43393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.792677 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:11.793209 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.967089 (+173880us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:11.967129 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.967196 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.968910 (+  1714us) raft_consensus.cc:1685] finished
0709 13:08:11.968928 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.969246 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48445,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":6715,"thread_start_us":128391,"threads_started":2,"wal-append.queue_time_us":123886}
I20260709 13:08:11.845531 26586 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.234s
I20260709 13:08:11.992787 26431 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:11.859437 26361 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.220s
I20260709 13:08:11.970039 26209 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.172s
I20260709 13:08:11.859131 26477 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:12.127930 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.140420 22753 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:12.204344 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:12.226822 22755 raft_consensus.cc:2410] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 13:08:12.229413 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.234071 22756 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:12.086170 22754 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:12.036633 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 60436.2us
I20260709 13:08:12.237142 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:12.238407 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:12.241518 22831 rpcz_store.cc:276] 0709 13:08:10.771042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.953734 (+1182692us) service_pool.cc:224] Handling call
0709 13:08:11.985621 (+ 31887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.985765 (+   144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.985790 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.985827 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:11.985862 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.985917 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.986026 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:12.244237 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 271770us
I20260709 13:08:12.249575 22828 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:12.250250 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:12.248721 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:12.253063 22829 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:12.253409 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:08:12.154103 26588 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:08:12.250450 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:12.259150 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.255560 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:12.264995 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2112) took 4855 ms. Trace:
I20260709 13:08:12.265558 22903 rpcz_store.cc:276] 0709 13:08:07.399087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.399213 (+   126us) service_pool.cc:224] Handling call
0709 13:08:07.399457 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.005577 (+1606120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.247561 (+3241984us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54003e9754. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:12.254578 (+  7017us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3552411392}
I20260709 13:08:12.170032 26584 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:12.267208 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:12.273006 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:12.258044 22756 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:12.274389 22756 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:12.202389 26192 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.271296 22829 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:12.079532 26328 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:12.079068 26509 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:12.280089 22828 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:12.281517 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:12.280800 26589 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:12.287173 22754 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:12.288975 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:12.288149 23447 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.001s
I20260709 13:08:12.252942 26404 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:12.079010 26271 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:12.131453 26521 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.179s
I20260709 13:08:12.295588 26477 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:12.296185 26477 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:12.078915 26590 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:12.296537 26477 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.296865 26590 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:12.158372 26503 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.109s
I20260709 13:08:12.159101 26576 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.130s
I20260709 13:08:12.259048 26591 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:12.298837 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:12.289824 26192 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:12.294595 26271 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:12.296952 26477 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:12.256685 26357 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:12.253083 26468 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:08:12.304574 26590 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:12.305563 26594 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:12.258210 26592 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:08:12.296553 26521 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:12.147712 26250 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:12.298761 26591 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:12.332908 22756 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:12.290463 23447 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
W20260709 13:08:12.340924 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.342994 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2411) took 2790 ms. Trace:
I20260709 13:08:12.343501 22826 rpcz_store.cc:276] 0709 13:08:09.552793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.723178 (+2170385us) service_pool.cc:224] Handling call
0709 13:08:11.737201 (+ 14023us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.828106 (+ 90905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.828124 (+    18us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:11.841431 (+ 13307us) log.cc:844] Serialized 26 byte log entry
0709 13:08:12.099864 (+258433us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:12.099914 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.100016 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:12.342720 (+242704us) raft_consensus.cc:1685] finished
0709 13:08:12.342747 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.342919 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":49557,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":5017,"thread_start_us":243653,"threads_started":2,"wal-append.queue_time_us":405597}
I20260709 13:08:12.339258 26594 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:08:12.335675 26600 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:08:12.360144 26495 consensus_meta.cc:287] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.589s	user 0.005s	sys 0.001s
I20260709 13:08:12.340274 26591 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:12.360600 26495 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:12.361617 26244 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:12.362818 26579 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:12.372008 23445 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:12.319193 26357 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:12.374408 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 126436us
I20260709 13:08:12.393566 23453 raft_consensus.cc:2368] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12.
I20260709 13:08:12.451327 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:12.482537 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 89161.3us
W20260709 13:08:12.517509 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.517954 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.400610 23447 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:08:12.524173 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:12.482928 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 209970us
I20260709 13:08:12.530289 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:12.511977 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:12.570958 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.422313 26593 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:08:12.535518 26608 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:08:12.392297 26600 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:12.589752 26595 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:08:12.421103 26404 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:12.644001 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 132115us
I20260709 13:08:12.650204 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2409) took 3098 ms. Trace:
W20260709 13:08:12.602934 26609 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:08:12.654023 22823 rpcz_store.cc:276] 0709 13:08:09.552181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.349884 (+1797703us) service_pool.cc:224] Handling call
0709 13:08:11.682208 (+332324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.682880 (+   672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.682909 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:11.683390 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.795353 (+111963us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:11.795394 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.795459 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:12.509496 (+714037us) raft_consensus.cc:1685] finished
0709 13:08:12.509523 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.650128 (+140605us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":78,"thread_start_us":59568,"threads_started":1,"wal-append.queue_time_us":153436}
I20260709 13:08:12.655409 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:12.536052 26614 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:12.398018 26610 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.276s
I20260709 13:08:12.627426 26593 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:12.631111 26603 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:12.643889 26431 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:12.648305 26351 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:12.742645 23416 leader_election.cc:336] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.530s (SENT)
I20260709 13:08:12.478616 26357 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:13.278326 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.743793 26603 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:13.078573 22825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 668) took 3457 ms (client timeout 1610 ms). Trace:
W20260709 13:08:13.320633 22825 rpcz_store.cc:269] 0709 13:08:09.621452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.761953 (+140501us) service_pool.cc:224] Handling call
0709 13:08:13.078014 (+3316061us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54004123d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:13.078306 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663513856}
I20260709 13:08:13.111187 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 706) took 3917 ms. Trace:
I20260709 13:08:13.322265 23450 rpcz_store.cc:276] 0709 13:08:09.193245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.193367 (+   122us) service_pool.cc:224] Handling call
0709 13:08:09.242539 (+ 49172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.021897 (+1779358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.110609 (+2088712us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b540050f854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:13.110996 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4080130816}
I20260709 13:08:13.323239 26603 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:12.741417 26614 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:13.324887 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:13.326682 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:13.093986 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:13.100225 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.055920 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.043836 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.108729 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 708) took 3913 ms. Trace:
W20260709 13:08:13.046620 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.044963 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.071117 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1006) took 2003 ms. Trace:
I20260709 13:08:10.432296 22750 rpcz_store.cc:276] 0709 13:08:09.407732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.409754 (+  2022us) service_pool.cc:224] Handling call
0709 13:08:09.410032 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.410263 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:09.540302 (+130039us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.430898 (+890596us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:10.430961 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:10.431014 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.431094 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.431224 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":187020,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":1551,"thread_start_us":1019078,"threads_started":2}
I20260709 13:08:13.292990 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1594) took 4003 ms. Trace:
I20260709 13:08:13.263654 26361 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:13.332126 26361 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:13.331977 22897 rpcz_store.cc:276] 0709 13:08:09.289411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.486036 (+196625us) service_pool.cc:224] Handling call
0709 13:08:09.491898 (+  5862us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:09.723646 (+231748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.077735 (+3354089us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54000d0554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:13.292749 (+215014us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531629952}
W20260709 13:08:13.334519 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.334127 26361 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.335775 26472 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:13.331282 23446 rpcz_store.cc:276] 0709 13:08:09.195346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.233599 (+ 38253us) service_pool.cc:224] Handling call
0709 13:08:09.235351 (+  1752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:10.697149 (+1461798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.108188 (+2411039us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54005550d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:13.108534 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3341290752}
I20260709 13:08:13.338390 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:13.339727 23446 raft_consensus.cc:2410] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:13.331559 22747 rpcz_store.cc:276] 0709 13:08:11.067995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.068142 (+   147us) service_pool.cc:224] Handling call
0709 13:08:11.068424 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:11.695660 (+627236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.695688 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:11.695735 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:11.695777 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.070487 (+1374710us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54002689d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:13.070569 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.070860 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":76274,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":27764,"spinlock_wait_cycles":1267089792}
W20260709 13:08:13.294596 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 551792us
I20260709 13:08:13.342403 23446 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:08:13.343225 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1091) took 1547 ms. Trace:
I20260709 13:08:13.343611 23446 rpcz_store.cc:276] 0709 13:08:11.795609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.341342 (+1545733us) service_pool.cc:224] Handling call
0709 13:08:13.341923 (+   581us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:13.342867 (+   944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.343151 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:13.344584 26390 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:13.345090 26390 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:12.785892 26404 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.346639 26390 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.347468 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:13.348014 26390 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:13.348246 22750 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:10.772310 22749 rpcz_store.cc:276] 0709 13:08:09.408633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.516564 (+107931us) service_pool.cc:224] Handling call
0709 13:08:09.555069 (+ 38505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.555277 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:10.237090 (+681813us) log.cc:844] Serialized 14 byte log entry
0709 13:08:10.768762 (+531672us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:10.768833 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:10.768889 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:10.768969 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.769101 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53993,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":3118,"thread_start_us":1192335,"threads_started":2}
I20260709 13:08:13.352298 23447 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:13.353112 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1094) took 1528 ms. Trace:
W20260709 13:08:13.300450 23453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1093) took 1500 ms (client timeout 1979 ms). Trace:
W20260709 13:08:13.300747 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 770367us
W20260709 13:08:12.811523 26445 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.922s	user 0.010s	sys 0.000s
I20260709 13:08:12.779234 26495 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.064046 26503 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:12.912029 26328 consensus_meta.cc:287] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.619s	user 0.007s	sys 0.002s
W20260709 13:08:13.302582 26477 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.986s	user 0.000s	sys 0.015s
I20260709 13:08:13.303552 26192 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.069304 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1598) took 1256 ms. Trace:
W20260709 13:08:12.811671 26509 consensus_meta.cc:287] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.000s
W20260709 13:08:12.933594 26482 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.027s	sys 0.064s
I20260709 13:08:13.304207 26250 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:13.367588 22905 rpcz_store.cc:276] 0709 13:08:11.812615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.813211 (+   596us) service_pool.cc:224] Handling call
0709 13:08:13.068726 (+1255515us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54000d0f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:13.069009 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535839104}
W20260709 13:08:13.306238 26510 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:08:13.308313 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2429) took 3489 ms. Trace:
I20260709 13:08:13.059794 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1086) took 1265 ms. Trace:
I20260709 13:08:13.080157 22827 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:08:13.061067 22822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 674) took 2138 ms (client timeout 1663 ms). Trace:
I20260709 13:08:13.365074 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1092) took 1568 ms. Trace:
I20260709 13:08:13.312350 26321 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.312531 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2631) took 3904 ms. Trace:
I20260709 13:08:12.851204 26591 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:13.042673 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 365158us
W20260709 13:08:13.040937 22719 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.002s	sys 0.006s
I20260709 13:08:12.711592 26244 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:08:13.313400 23444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 695) took 6007 ms (client timeout 2084 ms). Trace:
I20260709 13:08:13.368013 26503 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.368371 26328 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.316859 26580 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:12.753129 26593 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.319084 26459 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.107395 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1144) took 2464 ms. Trace:
I20260709 13:08:13.384840 23445 rpcz_store.cc:276] 0709 13:08:10.642973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.771073 (+1128100us) service_pool.cc:224] Handling call
0709 13:08:11.771372 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.373990 (+602618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.106829 (+732839us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b54004f3154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:13.107198 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1366863104}
I20260709 13:08:13.385072 26459 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.386694 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2416) took 3832 ms. Trace:
I20260709 13:08:13.386996 23445 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:13.387091 22827 rpcz_store.cc:276] 0709 13:08:09.554268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.772470 (+2218202us) service_pool.cc:224] Handling call
0709 13:08:13.079321 (+1306851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:13.372326 (+293005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.386488 (+ 14162us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54004123d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:13.386620 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2985667456}
I20260709 13:08:13.387717 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1095) took 1557 ms. Trace:
I20260709 13:08:13.388173 23445 rpcz_store.cc:276] 0709 13:08:11.830742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.386179 (+1555437us) service_pool.cc:224] Handling call
0709 13:08:13.386562 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:13.387498 (+   936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.387670 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:13.388585 26482 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.029s	sys 0.096s
I20260709 13:08:13.390705 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:13.390981 26620 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.077s	user 0.000s	sys 0.003s
I20260709 13:08:13.391914 26446 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:13.392423 26446 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:13.392882 26446 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:13.392907 26243 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:13.393433 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:13.394739 26243 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:13.395195 26243 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:13.396659 26243 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.397774 26509 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.399235 26424 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:13.399425 26611 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.399993 26424 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.400434 26509 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:12.754899 26600 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.407034 26576 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:13.407742 26576 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.408476 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:13.410279 26576 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:13.353526 23447 rpcz_store.cc:276] 0709 13:08:11.825005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.351637 (+1526632us) service_pool.cc:224] Handling call
0709 13:08:13.352016 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:13.352894 (+   878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.353065 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:13.357965 26445 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.412078 26593 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:13.412876 26445 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:13.359159 26448 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:13.413693 26448 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.415993 26448 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.419600 26374 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.420210 26374 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.421656 26351 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.422334 26351 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.424803 26351 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.360333 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:13.425813 22747 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:13.369555 22824 rpcz_store.cc:276] 0709 13:08:09.818581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.951164 (+2132583us) service_pool.cc:224] Handling call
0709 13:08:11.951435 (+   271us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:11.970348 (+ 18913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.970384 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:11.993883 (+ 23499us) log.cc:844] Serialized 52 byte log entry
0709 13:08:12.290153 (+296270us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:12.290205 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.290265 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.046984 (+756719us) raft_consensus.cc:1685] finished
0709 13:08:13.047085 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.308090 (+261005us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83831,"raft.queue_time_us":19084,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":10897,"thread_start_us":119198,"threads_started":1,"wal-append.queue_time_us":203636}
W20260709 13:08:13.356341 23453 rpcz_store.cc:269] 0709 13:08:11.799608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.903311 (+103703us) service_pool.cc:224] Handling call
0709 13:08:13.097411 (+1194100us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5400567cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:13.300191 (+202780us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007635328}
I20260709 13:08:13.427232 22753 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:08:13.427615 22753 pending_rounds.cc:85] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 6
I20260709 13:08:13.427829 22753 pending_rounds.cc:107] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:08:13.428097 22753 raft_consensus.cc:2889] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:13.428181 23453 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:08:13.365532 26603 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:13.429819 26603 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.430441 26374 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.362391 26443 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.362562 26458 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:13.431383 26324 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:13.432008 26324 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:12.754206 26606 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 13:08:13.370607 23446 rpcz_store.cc:276] 0709 13:08:11.795893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.344344 (+1548451us) service_pool.cc:224] Handling call
0709 13:08:13.363803 (+ 19459us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:13.335772 26361 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:13.370633 22751 rpcz_store.cc:276] 0709 13:08:09.408336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.410487 (+  2151us) service_pool.cc:224] Handling call
0709 13:08:09.410778 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:09.869271 (+458493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.077187 (+3207916us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b540016ab54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:13.312353 (+235166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052842240}
I20260709 13:08:13.372717 26321 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:13.373091 23444 rpcz_store.cc:269] 0709 13:08:07.306177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.306288 (+   111us) service_pool.cc:224] Handling call
0709 13:08:13.109959 (+5803671us) spinlock_profiling.cc:243] Waited 2.24 s on lock 0x7b5400671454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:13.313108 (+203149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5158634752}
I20260709 13:08:12.845327 26478 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:08:13.369992 22822 rpcz_store.cc:269] 0709 13:08:10.922808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.956943 (+ 34135us) service_pool.cc:224] Handling call
0709 13:08:13.060446 (+2103503us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400381dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:13.060714 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861963136}
W20260709 13:08:13.317507 26575 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.967s	user 0.011s	sys 0.029s
I20260709 13:08:13.455881 26448 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:13.377063 26503 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.372792 26244 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:13.374926 26580 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:13.377662 26328 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:13.473698 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2132) took 2805 ms. Trace:
I20260709 13:08:13.382825 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:13.474843 26321 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.383625 26209 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:13.474102 22904 rpcz_store.cc:276] 0709 13:08:10.668105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.693459 (+ 25354us) service_pool.cc:224] Handling call
0709 13:08:10.693871 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:10.867113 (+173242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:10.867143 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:10.867686 (+   543us) log.cc:844] Serialized 52 byte log entry
0709 13:08:11.323538 (+455852us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:11.323749 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.323840 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.294231 (+1970391us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:13.459893 (+165662us) raft_consensus.cc:1685] finished
0709 13:08:13.459911 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.473535 (+ 13624us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2643013,"raft.queue_time_us":172876,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":2470441,"spinlock_wait_cycles":10211072,"thread_start_us":428028,"threads_started":1,"wal-append.queue_time_us":459720}
I20260709 13:08:13.475713 26209 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:13.477101 26209 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.478385 26209 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:13.479800 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1085) took 1685 ms. Trace:
I20260709 13:08:13.369656 23449 rpcz_store.cc:276] 0709 13:08:11.794378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.015279 (+220901us) service_pool.cc:224] Handling call
0709 13:08:12.249423 (+234144us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:12.287719 (+ 38296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:12.287763 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:12.291985 (+  4222us) log.cc:844] Serialized 26 byte log entry
0709 13:08:12.458288 (+166303us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:12.458340 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.458416 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.059015 (+600599us) raft_consensus.cc:1685] finished
0709 13:08:13.059068 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.059480 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37908,"raft.queue_time_us":43605,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":5235,"thread_start_us":139885,"threads_started":1,"wal-append.queue_time_us":211763}
I20260709 13:08:13.398221 26243 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:13.480319 23448 rpcz_store.cc:276] 0709 13:08:11.794119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.972852 (+178733us) service_pool.cc:224] Handling call
0709 13:08:11.973129 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:12.037020 (+ 63891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:12.037063 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:12.037600 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:08:12.302663 (+265063us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:12.417570 (+114907us) log.cc:844] Serialized 14 byte log entry
0709 13:08:12.456082 (+ 38512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.456186 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.479322 (+1023136us) raft_consensus.cc:1685] finished
0709 13:08:13.479363 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.479594 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22066,"prepare.queue_time_us":451258,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":34713,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":1393,"thread_start_us":428398,"threads_started":3,"wal-append.queue_time_us":355034}
I20260709 13:08:13.400215 26611 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.403332 26424 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:13.391158 26477 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:12.754387 26431 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:13.405709 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.405890 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:13.406014 26594 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.380090 26591 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:13.492761 26611 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:13.495401 22755 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:08:13.495796 22755 pending_rounds.cc:85] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 4
I20260709 13:08:13.496138 22755 pending_rounds.cc:107] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:08:13.496402 22755 raft_consensus.cc:2889] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:13.495847 26431 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.387665 26459 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:13.325990 26614 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.446646 26324 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:13.456666 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 63411.9us
W20260709 13:08:13.374208 22719 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.007s	sys 0.015s
I20260709 13:08:13.460512 22750 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:13.460100 26443 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:13.462847 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:13.432018 26603 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:13.463549 22871 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:13.470605 26478 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:13.471819 26244 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:13.472885 26580 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.475477 26448 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.401778 26599 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.490955 22754 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:13.490597 26594 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.512197 22754 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:13.500300 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:13.505713 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:13.509711 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 47122us
I20260709 13:08:13.513928 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:13.518731 26459 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.520183 22751 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:13.520771 26594 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:13.520273 26459 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:13.516600 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 1.00788e+06us
I20260709 13:08:13.512133 26611 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.528172 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:13.517660 26443 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:13.527434 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 64160.4us
I20260709 13:08:13.530539 22747 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:13.533664 22752 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:13.523928 26602 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:13.527740 26580 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:13.539884 26602 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.540751 26497 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.595712 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:13.594710 23450 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:13.589389 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:13.601593 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 181566us
I20260709 13:08:13.589860 26602 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:13.530176 26599 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.584909 26446 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:13.609493 22797 connection.cc:256] RPC call timeout handler was delayed by -2.3200795230001177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 13:08:13.621899 22797 connection.cc:256] RPC call timeout handler was delayed by -2.2846523590000061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 13:08:13.622323 22797 connection.cc:256] RPC call timeout handler was delayed by -2.2570795230001295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 13:08:13.622692 22797 connection.cc:256] RPC call timeout handler was delayed by -2.2030795230000422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
W20260709 13:08:13.623056 22797 connection.cc:256] RPC call timeout handler was delayed by -2.1814795230001209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
W20260709 13:08:13.623466 22797 connection.cc:256] RPC call timeout handler was delayed by -2.0581795230000353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
W20260709 13:08:13.623868 22797 connection.cc:256] RPC call timeout handler was delayed by -2.0356795230001126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 13:08:13.624228 22797 connection.cc:256] RPC call timeout handler was delayed by -1.8771651630000861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
W20260709 13:08:13.624580 22797 connection.cc:256] RPC call timeout handler was delayed by -1.876952359000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
W20260709 13:08:13.624940 22797 connection.cc:256] RPC call timeout handler was delayed by -1.8754795230000809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2351s for a response
W20260709 13:08:13.625295 22797 connection.cc:256] RPC call timeout handler was delayed by -1.8744651630000817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:08:13.626597 22797 connection.cc:256] RPC call timeout handler was delayed by -1.8520795230000431s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2377s for a response
W20260709 13:08:13.626991 22797 connection.cc:256] RPC call timeout handler was delayed by -1.8427523590000874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
W20260709 13:08:13.627478 22797 connection.cc:256] RPC call timeout handler was delayed by -1.8178795230001015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24150000000000002s for a response
W20260709 13:08:13.627874 22797 connection.cc:256] RPC call timeout handler was delayed by -1.8015651630000775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
W20260709 13:08:13.628237 22797 connection.cc:256] RPC call timeout handler was delayed by -1.792565163000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260709 13:08:13.628594 22797 connection.cc:256] RPC call timeout handler was delayed by -1.4217019019999952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23020000000000002s for a response
I20260709 13:08:13.607771 26594 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:13.600622 26611 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.631783 26448 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:13.637261 22755 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:13.641116 26461 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.641646 26461 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.595124 26591 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:08:13.640637 22747 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:13.629905 22754 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.645179 26594 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:13.681917 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:13.660550 26579 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:08:13.644953 26458 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:13.654646 26478 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.675349 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:13.684257 23447 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:13.680346 26443 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.686106 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:13.660194 26612 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:08:13.687196 23453 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:13.681361 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:13.687736 26612 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:13.688156 26612 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:13.671617 26244 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.652118 23452 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:08:13.678164 26591 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:13.689723 23452 pending_rounds.cc:85] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 10
I20260709 13:08:13.689848 26591 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:13.655524 26446 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:13.690419 26244 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:13.659739 23450 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:13.684969 26458 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:13.655263 26459 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.658540 26630 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:08:13.646149 26477 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:13.654688 26599 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:13.692168 26458 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.692418 26630 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:13.693259 22754 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:13.693459 26630 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:13.694494 26630 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:13.685408 26478 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:13.638700 26497 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.690016 23452 pending_rounds.cc:107] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:08:13.658138 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:13.703409 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45351.3us
I20260709 13:08:13.704268 23452 raft_consensus.cc:2889] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:13.660130 22871 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:13.706228 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.706538 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.706993 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.707449 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.707911 22750 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:13.712441 26594 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:13.716081 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.717657 22751 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
W20260709 13:08:13.719009 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 73819.1us
I20260709 13:08:13.719607 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:13.720678 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.721101 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.721493 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.721961 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.724928 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:13.730407 26360 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.731009 26360 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.732019 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1142) took 3014 ms. Trace:
I20260709 13:08:13.682709 26461 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.733434 22797 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:13.734879 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:13.680197 26611 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:13.745412 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.691007 26591 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.715962 26472 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:13.747713 26244 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:13.754473 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:13.756738 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 96830.3us
I20260709 13:08:13.758085 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:13.760499 26579 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:08:13.761399 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.761328 26579 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:13.669075 26602 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.762013 22797 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:13.764437 26602 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:13.766049 26579 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:08:13.771860 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 55918.8us
I20260709 13:08:13.772987 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:13.773337 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:13.765452 26446 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:13.773389 26599 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:08:13.776011 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:13.785514 22754 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:13.776522 26630 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:13.689220 26612 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.776731 26244 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:13.798712 26612 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:13.777483 26243 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:13.799028 26497 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:13.769418 26610 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:13.761245 26459 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:13.810254 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:13.779680 26600 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:13.811353 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:13.784385 23450 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:13.811731 26610 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:13.784685 23447 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:13.812713 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:13.785605 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.813429 22899 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:13.812868 26610 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.732401 23451 rpcz_store.cc:276] 0709 13:08:10.632120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.345963 (+713843us) service_pool.cc:224] Handling call
0709 13:08:11.392647 (+ 46684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.412970 (+ 20323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.413001 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:13.234858 (+1821857us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b2c0013bd48. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:08:13.236185 (+  1327us) log.cc:844] Serialized 26 byte log entry
0709 13:08:13.588529 (+352344us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:13.594532 (+  6003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.604844 (+ 10312us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.639022 (+ 34178us) raft_consensus.cc:1685] finished
0709 13:08:13.639041 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.646403 (+  7362us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20067,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":84,"spinlock_wait_cycles":23622912,"thread_start_us":351690,"threads_started":1,"wal-append.queue_time_us":397578}
I20260709 13:08:13.752007 26472 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.811828 26600 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:13.816406 26472 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:13.816681 26600 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:13.787155 23419 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:13.793412 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.812868 23450 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:13.797425 26328 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:13.778074 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:13.801509 23453 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:08:13.810951 26614 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:13.816819 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:13.829380 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.832597 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.843327 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.849196 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:13.849576 23446 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:13.849915 22756 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
W20260709 13:08:13.855649 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.880682 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:13.840508 26209 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:13.890396 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:13.893491 22754 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:13.905754 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:13.906396 26600 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.975366 26351 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:13.991510 26351 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.978870 26328 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:13.980398 26637 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.218s	user 0.001s	sys 0.002s
W20260709 13:08:13.985740 26640 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
I20260709 13:08:13.994199 26328 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:13.978907 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:13.909886 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:13.996531 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66264.2us
W20260709 13:08:13.941891 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.998302 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:13.998162 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 182225us
I20260709 13:08:13.978377 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1137) took 4464 ms. Trace:
I20260709 13:08:14.001809 23419 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:14.006572 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1096) took 2163 ms. Trace:
I20260709 13:08:13.849149 26614 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:14.008533 23452 rpcz_store.cc:276] 0709 13:08:09.514699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.183167 (+1668468us) service_pool.cc:224] Handling call
0709 13:08:11.183473 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:13.729973 (+2546500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.925811 (+195838us) spinlock_profiling.cc:243] Waited 2.36 s on lock 0x7b54001716d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:13.978296 (+ 52485us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5435574016}
I20260709 13:08:14.017877 23449 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:13.979417 26209 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:14.021025 26321 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:14.024055 26321 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.024464 26321 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:14.024509 26209 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:14.025652 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.026968 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.028259 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.028998 22900 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:14.029963 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.031258 22903 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:14.031802 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.032209 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.032651 22901 raft_consensus.cc:2410] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:08:14.011541 23445 rpcz_store.cc:276] 0709 13:08:11.842759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.388939 (+1546180us) service_pool.cc:224] Handling call
0709 13:08:13.389387 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:13.473853 (+ 84466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:13.473889 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:13.474420 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:08:13.601141 (+126721us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:13.643783 (+ 42642us) log.cc:844] Serialized 14 byte log entry
0709 13:08:13.644505 (+   722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.751294 (+106789us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b54006329d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:13.751342 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:13.999776 (+248434us) raft_consensus.cc:1685] finished
0709 13:08:14.006395 (+  6619us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:14.006518 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":250149,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":46,"raft.queue_time_us":333,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":7464,"spinlock_wait_cycles":194619520,"thread_start_us":128631,"threads_started":2,"wal-append.queue_time_us":303844}
I20260709 13:08:14.030081 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.034617 22898 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:14.035120 22900 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:14.035917 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.036216 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.036958 22902 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:14.037889 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.038020 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:13.982239 26591 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:14.021304 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:14.001498 26600 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:14.051486 22905 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:14.014757 26614 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:14.052829 23416 leader_election.cc:336] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.610s (SENT)
I20260709 13:08:14.060788 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.018333 26360 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:14.073803 22899 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:14.075636 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.066203 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.077646 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.095144 22903 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:14.079278 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56822.7us
I20260709 13:08:14.074911 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.093294 22754 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:13.914165 26517 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.207s
I20260709 13:08:14.100111 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.101486 23452 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:14.078153 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.106118 22901 raft_consensus.cc:2410] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:08:14.107842 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.069392 26459 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:14.110186 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:14.110261 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.113623 22903 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:14.115834 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.116776 22900 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:14.117056 22902 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:08:14.118060 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.119745 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.118834 22898 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:14.102415 26351 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:14.118741 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.122601 22897 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:14.116276 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:14.123116 23452 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:14.126088 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.126204 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:14.127090 23445 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:14.066581 26472 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:14.130424 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 119377us
W20260709 13:08:14.135248 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.131475 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:14.137770 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.140374 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.141175 22899 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:08:14.121677 26360 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:14.141306 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.144096 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:14.144421 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:14.145443 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:14.152737 23448 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:14.163051 23445 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:14.163748 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.167248 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.152128 26360 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:14.165772 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:14.170466 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:14.173032 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:14.175758 23416 leader_election.cc:336] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.759s (SENT)
I20260709 13:08:14.169900 23452 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:14.177613 26321 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:14.178452 22797 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:14.179123 26459 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:14.179437 26497 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:14.180035 26321 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:14.180136 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.180795 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.180270 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:14.185846 23446 raft_consensus.cc:2368] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8.
I20260709 13:08:14.176350 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:14.189143 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:14.190836 22748 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:08:14.192281 22905 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:14.192653 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.192910 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.193655 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.193960 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.194281 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.194589 22896 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:14.197016 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.146202 26614 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:14.200220 22902 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
W20260709 13:08:14.202677 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 304910us
I20260709 13:08:14.188387 26459 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:14.215673 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:14.246078 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.325513 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 149798us
I20260709 13:08:14.237844 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:14.224140 22903 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:14.299465 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:14.252833 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.199954 26580 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:14.318509 22898 raft_consensus.cc:2410] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:08:14.313967 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.258742 22747 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:08:14.330470 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.219393 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 53607.9us
W20260709 13:08:14.305187 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.322271 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.336575 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.270889 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:14.338274 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.338829 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.214551 22900 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:14.224422 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:14.337587 22755 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:14.351788 22896 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
W20260709 13:08:14.362916 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 182703us
I20260709 13:08:14.363891 22750 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:14.365607 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:14.214586 26243 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:14.367604 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:14.368880 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.374586 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.375412 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:14.254882 26614 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:14.253363 26472 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:14.385547 22897 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:14.380975 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:14.386977 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.379000 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:14.388715 22755 raft_consensus.cc:2410] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 11 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:08:14.382790 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:14.391387 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:14.391711 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.391975 22756 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:14.392540 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.392659 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.384855 22900 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:14.396317 22753 raft_consensus.cc:2368] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10.
W20260709 13:08:14.405201 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.407994 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.411692 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:14.415093 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.417129 22751 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:14.420096 22752 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:08:14.421638 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.423456 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:14.426468 22905 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
W20260709 13:08:14.427340 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 54494.1us
W20260709 13:08:14.428241 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.433197 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.272194 26360 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:14.434930 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.436075 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:14.436902 22899 raft_consensus.cc:2393] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 7: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:14.439270 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.441160 22897 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
W20260709 13:08:14.447685 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 61730.9us
I20260709 13:08:14.456857 22902 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:14.460752 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:14.461346 22902 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:08:14.463018 22903 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:14.466138 22896 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:14.466344 22749 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:14.470312 22749 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:14.468437 22748 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:14.443116 26347 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:08:14.468485 22905 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:14.468967 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:14.467733 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.474265 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.474638 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.474992 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.475234 26243 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:14.475775 22749 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:14.477103 22749 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:14.482509 22756 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:14.484793 22752 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:08:14.487149 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 354939us
I20260709 13:08:14.488627 23419 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:14.487938 22751 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:08:14.491460 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:14.492215 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56258.7us
I20260709 13:08:14.489286 22797 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:14.470221 26321 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:08:14.500584 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:14.494402 26328 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:14.492057 26600 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:08:14.505765 26604 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:08:14.502664 26591 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:14.517560 23416 leader_election.cc:336] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.502s (SENT)
I20260709 13:08:14.517969 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:14.502525 26610 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:08:14.508834 26594 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:14.511282 26328 raft_consensus.cc:697] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:14.538587 26328 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:14.539427 22899 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:08:14.526477 26498 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:14.545873 26243 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:14.521884 26594 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:14.546732 26652 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:08:14.547703 22900 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: [7.4-9.5]
W20260709 13:08:14.570972 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56242.6us
I20260709 13:08:14.565004 22749 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:08:14.551628 26649 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 13:08:14.554205 26431 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:14.554965 26604 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:14.557540 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:14.558485 26443 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:14.561932 26459 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:08:14.572211 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45147us
I20260709 13:08:14.578783 22650 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 339) took 8260 ms. Trace:
I20260709 13:08:14.570990 26594 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:14.596822 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 82647.5us
I20260709 13:08:14.578627 26390 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
W20260709 13:08:14.597681 23416 leader_election.cc:336] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.592s (SENT)
I20260709 13:08:14.598141 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:14.599256 26347 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:08:14.599452 23416 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.835s (SENT)
I20260709 13:08:14.599949 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:14.604803 22756 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:14.572916 26498 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:14.605831 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 84927us
I20260709 13:08:14.606230 26498 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:14.607018 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:14.607983 26614 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:14.608305 26614 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:14.609081 26443 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:14.609145 26614 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:14.610059 26243 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:14.610529 26614 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:14.611621 23416 leader_election.cc:336] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 2.088s (SENT)
I20260709 13:08:14.611272 26390 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:14.613991 26390 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:14.614481 26390 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:14.574790 26209 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:14.616550 26209 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:14.617646 26390 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:14.618741 22653 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 6 to 7, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 7 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.621086 26209 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:14.612087 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:14.622768 22749 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:08:14.699995 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.713861 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.725775 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 165822us
W20260709 13:08:14.732659 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 121117us
W20260709 13:08:14.745455 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 222764us
I20260709 13:08:14.731129 26610 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:08:14.762615 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 2.986s (SENT) [suppressed 48 similar messages]
I20260709 13:08:14.743928 26244 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:08:14.764861 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.777513 22871 leader_election.cc:400] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10.
W20260709 13:08:14.778349 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.780202 22871 leader_election.cc:403] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:08:14.781906 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 175007us
I20260709 13:08:14.769017 26390 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:14.767974 26244 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:14.797853 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:14.801702 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.579173 22650 rpcz_store.cc:276] 0709 13:08:06.317742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.317918 (+   176us) service_pool.cc:224] Handling call
0709 13:08:14.578517 (+8260599us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:08.305792 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:08.831416 (+525624us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:08.831451 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:08:08.831470 (+    19us) tablet.cc:662] Decoding operations
0709 13:08:08.833183 (+  1713us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:08.833240 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:08.833269 (+    29us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:08:08.834630 (+  1361us) tablet.cc:701] Row locks acquired
0709 13:08:08.834647 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:08:08.834779 (+   132us) write_op.cc:270] Start()
0709 13:08:08.834884 (+   105us) write_op.cc:276] Timestamp: P: 1783602488834732 usec, L: 0
0709 13:08:08.834903 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:08:08.835536 (+   633us) log.cc:844] Serialized 7784 byte log entry
0709 13:08:10.832240 (+1996704us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb71cf10ac 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b
0709 13:08:11.984280 (+1152040us) op_driver.cc:464] REPLICATION: finished
0709 13:08:12.297339 (+313059us) write_op.cc:301] APPLY: starting
0709 13:08:12.297431 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:12.455210 (+157779us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:12.462607 (+  7397us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:12.752476 (+289869us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:13.305066 (+552590us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 13:08:13.305192 (+   126us) write_op.cc:312] APPLY: finished
0709 13:08:13.306651 (+  1459us) log.cc:844] Serialized 139 byte log entry
0709 13:08:13.381778 (+ 75127us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb71cf10ac 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b
0709 13:08:13.640611 (+258833us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:13.665115 (+ 24504us) write_op.cc:454] Released schema lock
0709 13:08:13.665207 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:13.712288 (+ 47081us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:13.712321 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":556241,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":283262,"apply.run_cpu_time_us":56023,"apply.run_wall_time_us":1411205,"num_ops":20,"prepare.queue_time_us":381515,"prepare.run_cpu_time_us":68780,"prepare.run_wall_time_us":3549342,"raft.queue_time_us":162677,"raft.run_cpu_time_us":10667,"raft.run_wall_time_us":250019,"replication_time_us":3149205,"spinlock_wait_cycles":292069504,"thread_start_us":3566857,"threads_started":5,"wal-append.queue_time_us":3259277,"wal-append.run_cpu_time_us":10299,"wal-append.run_wall_time_us":1246769}]]}
I20260709 13:08:14.788158 22653 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:08:14.808358 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 253779us
W20260709 13:08:14.818861 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 71573.8us
W20260709 13:08:14.819538 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.832567 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:14.791097 26478 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:08:14.847126 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:14.852542 22827 raft_consensus.cc:2376] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8. Re-sending same reply.
I20260709 13:08:14.857089 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:14.859812 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:14.799582 26610 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:08:14.847798 26351 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:14.868947 22824 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:14.872395 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:14.873329 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53671us
W20260709 13:08:14.883426 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.876502 22823 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:14.850117 26600 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:08:14.876907 22653 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 9, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 9 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.872187 26244 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:14.894220 22824 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:14.897754 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:14.898239 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 103376us
W20260709 13:08:14.905901 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.906461 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.908139 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73594.6us
I20260709 13:08:14.908599 22826 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:14.909968 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:14.910494 22827 raft_consensus.cc:2376] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10. Re-sending same reply.
I20260709 13:08:14.912552 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:14.905572 26667 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:08:14.920267 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:14.920994 22829 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
W20260709 13:08:14.921883 26665 log.cc:927] Time spent T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:08:14.924170 26684 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:14.918877 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:14.926599 22824 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:14.927385 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.903292 26671 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10.
I20260709 13:08:14.933220 22825 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
W20260709 13:08:14.940439 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66451.7us
I20260709 13:08:14.941951 22719 leader_election.cc:400] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:14.942330 22719 leader_election.cc:403] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:08:14.944132 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 120930us
I20260709 13:08:14.945785 26667 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:14.924165 26244 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:14.948577 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:14.930729 26517 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:14.912011 26662 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 13:08:14.953497 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.942644 22653 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.963796 22825 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:14.937894 26591 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:14.949044 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:14.969794 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50195.4us
I20260709 13:08:14.970897 22830 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:14.936648 26580 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:08:14.954180 26671 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:14.981514 26671 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:14.982520 26671 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:14.983827 26671 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:14.969583 26662 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:14.972911 22653 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 9 to 11, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 11 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.985690 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:14.949756 26667 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:15.000327 26667 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:15.001485 26667 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:15.002874 22653 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 10, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 10 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:15.006148 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.007577 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:14.985122 22824 raft_consensus.cc:2410] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:08:14.985267 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:14.980433 22823 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:14.997957 26691 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:08:14.998706 26663 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:08:14.999219 26670 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:15.021955 22827 raft_consensus.cc:2376] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8. Re-sending same reply.
I20260709 13:08:15.020808 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.020087 22653 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:15.033277 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.038275 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:15.044544 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 75729.5us
I20260709 13:08:15.044961 22899 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:15.044797 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:15.045959 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:15.047389 26671 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:15.048105 23452 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:15.044579 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:15.048629 26691 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:08:15.050307 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 64774.6us
W20260709 13:08:15.053920 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 111667us
I20260709 13:08:14.971794 26604 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
W20260709 13:08:15.058609 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.031164 26670 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:15.069438 22826 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:15.071031 26670 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.072618 26670 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:15.064975 26676 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:15.065618 26604 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:15.050829 26517 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:15.065706 22653 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 12, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 12 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:08:15.079708 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 115512us
I20260709 13:08:15.017145 26688 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:08:15.098326 22653 catalog_manager.cc:5697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 2 to 4, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:15.053074 26691 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:15.102383 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 167505us
W20260709 13:08:15.094724 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.103008 22747 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:08:15.103233 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.096572 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.028602 26594 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:15.110875 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 64973.8us
I20260709 13:08:15.105800 22752 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:15.111567 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:15.089944 26604 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:15.114727 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:15.084017 26676 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:15.117909 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:15.117990 22653 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 7 to 8, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 8 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:15.119342 22899 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:15.120754 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:15.105527 26580 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:15.124930 22896 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:15.129030 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:15.126617 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:15.130283 22904 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:15.131428 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:15.134312 26676 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:15.150100 26676 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:15.141037 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:15.151710 22896 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:15.142855 22868 leader_election.cc:400] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12.
I20260709 13:08:15.154954 26688 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:15.156682 26696 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:15.157033 26696 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:15.157908 26696 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.158921 26696 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:15.126400 26689 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.161935 26689 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:15.162532 26666 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:08:15.163800 22868 consensus_peers.cc:597] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 135e2f7520a34b53aaf4ddb360a8a029. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:15.146544 26690 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:08:15.146652 26594 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:15.140810 26672 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:15.164424 26689 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:15.165818 22902 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:08:15.197105 22902 pending_rounds.cc:85] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 9
I20260709 13:08:15.197356 22902 pending_rounds.cc:107] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:08:15.197588 22902 raft_consensus.cc:2889] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:15.142710 26604 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:15.195535 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 65596.8us
W20260709 13:08:15.197796 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 86291.1us
I20260709 13:08:15.198648 23449 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:08:15.202962 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 70617.5us
I20260709 13:08:15.203440 26443 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:15.203960 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:15.204628 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 80304.3us
I20260709 13:08:15.204763 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:15.205425 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:15.205586 26497 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:15.206261 26497 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.206645 26497 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:15.206743 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:15.207259 26668 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:15.213541 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:15.214121 22902 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:15.205108 26443 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.198796 26688 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:15.197751 26687 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:15.211778 26614 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:08:15.314917 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 84845.6us
W20260709 13:08:15.315059 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.322391 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 123687us
I20260709 13:08:15.198400 26610 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:08:15.212051 26684 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:15.325986 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.316668 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 107334us
W20260709 13:08:15.326046 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.198271 26328 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
W20260709 13:08:15.329326 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 115265us
W20260709 13:08:15.330327 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.334343 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.228231 26688 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:15.337764 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:15.340077 26694 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.217242 26689 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 13:08:15.226979 26675 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:08:15.228519 26443 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.341187 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:15.220680 26685 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.125s
W20260709 13:08:15.390748 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.345110 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:15.182337 26691 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:15.410789 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.211505 22826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:08:15.411463 22826 raft_consensus.cc:606] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:15.411877 22826 raft_consensus.cc:614] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:15.414398 26461 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:15.415205 26689 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:15.339740 26687 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
W20260709 13:08:15.416253 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 71208us
W20260709 13:08:15.426378 26701 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:08:15.426993 26698 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:08:15.427122 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 96040.4us
I20260709 13:08:15.427532 26698 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:15.427745 22822 raft_consensus.cc:1217] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Deduplicated request from leader. Original: 9.10->[10.11-10.11]   Dedup: 10.11->[]
I20260709 13:08:15.427995 26698 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:15.428284 22822 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: [10.11-10.11]
W20260709 13:08:15.429665 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 88564.4us
I20260709 13:08:15.429538 26321 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:15.431131 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:15.431434 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:15.432152 22751 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:15.424289 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:15.438772 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 230796us
I20260709 13:08:15.445894 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:15.448678 22868 connection.cc:256] RPC call timeout handler was delayed by -1.828052365000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
I20260709 13:08:15.340583 26694 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.386466 26670 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:08:15.457455 23448 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:08:15.456545 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:15.475282 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:15.478322 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:15.486423 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.486716 22751 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:15.487866 23444 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:15.452181 26459 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:15.449498 26600 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:15.433470 26671 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 13:08:15.488880 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 140834us
I20260709 13:08:15.489913 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:15.490972 26679 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:15.496261 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:15.495008 26478 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:15.491945 26328 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.510552 26351 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:15.515671 26713 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:15.516393 26713 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:15.518661 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.519251 22830 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:15.519670 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.520226 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.520731 22826 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:08:15.495005 26668 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:15.521013 22824 raft_consensus.cc:2410] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:15.520855 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.513759 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:15.519807 22822 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:15.519271 22825 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:15.494292 26459 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:15.527920 22829 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:08:15.530818 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.521030 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.534631 22828 raft_consensus.cc:2410] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:15.536464 22824 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:15.537819 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.508895 26390 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:15.537762 22829 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:15.512006 26478 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:15.458246 26688 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:15.538831 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.538535 22823 raft_consensus.cc:2376] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9. Re-sending same reply.
I20260709 13:08:15.522498 26671 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:08:15.543503 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.546101 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.548101 22825 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:08:15.550344 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 62260.8us
W20260709 13:08:15.553117 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.554024 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.516403 26351 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:08:15.537895 26461 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.545707 26478 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:15.521790 26668 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.562556 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.563905 22719 leader_election.cc:400] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8.
I20260709 13:08:15.564229 22719 leader_election.cc:403] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:15.545048 26687 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:15.549458 26671 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:15.556417 26684 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:08:15.567546 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:08:15.549122 26390 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:15.572199 22826 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:08:15.573040 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:15.581971 22905 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:15.533377 26328 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.597173 22902 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:08:15.614382 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.614895 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.575641 26687 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.565161 26461 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.645844 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:15.559851 26459 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:15.661984 26679 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:15.627617 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.628906 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:15.635748 26704 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:08:15.635402 26328 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:15.636348 26706 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:08:15.618953 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:15.636566 26708 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:08:15.635370 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:08:15.670795 26707 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:08:15.691257 22830 raft_consensus.cc:2410] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:15.636557 26685 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.695127 22824 raft_consensus.cc:2376] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9. Re-sending same reply.
I20260709 13:08:15.695967 26685 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.696455 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.671300 26671 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.696564 22824 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:08:15.611186 22823 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:15.618618 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.699397 26671 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:15.688691 26459 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:15.707180 22829 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:15.596844 26670 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8.
I20260709 13:08:15.709924 26670 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:15.710546 26670 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.598002 26720 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:08:15.634269 26709 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:08:15.692135 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.659462 26715 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.715555 26715 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.717729 26715 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:15.719121 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:15.719695 23453 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
I20260709 13:08:15.720841 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:15.713044 26670 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:15.715924 26685 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:15.727556 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.728953 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:15.742313 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.744730 22828 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:15.745591 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.747495 26478 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.688827 22753 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Committing config change with OpId 8.7: config changed from index -1 to 7, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:08:15.749861 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.750397 22823 raft_consensus.cc:2376] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6. Re-sending same reply.
I20260709 13:08:15.676152 26687 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:15.756932 22829 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:08:15.755520 22828 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:15.635878 26719 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:15.688858 26717 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:15.758204 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:15.758378 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:15.635469 26694 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:15.763046 22749 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:15.784667 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:15.788818 22904 consensus_meta.cc:287] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.655s	user 0.001s	sys 0.004s
I20260709 13:08:15.752264 26328 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.803360 22830 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:15.789237 22904 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:15.792469 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.800982 26684 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:15.762315 26725 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:08:15.761533 26459 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:15.805524 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:15.826155 26328 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.806712 26697 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:15.829627 26729 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:08:15.829411 26684 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.829851 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.830335 26478 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:15.779981 26719 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:15.832785 26705 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.408s	user 0.000s	sys 0.001s
I20260709 13:08:15.838388 26390 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:15.829624 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:15.807683 22900 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:15.807536 22824 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:15.850528 26498 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:15.850966 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:15.883980 22904 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
W20260709 13:08:15.851647 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.852165 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.858872 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 137221us
I20260709 13:08:15.869226 22831 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
W20260709 13:08:15.869256 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.873054 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.805559 26668 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:15.846967 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.838877 26694 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.851953 26459 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.882714 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:15.852870 26670 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.851038 26720 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:15.940279 23446 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:15.967208 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46148.3us
I20260709 13:08:15.965888 22754 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
W20260709 13:08:15.921761 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.948192 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.918963 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:15.920336 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 135765us
I20260709 13:08:15.960683 22825 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 7: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:15.886808 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.962056 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:15.964056 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.969739 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:15.977277 22754 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:15.977274 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:15.978286 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:15.978447 22904 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
W20260709 13:08:15.916204 26676 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.766s	user 0.010s	sys 0.000s
I20260709 13:08:15.981016 22830 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:15.916811 26718 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.990355 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:15.987404 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:15.995523 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:15.973671 26696 consensus_meta.cc:287] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.810s	user 0.012s	sys 0.000s
I20260709 13:08:15.996214 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.997606 22751 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:15.882225 26244 raft_consensus.cc:2764] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:08:15.999071 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:16.013489 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.962073 26459 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:15.979691 26498 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:16.016080 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.014217 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:16.006728 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:15.914031 26461 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:16.008625 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 121503us
I20260709 13:08:16.009518 26321 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:16.011363 26679 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:15.977272 26328 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:16.031086 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.011754 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:08:16.018034 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.016961 22831 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
W20260709 13:08:16.032513 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.021013 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:16.023535 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:15.915443 26719 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:16.026340 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:15.868866 26697 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:15.980698 26694 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:16.030186 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:15.972890 26670 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:16.064018 22901 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:16.067003 26734 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:16.067374 26670 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:16.060573 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:15.977566 26668 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.075896 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:16.077169 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:16.073118 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 52032.7us
I20260709 13:08:16.074999 26726 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:08:16.075798 26497 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.869s	user 0.011s	sys 0.000s
I20260709 13:08:15.979198 26390 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:08:16.082726 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.083722 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.087806 22750 pending_rounds.cc:85] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 4
I20260709 13:08:16.088173 22750 pending_rounds.cc:107] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:08:16.088387 22750 raft_consensus.cc:2889] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:16.006945 26718 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:16.098907 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:16.105109 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 78777.6us
I20260709 13:08:16.105862 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:16.064777 26697 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:16.107074 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.086814 26497 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:16.110342 26740 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:08:16.111138 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.111574 26497 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:16.112149 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.078652 26328 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:16.078668 26694 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:16.202527 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.026192 26591 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:16.204847 22828 raft_consensus.cc:2368] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7.
W20260709 13:08:16.092067 26698 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.662s	user 0.004s	sys 0.006s
I20260709 13:08:16.119927 22830 raft_consensus.cc:2368] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11.
I20260709 13:08:16.208626 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:16.204269 22831 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:16.107404 26580 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:16.097774 26726 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:16.218433 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:16.227423 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.229843 22825 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
W20260709 13:08:16.235766 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 268625us
W20260709 13:08:16.236428 22868 leader_election.cc:336] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.748s (SENT)
I20260709 13:08:16.245858 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:16.257828 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.207978 26591 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.271080 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:16.273976 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.274974 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.207064 26328 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:16.276811 22868 leader_election.cc:336] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.926s (SENT)
I20260709 13:08:16.248735 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.280169 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.281502 22822 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:16.281252 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:16.281481 22831 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:08:16.231204 26461 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:08:16.296293 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 190451us
I20260709 13:08:16.299782 22825 raft_consensus.cc:1217] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Deduplicated request from leader. Original: 6.4->[8.5-8.5]   Dedup: 8.5->[]
I20260709 13:08:16.300309 22825 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:08:16.299926 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 91365.5us
I20260709 13:08:16.301445 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:16.242980 26390 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.244836 26580 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:16.307235 22901 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:08:16.308640 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:16.310261 22901 pending_rounds.cc:85] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 6
I20260709 13:08:16.311290 22901 pending_rounds.cc:107] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:08:16.311574 22901 raft_consensus.cc:2889] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:16.289613 26328 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:16.283075 26591 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:16.314139 26728 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.457s	user 0.007s	sys 0.001s
I20260709 13:08:16.233771 26698 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.312862 26390 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:16.322506 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45714.1us
I20260709 13:08:16.323007 26715 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:16.282769 26497 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:16.323889 26580 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.325567 26755 ts_tablet_manager.cc:933] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:16.326234 22902 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:08:16.327589 26711 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.817s	user 0.000s	sys 0.001s
I20260709 13:08:16.313715 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.113209 26720 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:16.329631 22824 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: [6.8-7.9]
I20260709 13:08:16.330447 26675 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:16.331282 26684 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:16.401059 26684 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.331499 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:16.331661 26600 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Committing config change with OpId 8.7: config changed from index -1 to 7, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:08:16.336702 22747 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:16.314846 26710 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.315821 26722 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:16.315425 26749 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:16.339349 22822 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
W20260709 13:08:16.339535 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.309741 26719 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:16.270824 26498 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.407935 22756 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 5.
W20260709 13:08:16.394166 26672 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.183s	user 0.005s	sys 0.005s
W20260709 13:08:16.396574 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50288.1us
I20260709 13:08:16.429075 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:16.431421 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:16.409490 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 310607us
W20260709 13:08:16.424247 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.398201 26668 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:16.434789 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.421590 22868 leader_election.cc:336] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.976s (SENT)
I20260709 13:08:16.437815 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:08:16.439277 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 107859us
I20260709 13:08:16.427240 26675 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:16.452090 26694 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:16.401628 26684 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:16.456454 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.457708 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.456224 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:16.434892 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:16.461537 26743 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:08:16.461791 26747 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:08:16.461887 26390 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:16.417738 26710 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.437134 26670 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:08:16.447997 26672 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:16.506621 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 71816us
I20260709 13:08:16.458242 26694 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:16.485666 26668 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:16.517561 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:16.489198 26696 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:16.500075 26662 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:16.504154 26719 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.529850 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:16.520905 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 99320.9us
W20260709 13:08:16.527383 26744 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:08:16.538631 26710 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:16.506994 26718 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:16.531756 26672 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:16.520540 26694 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:16.531885 26698 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:16.426793 26726 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:16.534109 26687 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.758s	user 0.008s	sys 0.004s
W20260709 13:08:16.534821 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.509491 26610 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:16.528554 26662 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:16.488801 26675 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:08:16.548681 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 117340us
I20260709 13:08:16.540056 26759 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:08:16.558626 26749 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:08:16.582679 26759 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:16.583371 26759 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:16.527614 26390 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:16.530762 22868 leader_election.cc:336] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.982s (SENT)
W20260709 13:08:16.571049 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.560037 26672 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:16.575003 26767 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:08:16.574937 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:16.558073 26472 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:08:16.576746 26684 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:16.560160 26351 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:08:16.543670 26765 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:16.633285 26754 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.324s	user 0.000s	sys 0.002s
I20260709 13:08:16.581295 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:16.634933 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 53710.3us
I20260709 13:08:16.635459 26691 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:16.636420 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:16.636250 26328 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:16.638526 26734 consensus_meta.cc:287] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.006s
I20260709 13:08:16.638782 26667 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.638913 26734 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:16.639263 26667 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:16.590256 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.641682 26772 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:16.642092 26772 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.642241 26698 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:16.642463 26772 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:16.642773 26698 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:16.643676 26698 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.644737 26328 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:16.646085 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:16.646770 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 900) took 1068 ms. Trace:
W20260709 13:08:16.595410 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 133183us
I20260709 13:08:16.584792 26759 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.594722 26390 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:16.647267 22900 rpcz_store.cc:276] 0709 13:08:15.557126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.557251 (+   125us) service_pool.cc:224] Handling call
0709 13:08:15.803984 (+246733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.879741 (+ 75757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.609717 (+729976us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400059bd0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:16.625003 (+ 15286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552110848}
W20260709 13:08:16.604377 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73624.2us
W20260709 13:08:16.592336 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.649741 22868 leader_election.cc:336] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.070s (SENT)
I20260709 13:08:16.549194 26719 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.652442 26443 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:16.596709 26762 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 13:08:16.613374 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.570696 26478 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:16.654672 26759 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:16.655323 26431 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.655648 26478 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:16.625716 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2448) took 1011 ms. Trace:
I20260709 13:08:16.651229 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:16.657166 22750 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:16.627661 23449 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:16.632697 26672 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.646813 22898 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:16.656340 22826 rpcz_store.cc:276] 0709 13:08:15.614313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.568146 (+953833us) service_pool.cc:224] Handling call
0709 13:08:16.625662 (+ 57516us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:16.576553 26662 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:16.603560 26763 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 13:08:16.650305 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.567611 26687 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:16.624857 26749 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:16.651229 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:16.656087 26478 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:16.660813 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:16.641173 26667 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:16.663337 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:16.665979 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:16.669641 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:16.668247 26321 consensus_meta.cc:287] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.002s
I20260709 13:08:16.594230 26755 tablet_copy_client.cc:323] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:08:16.628240 26580 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.655807 26431 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.671773 22756 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:16.673276 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:16.738830 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.744557 22752 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:16.756372 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:16.764714 26721 raft_consensus.cc:2764] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:08:16.672223 26321 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:08:16.769786 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 109476us
I20260709 13:08:16.771637 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:16.805424 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 130519us
W20260709 13:08:16.822963 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.831797 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.882989 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:16.886561 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.664260 26328 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.116s
I20260709 13:08:16.674652 26734 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:16.743746 26517 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:16.893505 26775 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:16.887647 26443 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.771387 26591 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.743541 26719 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:16.900305 22750 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:16.806447 26321 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:16.916455 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.937567 26675 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:16.671361 26749 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:16.941097 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2245) took 1026 ms. Trace:
I20260709 13:08:16.990968 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:16.921365 26734 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.903384 26610 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:17.008720 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:17.011143 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:16.944070 26431 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.005182 26765 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:17.014530 22749 rpcz_store.cc:276] 0709 13:08:15.914398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.982021 (+ 67623us) service_pool.cc:224] Handling call
0709 13:08:15.982299 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:16.070076 (+ 87777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:16.070114 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:16.070573 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:08:16.401216 (+330643us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:16.402435 (+  1219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.677445 (+275010us) spinlock_profiling.cc:243] Waited 87.3 ms on lock 0x7b5400077bd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:16.677507 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:16.904119 (+226612us) raft_consensus.cc:1685] finished
0709 13:08:16.904136 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:16.941023 (+ 36887us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":842,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":200883968,"thread_start_us":287099,"threads_started":1,"wal-append.queue_time_us":327153}
I20260709 13:08:16.901674 26580 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.015946 26390 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:17.016822 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 391474us
I20260709 13:08:17.014549 26698 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:17.021785 26734 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:17.044730 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:17.046250 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:17.029829 26712 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:17.008222 26443 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.021154 26738 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:17.014292 26517 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:17.062158 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 345118us
I20260709 13:08:17.068261 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:17.062387 26719 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.033739 26765 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:17.049278 26459 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.013s	user 0.011s	sys 0.000s
I20260709 13:08:17.007558 26775 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.043900 26431 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:08:17.074589 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:17.075335 22827 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:17.076536 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:17.080046 22828 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:17.021451 26675 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.082474 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:17.084872 22823 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:17.083292 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:17.083894 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.087028 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:17.088367 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:17.092944 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:17.057936 26498 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:08:17.097784 26685 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:17.098556 26685 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:08:17.099110 22719 connection.cc:256] RPC call timeout handler was delayed by -1.4517475000001241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
I20260709 13:08:17.067045 26738 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:17.077491 26765 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:17.046401 26696 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:08:17.078972 26737 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:17.152138 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50624.3us
W20260709 13:08:17.152923 23449 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1107) took 1718 ms (client timeout 1607 ms). Trace:
W20260709 13:08:17.163895 23449 rpcz_store.cc:269] 0709 13:08:15.434625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.434842 (+   217us) service_pool.cc:224] Handling call
0709 13:08:17.152720 (+1717878us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000b13d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:17.152844 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2499882752}
I20260709 13:08:17.063099 26724 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:08:17.060735 26698 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.110394 26738 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:17.184051 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 412443us
I20260709 13:08:17.083554 26719 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.081645 26459 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.095657 26244 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:08:17.153121 26687 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:17.154465 26610 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:17.218394 22829 raft_consensus.cc:2368] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6.
W20260709 13:08:17.255112 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 207815us
I20260709 13:08:17.227972 26693 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:17.249166 26694 consensus_meta.cc:287] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.000s
W20260709 13:08:17.261974 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 74464.2us
I20260709 13:08:17.240722 26591 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:17.268515 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 104376us
W20260709 13:08:17.268184 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 168518us
I20260709 13:08:17.225915 26610 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.250272 26712 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:17.254163 26749 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:17.266809 26693 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.281594 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:17.225562 26737 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:08:17.284745 26443 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:17.275846 26779 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
W20260709 13:08:17.285691 26772 consensus_meta.cc:287] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.005s
I20260709 13:08:17.274714 26782 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.280778 26580 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:17.265831 26694 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:17.291402 26780 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 13:08:17.305434 22719 leader_election.cc:336] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.873s (SENT)
I20260709 13:08:17.282444 26685 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:08:17.316776 22755 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:17.310460 26795 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.318282 26795 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.320993 26795 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:17.300974 26772 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:17.321985 26726 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.322494 26726 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:17.323132 26787 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:08:17.323954 26772 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:17.326504 26785 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:08:17.299949 26600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:42486 (request call id 17) took 7561 ms. Trace:
I20260709 13:08:17.328732 26600 rpcz_store.cc:276] 0709 13:08:09.745806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.750414 (+  4608us) service_pool.cc:224] Handling call
0709 13:08:09.841886 (+ 91472us) log.cc:844] Serialized 499 byte log entry
0709 13:08:10.864468 (+1022582us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b18003f1c60. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb74d54b0b 00007fcb74dd9b63 00007fcb74e01752 00007fcb79dc1d29
0709 13:08:17.285131 (+6420663us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":6589,"mutex_wait_us":533976,"raft.queue_time_us":662581,"raft.run_cpu_time_us":6677,"raft.run_wall_time_us":87362,"spinlock_wait_cycles":55261056,"thread_start_us":627275,"threads_started":2,"wal-append.queue_time_us":578104,"wal-append.run_cpu_time_us":13903,"wal-append.run_wall_time_us":1656288}
W20260709 13:08:17.330269 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.324575 26726 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:17.333869 26773 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.439s	user 0.001s	sys 0.001s
I20260709 13:08:17.335018 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2247) took 1420 ms. Trace:
I20260709 13:08:17.335345 22751 rpcz_store.cc:276] 0709 13:08:15.914921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.033087 (+118166us) service_pool.cc:224] Handling call
0709 13:08:16.091257 (+ 58170us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:16.117709 (+ 26452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:16.117736 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:16.118160 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:08:16.562486 (+444326us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:16.562539 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:16.562631 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:17.333191 (+770560us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:17.334780 (+  1589us) raft_consensus.cc:1685] finished
0709 13:08:17.334797 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.334969 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19192,"raft.queue_time_us":26791,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":7322,"spinlock_wait_cycles":16432896,"thread_start_us":283112,"threads_started":1,"wal-append.queue_time_us":430555}
I20260709 13:08:17.337970 26712 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:17.342352 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.344578 26610 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:17.345929 26604 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:17.347276 26328 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.347890 26328 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:17.349983 26789 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:08:17.350450 26328 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:17.351411 26790 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:08:17.354092 26688 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:17.312319 26798 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:08:17.359315 26478 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.684s	user 0.008s	sys 0.000s
I20260709 13:08:17.359727 26478 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:17.363530 26775 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.364667 26662 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:17.365621 26786 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:08:17.366895 26719 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:17.369697 26691 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.734s	user 0.009s	sys 0.001s
I20260709 13:08:17.370380 26459 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.304848 26693 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.372082 26693 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:17.372126 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 59020.5us
I20260709 13:08:17.372728 26693 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.373735 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:17.374473 26720 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:17.374635 26693 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.375106 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:17.307711 26675 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.376235 26720 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:17.377300 26766 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.871s	user 0.003s	sys 0.007s
I20260709 13:08:17.377615 26741 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.377463 26594 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:17.378108 26741 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:17.379417 26703 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.490s	user 0.000s	sys 0.016s
I20260709 13:08:17.379701 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:17.381490 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:17.381886 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2166) took 1704 ms. Trace:
I20260709 13:08:17.382426 26594 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:17.383123 26694 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.381507 26685 consensus_queue.cc:1243] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:08:17.383531 26693 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:17.384531 26693 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:17.385725 26693 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.388631 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:17.389734 23449 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 15 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:08:17.378443 26517 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:17.349073 26321 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:17.393039 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.400067 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:17.393576 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:17.394877 22748 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:08:17.393211 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:17.401872 22749 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:17.402671 22748 pending_rounds.cc:85] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:08:17.402995 22748 pending_rounds.cc:107] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:08:17.403270 22748 raft_consensus.cc:2889] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:17.407755 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:17.408836 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:17.409490 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:17.411646 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.412515 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:17.381786 26698 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.416877 22754 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:17.418949 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:17.423056 22753 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:08:17.376966 26720 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:08:17.449568 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:17.450336 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:17.451308 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:17.453516 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:17.455039 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:17.459956 22823 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:17.459057 22898 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:17.537616 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:17.382305 22903 rpcz_store.cc:276] 0709 13:08:15.677968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.678198 (+   230us) service_pool.cc:224] Handling call
0709 13:08:15.725637 (+ 47439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.921516 (+195879us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:16.018415 (+ 96899us) log.cc:844] Serialized 14 byte log entry
0709 13:08:16.074377 (+ 55962us) log.cc:844] Serialized 14 byte log entry
0709 13:08:16.077898 (+  3521us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:16.078077 (+   179us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:16.078383 (+   306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:17.381690 (+1303307us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b5400275454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:17.381731 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:17.381835 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16738,"prepare.queue_time_us":784507,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":68,"raft.queue_time_us":59240,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":5530,"spinlock_wait_cycles":449911680,"thread_start_us":27895,"threads_started":2,"wal-append.queue_time_us":252518,"wal-append.run_cpu_time_us":1426,"wal-append.run_wall_time_us":1041183}
I20260709 13:08:17.547573 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:17.461862 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:17.550735 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.543915 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:17.546866 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:17.553165 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:17.410895 26431 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.465006 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:17.554689 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:17.412171 26667 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.470703 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:17.563460 26600 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:17.855978 26600 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.856357 26600 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:17.860765 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.388954 26798 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.181s
I20260709 13:08:17.569628 26691 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:17.864094 26691 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.864461 26691 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:17.569723 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 135e2f7520a34b53aaf4ddb360a8a029 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:08:17.870078 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 135e2f7520a34b53aaf4ddb360a8a029 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-135e2f7520a34b53aaf4ddb360a8a029
I20260709 13:08:17.570366 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:17.570995 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:17.421355 26737 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.873984 23448 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:17.874054 26737 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.875237 22901 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:17.875849 26694 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:17.876194 26737 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:17.561455 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 265426us
W20260709 13:08:19.595932 22720 connection.cc:256] RPC call timeout handler was delayed by -1.9453579910000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:08:22.003907 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:18.239285 22747 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:19.653697 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1078) took 2261 ms. Trace:
I20260709 13:08:22.059300 22750 rpcz_store.cc:276] 0709 13:08:17.392436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.392613 (+   177us) service_pool.cc:224] Handling call
0709 13:08:17.410910 (+ 18297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:17.422280 (+ 11370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:17.422316 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:17.971779 (+549463us) log.cc:844] Serialized 26 byte log entry
0709 13:08:18.334771 (+362992us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:18.334822 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:18.334915 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:19.653417 (+1318502us) raft_consensus.cc:1685] finished
0709 13:08:19.653441 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:19.653620 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43208,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":74361,"thread_start_us":362574,"threads_started":1,"wal-append.queue_time_us":1678996,"wal-append.run_cpu_time_us":2084,"wal-append.run_wall_time_us":1003476}
I20260709 13:08:17.767089 22748 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:22.064071 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2690) took 4652 ms. Trace:
I20260709 13:08:22.064482 22748 rpcz_store.cc:276] 0709 13:08:17.411184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.766291 (+355107us) service_pool.cc:224] Handling call
0709 13:08:17.766676 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.063744 (+4297068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.063983 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:17.558681 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:22.066569 22749 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:19.589352 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1074) took 2997 ms. Trace:
I20260709 13:08:19.613018 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1086) took 2049 ms. Trace:
I20260709 13:08:22.075250 22756 rpcz_store.cc:276] 0709 13:08:17.564161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.586941 (+2022780us) service_pool.cc:224] Handling call
0709 13:08:19.587269 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:19.587463 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:19.610361 (+ 22898us) log.cc:844] Serialized 14 byte log entry
0709 13:08:19.612666 (+  2305us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:19.612720 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:19.612770 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.612846 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:19.612962 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68894,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":1355154,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":24562,"threads_started":2,"wal-append.queue_time_us":43567,"wal-append.run_cpu_time_us":1164,"wal-append.run_wall_time_us":1001345}
W20260709 13:08:18.349565 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 390857us
I20260709 13:08:22.080559 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:20.249964 26824 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:22.086247 26824 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:20.249069 26825 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:19.078742 22830 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2463) took 1669 ms (client timeout 1965 ms). Trace:
W20260709 13:08:19.619355 22852 debug-util.cc:398] Leaking SignalData structure 0x7b08000a5400 after lost signal to thread 21632
W20260709 13:08:22.101485 22852 debug-util.cc:398] Leaking SignalData structure 0x7b080039cb80 after lost signal to thread 22688
I20260709 13:08:17.820741 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:17.811695 22897 raft_consensus.cc:2410] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:08:22.106244 26604 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.106803 26604 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.608428 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:17.894582 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.111663 22900 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
W20260709 13:08:22.112134 22900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 914) took 4572 ms (client timeout 1914 ms). Trace:
W20260709 13:08:22.112465 22900 rpcz_store.cc:269] 0709 13:08:17.539951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.894361 (+354410us) service_pool.cc:224] Handling call
0709 13:08:22.112061 (+4217700us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:19.678544 22901 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2176) took 2270 ms (client timeout 1807 ms). Trace:
W20260709 13:08:22.114468 22901 rpcz_store.cc:269] 0709 13:08:17.408072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.408178 (+   106us) service_pool.cc:224] Handling call
0709 13:08:19.678289 (+2270111us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b540031f954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:19.678433 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746825472}
W20260709 13:08:22.115526 22901 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1629) took 4572 ms (client timeout 2018 ms). Trace:
W20260709 13:08:22.115875 22901 rpcz_store.cc:269] 0709 13:08:17.542691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.115361 (+4572670us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.115468 (+   107us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.116611 22901 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1626) took 4575 ms (client timeout 2046 ms). Trace:
I20260709 13:08:17.611304 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.118633 22902 raft_consensus.cc:2410] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:08:22.119037 22902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 913) took 4579 ms (client timeout 1610 ms). Trace:
W20260709 13:08:22.119385 22902 rpcz_store.cc:269] 0709 13:08:17.539690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.590128 (+ 50438us) service_pool.cc:224] Handling call
0709 13:08:22.118974 (+4528846us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:22.120142 22902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 910) took 4581 ms (client timeout 2216 ms). Trace:
W20260709 13:08:22.120455 22902 rpcz_store.cc:269] 0709 13:08:17.538870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.119979 (+4581109us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.120085 (+   106us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:17.586083 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.122637 22904 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
W20260709 13:08:22.123148 22904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 916) took 4582 ms (client timeout 1510 ms). Trace:
W20260709 13:08:22.123481 22904 rpcz_store.cc:269] 0709 13:08:17.540484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.549413 (+  8929us) service_pool.cc:224] Handling call
0709 13:08:22.123072 (+4573659us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:22.124228 22904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 911) took 4585 ms (client timeout 2983 ms). Trace:
I20260709 13:08:17.887773 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:22.126473 22905 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1628) took 4584 ms (client timeout 1836 ms). Trace:
W20260709 13:08:22.126870 22905 rpcz_store.cc:269] 0709 13:08:17.542404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.887591 (+345187us) service_pool.cc:224] Handling call
0709 13:08:22.126400 (+4238809us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:22.127688 22905 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 917) took 4587 ms (client timeout 3371 ms). Trace:
W20260709 13:08:22.127996 22905 rpcz_store.cc:269] 0709 13:08:17.540749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.127529 (+4586780us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.127640 (+   111us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.128718 22905 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2181) took 2447 ms (client timeout 1746 ms). Trace:
I20260709 13:08:17.893453 22914 tablet_copy_source_session.cc:215] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:22.132737 22914 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:42490 (request call id 907) took 4709 ms. Trace:
W20260709 13:08:21.040129 22903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1632) took 3496 ms (client timeout 2010 ms). Trace:
W20260709 13:08:21.303325 22871 connection.cc:256] RPC call timeout handler was delayed by -2.350132212999938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
W20260709 13:08:22.133590 22871 connection.cc:256] RPC call timeout handler was delayed by -1.8246692870000061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
W20260709 13:08:22.133473 22903 rpcz_store.cc:269] 0709 13:08:17.543606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.039976 (+3496370us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:21.040076 (+   100us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.134460 22903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2182) took 2453 ms (client timeout 1797 ms). Trace:
W20260709 13:08:22.134794 22903 rpcz_store.cc:269] 0709 13:08:19.681441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.134304 (+2452863us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.134407 (+   103us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.135553 22903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2180) took 2454 ms (client timeout 2411 ms). Trace:
W20260709 13:08:22.135874 22903 rpcz_store.cc:269] 0709 13:08:19.680706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.135407 (+2454701us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.135504 (+    97us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.136371 22871 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.65:37213 timed out after 1.781s (SENT) [suppressed 11 similar messages]
W20260709 13:08:21.310124 22752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 1085) took 3877 ms (client timeout 3371 ms). Trace:
W20260709 13:08:22.137665 22752 rpcz_store.cc:269] 0709 13:08:17.433208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.309903 (+3876695us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:21.310033 (+   130us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:21.311487 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:20.248646 23416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.004s	sys 0.008s
I20260709 13:08:22.133126 22914 rpcz_store.cc:276] 0709 13:08:17.537859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.569469 (+ 31610us) service_pool.cc:224] Handling call
0709 13:08:22.132633 (+4563164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:22.142980 23416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.451s	user 0.008s	sys 0.008s
W20260709 13:08:20.250982 23417 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.005s	sys 0.008s
W20260709 13:08:22.144272 23417 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.453s	user 0.005s	sys 0.009s
W20260709 13:08:20.249655 23419 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.005s	sys 0.008s
W20260709 13:08:22.146374 23419 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.455s	user 0.006s	sys 0.008s
I20260709 13:08:17.633047 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:17.891343 23445 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:22.150890 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1173) took 4695 ms. Trace:
I20260709 13:08:22.151090 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:17.893149 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:17.584885 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:22.153203 23453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1118) took 4753 ms (client timeout 1782 ms). Trace:
I20260709 13:08:17.876672 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:17.622100 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.151291 23445 rpcz_store.cc:276] 0709 13:08:17.455920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.456340 (+   420us) service_pool.cc:224] Handling call
0709 13:08:17.890824 (+434484us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:08:17.891018 (+   194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.150619 (+4259601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.150835 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21358976}
I20260709 13:08:22.154824 23450 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
W20260709 13:08:22.155267 23450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1119) took 4558 ms (client timeout 2124 ms). Trace:
W20260709 13:08:22.155292 23445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 737) took 4472 ms (client timeout 1584 ms). Trace:
I20260709 13:08:17.596189 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:22.156600 23448 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1182) took 4490 ms (client timeout 1527 ms). Trace:
I20260709 13:08:22.156841 23452 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:17.810393 26244 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.269376 26321 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.158656 26244 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.159452 26321 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.587212 26431 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.162575 26244 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:22.161808 23445 rpcz_store.cc:269] 0709 13:08:17.683238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.155161 (+4471923us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.155245 (+    84us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.163656 26431 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:17.960616 26459 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:22.164908 26459 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.167202 26459 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.399499 26461 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:22.168387 26461 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.605052 26478 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:21.313035 26594 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:22.173899 26321 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:17.883486 26600 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.177141 26600 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:17.600618 26666 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:08:17.679729 26672 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.589684 26667 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.183234 26667 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:17.817788 26688 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.190107 26688 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:22.191608 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43910.5us
I20260709 13:08:22.192162 26688 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.886885 26691 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.876397 26694 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.194656 26691 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:22.195586 26694 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:19.686646 26696 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.196820 26696 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:22.197829 23419 connection.cc:256] RPC call timeout handler was delayed by -1.7596686000000545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3371s for a response
I20260709 13:08:21.303481 26724 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.200515 26321 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:22.200783 26724 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.203636 26724 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:17.665699 26731 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.643s	user 0.015s	sys 0.006s
W20260709 13:08:17.635547 26734 consensus_meta.cc:287] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.000s
I20260709 13:08:17.624720 26738 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:22.207484 26734 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.209470 26738 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:22.209640 26734 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:17.560115 26741 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:17.635277 26759 consensus_meta.cc:287] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.980s	user 0.011s	sys 0.000s
I20260709 13:08:17.634295 26765 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.556360 26775 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:22.216595 26775 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.218475 22826 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:22.218647 26775 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:17.735586 26781 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.033s	sys 0.020s
W20260709 13:08:22.222072 26781 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 5.019s	user 0.036s	sys 0.020s
I20260709 13:08:17.303057 26782 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.228499 26782 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:17.584654 26793 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.638563 26794 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:17.666680 26801 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:08:17.728529 26804 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.817901 26811 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.243577 26811 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.818828 26813 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:21.379925 22829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1731) took 5383 ms (client timeout 1803 ms). Trace:
W20260709 13:08:21.885159 22719 leader_election.cc:336] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.992s (SENT)
W20260709 13:08:22.247745 22829 rpcz_store.cc:269] 0709 13:08:15.996953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.997056 (+   103us) service_pool.cc:224] Handling call
0709 13:08:21.379550 (+5382494us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540008b454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:21.379793 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2763231488}
I20260709 13:08:22.247746 26813 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:21.900612 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.906329 26737 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:21.952632 26720 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:22.249082 26737 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.249691 26720 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:22.250468 22826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1748) took 5183 ms (client timeout 2070 ms). Trace:
I20260709 13:08:22.247849 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:22.251055 26737 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:22.250975 22826 rpcz_store.cc:269] 0709 13:08:17.067556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.067697 (+   141us) service_pool.cc:224] Handling call
0709 13:08:22.250310 (+5182613us) spinlock_profiling.cc:243] Waited 5.12 s on lock 0x7b5400091fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:22.250406 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11784773888}
I20260709 13:08:22.013813 22751 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:22.018289 26721 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:22.288442 26721 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.290478 26721 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:22.036139 26675 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:22.040501 26819 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:22.291189 26819 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:22.291615 26819 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:22.001777 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 4.58239e+06us
I20260709 13:08:22.299389 26819 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:22.052259 22747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 1082) took 4654 ms (client timeout 1914 ms). Trace:
W20260709 13:08:22.301470 22747 rpcz_store.cc:269] 0709 13:08:17.397832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.397990 (+   158us) service_pool.cc:224] Handling call
0709 13:08:22.052040 (+4654050us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b5400406ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:22.052164 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1792825088}
I20260709 13:08:22.301975 26819 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:22.302587 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.303143 22747 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:22.304019 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.070035 22755 rpcz_store.cc:276] 0709 13:08:16.591579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.591707 (+   128us) service_pool.cc:224] Handling call
0709 13:08:16.625082 (+ 33375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:17.338609 (+713527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:19.588992 (+2250383us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54002689d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:19.589238 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590106880}
I20260709 13:08:22.307166 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.074143 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.077029 22654 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 4 to 7, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:22.078058 22749 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:08:22.085768 26770 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:22.309557 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:22.309770 26770 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:22.310372 26770 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.310988 26675 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.310783 22654 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 7 to 9, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 9 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:22.311435 26675 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.312247 26770 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.095378 26824 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:22.097780 22830 rpcz_store.cc:269] 0709 13:08:17.408782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.408936 (+   154us) service_pool.cc:224] Handling call
0709 13:08:19.078620 (+1669684us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:22.312953 26824 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:22.098322 26825 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.313367 26824 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.313534 26675 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.313771 26824 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:22.314918 26825 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:22.084884 26497 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:22.100180 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:22.316419 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 216318us
I20260709 13:08:22.316828 26759 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.317232 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:22.317281 26759 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.103060 26831 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.318723 26689 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:22.319058 26831 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:22.105393 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 4.28426e+06us
I20260709 13:08:22.319859 26591 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:22.319949 26712 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:08:22.107160 22897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 894) took 6985 ms (client timeout 2254 ms). Trace:
I20260709 13:08:22.320194 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:22.320495 26712 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:19.623904 22898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1630) took 2080 ms (client timeout 1924 ms). Trace:
I20260709 13:08:22.108163 22896 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
W20260709 13:08:22.109903 22852 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd560 after lost signal to thread 22780
I20260709 13:08:22.322960 22896 pending_rounds.cc:85] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 10
W20260709 13:08:22.320559 22897 rpcz_store.cc:269] 0709 13:08:15.122226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.122378 (+   152us) service_pool.cc:224] Handling call
0709 13:08:22.106925 (+6984547us) spinlock_profiling.cc:243] Waited 2.68 s on lock 0x7b54003ea654. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:22.107069 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6161643776}
I20260709 13:08:22.323343 22896 pending_rounds.cc:107] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:08:22.323619 22896 raft_consensus.cc:2889] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:22.323915 22852 debug-util.cc:398] Leaking SignalData structure 0x7b080024f8a0 after lost signal to thread 22855
I20260709 13:08:22.324115 22654 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 5 to 7, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 7 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
W20260709 13:08:22.325024 22852 debug-util.cc:398] Leaking SignalData structure 0x7b08001e0580 after lost signal to thread 22929
I20260709 13:08:22.111027 26604 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:22.113389 22653 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 333) took 17953 ms (client timeout 15000 ms). Trace:
W20260709 13:08:22.124552 22904 rpcz_store.cc:269] 0709 13:08:17.539127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.124061 (+4584934us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.124177 (+   116us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.113394 22900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1627) took 4571 ms (client timeout 2015 ms). Trace:
W20260709 13:08:22.129037 22905 rpcz_store.cc:269] 0709 13:08:19.681052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.128576 (+2447524us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.128671 (+    95us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.116915 22901 rpcz_store.cc:269] 0709 13:08:17.541818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.116458 (+4574640us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.116557 (+    99us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.137094 22903 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
W20260709 13:08:22.121147 22902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 912) took 4581 ms (client timeout 2254 ms). Trace:
W20260709 13:08:22.149652 23447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1120) took 4537 ms (client timeout 1781 ms). Trace:
I20260709 13:08:22.152015 22756 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:22.151906 26472 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.151500 22752 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
W20260709 13:08:22.153578 23453 rpcz_store.cc:269] 0709 13:08:17.399385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.399716 (+   331us) service_pool.cc:224] Handling call
0709 13:08:17.893011 (+493295us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b5800176198. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:08:22.153131 (+4260120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22991616}
W20260709 13:08:22.153940 23416 connection.cc:256] RPC call timeout handler was delayed by -1.5164050839999845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
I20260709 13:08:22.154191 26792 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:22.155649 23450 rpcz_store.cc:269] 0709 13:08:17.597252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.597394 (+   142us) service_pool.cc:224] Handling call
0709 13:08:22.155212 (+4557818us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:17.632746 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:22.156951 23448 rpcz_store.cc:269] 0709 13:08:17.666009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.874745 (+208736us) service_pool.cc:224] Handling call
0709 13:08:22.156518 (+4281773us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:22.161422 23452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1176) took 4600 ms (client timeout 1807 ms). Trace:
W20260709 13:08:22.164232 23445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 733) took 4483 ms (client timeout 1661 ms). Trace:
I20260709 13:08:22.172813 26461 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.173797 26594 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:22.180533 26672 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:17.570837 26684 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.198753 26696 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:17.959321 26713 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:22.206421 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 70075.2us
I20260709 13:08:22.214349 26765 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.229390 23449 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:08:22.230773 26793 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.231431 26794 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.242547 26804 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.246039 26811 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:22.005398 26830 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.252646 22899 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:22.272903 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.341485 23452 rpcz_store.cc:269] 0709 13:08:17.557216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.557411 (+   195us) service_pool.cc:224] Handling call
0709 13:08:22.157256 (+4599845us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:22.342471 23452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1180) took 4677 ms (client timeout 1746 ms). Trace:
W20260709 13:08:22.342828 23452 rpcz_store.cc:269] 0709 13:08:17.665305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.342303 (+4676998us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.342411 (+   108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.342999 22753 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:22.343459 22904 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:08:22.343680 23452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1181) took 4678 ms (client timeout 1797 ms). Trace:
W20260709 13:08:22.344084 22900 rpcz_store.cc:269] 0709 13:08:17.542128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.113201 (+4571073us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.113307 (+   106us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.344724 23444 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:22.344451 26594 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.345644 26594 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:22.345794 26793 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:22.346050 23444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1175) took 4793 ms (client timeout 3155 ms). Trace:
W20260709 13:08:22.346458 23444 rpcz_store.cc:269] 0709 13:08:17.553035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.554433 (+  1398us) service_pool.cc:224] Handling call
0709 13:08:22.345881 (+4791448us) spinlock_profiling.cc:243] Waited 4.47 s on lock 0x7b54002233d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:22.345992 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10282061056}
W20260709 13:08:22.347487 23444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1121) took 4719 ms (client timeout 1850 ms). Trace:
I20260709 13:08:22.347678 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.348093 26794 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:22.348397 23451 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:08:22.348863 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.349766 23450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1183) took 4683 ms (client timeout 1905 ms). Trace:
I20260709 13:08:22.349920 26804 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:22.350132 23450 rpcz_store.cc:269] 0709 13:08:17.666363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.349601 (+4683238us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.349695 (+    94us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.350996 23450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 734) took 4669 ms (client timeout 2046 ms). Trace:
I20260709 13:08:22.351231 22903 pending_rounds.cc:85] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 8
I20260709 13:08:22.351577 22903 pending_rounds.cc:107] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:08:22.351967 22903 raft_consensus.cc:2889] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:22.352707 22899 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 918) took 4811 ms (client timeout 1565 ms). Trace:
W20260709 13:08:22.251603 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 366464us
I20260709 13:08:22.256011 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.353715 23416 connection.cc:256] RPC call timeout handler was delayed by -1.4291050840000707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
I20260709 13:08:22.327129 26831 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.354070 22750 raft_consensus.cc:2410] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:08:22.354386 23416 connection.cc:256] RPC call timeout handler was delayed by -1.1951050840000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260709 13:08:22.325891 22852 debug-util.cc:398] Leaking SignalData structure 0x7b080038b680 after lost signal to thread 23451
I20260709 13:08:22.354486 26472 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:22.355280 22852 debug-util.cc:398] Leaking SignalData structure 0x7b08002117a0 after lost signal to thread 23478
W20260709 13:08:22.355880 22852 debug-util.cc:398] Leaking SignalData structure 0x7b0800337140 after lost signal to thread 26676
I20260709 13:08:22.356247 23446 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
W20260709 13:08:22.356628 22852 debug-util.cc:398] Leaking SignalData structure 0x7b080036df40 after lost signal to thread 26706
I20260709 13:08:22.356902 26472 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.254652 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.357309 23446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1177) took 4772 ms (client timeout 2244 ms). Trace:
I20260709 13:08:22.357511 26765 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:22.353523 22719 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.010s (SENT)
I20260709 13:08:22.358659 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:22.358091 26765 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:22.359838 22719 leader_election.cc:336] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.025s (SENT)
I20260709 13:08:22.275699 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.320971 26712 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:22.105230 26693 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:22.321409 26737 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:22.309592 22749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 2259) took 4909 ms (client timeout 2299 ms). Trace:
I20260709 13:08:22.313117 26782 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:08:22.342723 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1117) took 4947 ms. Trace:
I20260709 13:08:22.309710 26775 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:22.344009 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 915) took 4803 ms. Trace:
I20260709 13:08:22.364135 26775 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:08:22.362322 23453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1122) took 4724 ms (client timeout 2095 ms). Trace:
W20260709 13:08:22.365121 23453 rpcz_store.cc:269] 0709 13:08:17.637643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.362143 (+4724500us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.362247 (+   104us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.355906 22750 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:22.362495 26792 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:22.366243 23453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 735) took 4683 ms (client timeout 2070 ms). Trace:
I20260709 13:08:22.319365 26759 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:22.352027 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800147280 after lost signal to thread 21632
I20260709 13:08:22.352753 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2178) took 4895 ms. Trace:
I20260709 13:08:22.361394 22754 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
W20260709 13:08:22.362645 22902 rpcz_store.cc:269] 0709 13:08:17.539400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.120997 (+4581597us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.121102 (+   105us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.367391 22903 rpcz_store.cc:276] 0709 13:08:17.457384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.136417 (+4679033us) service_pool.cc:224] Handling call
0709 13:08:22.136743 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.352549 (+215806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.352711 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:22.368223 26792 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.368340 22754 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:08:22.362597 23447 rpcz_store.cc:269] 0709 13:08:17.611817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.611938 (+   121us) service_pool.cc:224] Handling call
0709 13:08:22.149561 (+4537623us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:22.368891 22902 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:08:22.320329 26591 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:22.369771 26591 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:22.363585 26684 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.352030 26830 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:22.369498 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800371b40 after lost signal to thread 22688
I20260709 13:08:22.372215 26830 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:22.372354 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08001481a0 after lost signal to thread 22749
I20260709 13:08:22.372747 26830 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:22.373044 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800147820 after lost signal to thread 22780
W20260709 13:08:22.369270 23447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1124) took 4549 ms (client timeout 1987 ms). Trace:
I20260709 13:08:22.373420 22755 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:22.343983 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.373814 23447 rpcz_store.cc:269] 0709 13:08:17.819630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.369131 (+4549501us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.369222 (+    91us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.252213 26720 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:22.373366 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800147660 after lost signal to thread 22781
W20260709 13:08:22.362546 23445 rpcz_store.cc:269] 0709 13:08:17.680983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.164045 (+4483062us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.164179 (+   134us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.322885 22898 rpcz_store.cc:269] 0709 13:08:17.542993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.623676 (+2080683us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:19.623810 (+   134us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.375778 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003eac00 after lost signal to thread 22855
W20260709 13:08:22.376209 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080029ff00 after lost signal to thread 22868
I20260709 13:08:22.366298 26775 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:22.364485 26782 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:22.250065 26813 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:22.343142 26672 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.386739 26782 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:22.326836 26497 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.363754 26461 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.388605 22898 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:22.388757 26497 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.388935 26461 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.389619 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:22.390148 22898 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:22.390818 23448 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:08:22.390734 26461 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.390964 26712 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.391556 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1179) took 4726 ms. Trace:
I20260709 13:08:22.361043 26765 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:22.376842 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08002eb020 after lost signal to thread 22897
I20260709 13:08:22.350070 26713 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:22.392287 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1c80 after lost signal to thread 22899
W20260709 13:08:22.392580 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9ee0 after lost signal to thread 22900
W20260709 13:08:22.392867 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003843c0 after lost signal to thread 22902
W20260709 13:08:22.374583 23447 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1184) took 4708 ms (client timeout 2299 ms). Trace:
I20260709 13:08:22.390733 26497 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:22.393443 23447 rpcz_store.cc:269] 0709 13:08:17.666695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.374437 (+4707742us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.374533 (+    96us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.393836 26713 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:22.369251 22902 pending_rounds.cc:85] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 11
I20260709 13:08:22.326118 22654 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 6 to 8, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 8 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:22.394932 22902 pending_rounds.cc:107] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:08:22.395249 22902 raft_consensus.cc:2889] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:22.395911 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2183) took 2714 ms. Trace:
I20260709 13:08:22.396221 22902 rpcz_store.cc:276] 0709 13:08:19.681774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.368294 (+2686520us) service_pool.cc:224] Handling call
0709 13:08:22.368589 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.395704 (+ 27115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.395864 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:22.366734 23453 rpcz_store.cc:269] 0709 13:08:17.682523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.366099 (+4683576us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.366190 (+    91us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.397537 26782 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.398417 23453 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:22.398610 26782 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:22.399163 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 738) took 4715 ms. Trace:
W20260709 13:08:22.348913 23451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1178) took 4737 ms (client timeout 2137 ms). Trace:
I20260709 13:08:22.342496 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 909) took 4803 ms. Trace:
W20260709 13:08:22.400022 23451 rpcz_store.cc:269] 0709 13:08:17.611204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.611500 (+   296us) service_pool.cc:224] Handling call
0709 13:08:22.348851 (+4737351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:22.400194 22905 rpcz_store.cc:276] 0709 13:08:17.538601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.341733 (+4803132us) service_pool.cc:224] Handling call
0709 13:08:22.342023 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.342190 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.342213 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:22.342249 (+    36us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:22.342283 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.342331 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.342423 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:22.401548 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.402040 22905 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:22.371577 26684 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.392244 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.403704 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:22.391911 23448 rpcz_store.cc:276] 0709 13:08:17.664866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.376477 (+4711611us) service_pool.cc:224] Handling call
0709 13:08:22.376772 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.391318 (+ 14546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.391509 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:22.407817 22654 catalog_manager.cc:5697] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 1 to 3, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:22.409439 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:22.351325 23450 rpcz_store.cc:269] 0709 13:08:17.681251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.350852 (+4669601us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.350945 (+    93us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.363858 23449 rpcz_store.cc:276] 0709 13:08:17.394879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.395016 (+   137us) service_pool.cc:224] Handling call
0709 13:08:17.395241 (+   225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.341484 (+4946243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.342519 (+  1035us) spinlock_profiling.cc:243] Waited 4.83 s on lock 0x7b5400750154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:22.342674 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11116567424}
I20260709 13:08:22.366866 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.411082 22750 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:08:22.357097 22852 debug-util.cc:398] Leaking SignalData structure 0x7b08003783e0 after lost signal to thread 26719
I20260709 13:08:22.412019 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.412703 22750 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:08:22.347842 23444 rpcz_store.cc:269] 0709 13:08:17.628466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.347331 (+4718865us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.347435 (+   104us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:08:22.344123 23452 rpcz_store.cc:269] 0709 13:08:17.665674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.343518 (+4677844us) service_pool.cc:211] Skipping call since client already timed out
0709 13:08:22.343634 (+   116us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:08:22.402194 23451 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:08:22.305022 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.363464 22749 rpcz_store.cc:269] 0709 13:08:17.400461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.425484 (+ 25023us) service_pool.cc:224] Handling call
0709 13:08:22.309518 (+4884034us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:22.414018 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1125) took 4594 ms. Trace:
W20260709 13:08:22.357786 23446 rpcz_store.cc:269] 0709 13:08:17.584521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.584662 (+   141us) service_pool.cc:224] Handling call
0709 13:08:22.357166 (+4772504us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54000b13d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:22.357250 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466066304}
I20260709 13:08:22.319218 26689 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.322186 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:22.383589 22751 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:22.383721 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.429970 22903 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:22.384687 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.431051 22755 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:22.432063 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.361569 22899 rpcz_store.cc:269] 0709 13:08:17.541031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.573884 (+ 32853us) service_pool.cc:224] Handling call
0709 13:08:22.352530 (+4778646us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400275454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:22.352642 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332106624}
I20260709 13:08:22.392668 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:22.393124 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800387b80 after lost signal to thread 22904
W20260709 13:08:22.341470 22797 connection.cc:256] RPC call timeout handler was delayed by -2.8288419809999823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2299s for a response
I20260709 13:08:22.414387 23451 rpcz_store.cc:276] 0709 13:08:17.820003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.401565 (+4581562us) service_pool.cc:224] Handling call
0709 13:08:22.401915 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.413807 (+ 11892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.413970 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:22.362848 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:22.433053 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6d00 after lost signal to thread 22905
W20260709 13:08:22.433874 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6960 after lost signal to thread 22914
W20260709 13:08:22.434486 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080031fd80 after lost signal to thread 22929
I20260709 13:08:22.435024 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:22.435534 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080022dc20 after lost signal to thread 23445
I20260709 13:08:22.434939 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:22.341500 22653 rpcz_store.cc:269] 0709 13:08:04.149978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.150157 (+   179us) service_pool.cc:224] Handling call
0709 13:08:22.102765 (+17952608us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:16.029041 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:16.063495 (+ 34454us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:16.063559 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:08:16.063574 (+    15us) tablet.cc:662] Decoding operations
0709 13:08:16.064799 (+  1225us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:16.064880 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:16.064911 (+    31us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:08:16.065911 (+  1000us) tablet.cc:701] Row locks acquired
0709 13:08:16.065972 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:08:16.066153 (+   181us) write_op.cc:270] Start()
0709 13:08:16.066297 (+   144us) write_op.cc:276] Timestamp: P: 1783602496066088 usec, L: 0
0709 13:08:16.066319 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:08:16.067354 (+  1035us) log.cc:844] Serialized 3960 byte log entry
0709 13:08:17.147594 (+1080240us) op_driver.cc:464] REPLICATION: finished
0709 13:08:17.386868 (+239274us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb71cf10ac 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb78831ef0
0709 13:08:17.750085 (+363217us) write_op.cc:301] APPLY: starting
0709 13:08:17.750186 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:17.750415 (+   229us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:17.750452 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:17.751848 (+  1396us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:17.752100 (+   252us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:08:17.752149 (+    49us) write_op.cc:312] APPLY: finished
0709 13:08:17.752444 (+   295us) log.cc:844] Serialized 77 byte log entry
0709 13:08:17.752612 (+   168us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:17.752749 (+   137us) write_op.cc:454] Released schema lock
0709 13:08:17.752796 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:22.222423 (+4469627us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:22.222460 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":75172,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":596086,"apply.run_cpu_time_us":3045,"apply.run_wall_time_us":3046,"num_ops":10,"prepare.queue_time_us":435164,"prepare.run_cpu_time_us":21112,"prepare.run_wall_time_us":202651,"raft.queue_time_us":424354,"raft.run_cpu_time_us":57973,"raft.run_wall_time_us":5156661,"replication_time_us":1012361,"spinlock_wait_cycles":47006976,"thread_start_us":5602800,"threads_started":4,"wal-append.queue_time_us":174257,"wal-append.run_cpu_time_us":104357,"wal-append.run_wall_time_us":4726397}]]}
W20260709 13:08:22.417063 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:22.419327 22852 debug-util.cc:398] Leaking SignalData structure 0x7b08003912e0 after lost signal to thread 26755
I20260709 13:08:22.409382 22654 catalog_manager.cc:5697] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 4 to 6, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 6 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:22.437292 22755 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:22.399586 23453 rpcz_store.cc:276] 0709 13:08:17.683446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.397893 (+4714447us) service_pool.cc:224] Handling call
0709 13:08:22.398215 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.398899 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.399103 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:22.438277 22825 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:08:22.433494 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 70717.1us
I20260709 13:08:22.419366 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.326715 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 899) took 6769 ms. Trace:
I20260709 13:08:22.435673 22753 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:22.364329 22904 rpcz_store.cc:276] 0709 13:08:17.540229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.342969 (+4802740us) service_pool.cc:224] Handling call
0709 13:08:22.343269 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.343793 (+   524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.343963 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:22.436252 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08002cfda0 after lost signal to thread 23446
W20260709 13:08:22.440476 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800098680 after lost signal to thread 23447
W20260709 13:08:22.440738 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003973a0 after lost signal to thread 23448
W20260709 13:08:22.440939 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08000855e0 after lost signal to thread 23450
W20260709 13:08:22.441118 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08002873a0 after lost signal to thread 23451
W20260709 13:08:22.441294 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800386160 after lost signal to thread 23452
W20260709 13:08:22.441789 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003c05c0 after lost signal to thread 23453
I20260709 13:08:22.392999 26712 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:22.439323 22750 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
W20260709 13:08:22.442521 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080022eba0 after lost signal to thread 23478
W20260709 13:08:22.449540 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080034da40 after lost signal to thread 26244
W20260709 13:08:22.449877 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800340ae0 after lost signal to thread 26431
W20260709 13:08:22.450140 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08002bdb00 after lost signal to thread 26459
W20260709 13:08:22.450387 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800391760 after lost signal to thread 26461
W20260709 13:08:22.450691 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003db380 after lost signal to thread 26478
W20260709 13:08:22.450989 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003ebea0 after lost signal to thread 26497
I20260709 13:08:22.447273 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1123) took 4787 ms. Trace:
I20260709 13:08:22.438926 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.451284 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003c4c60 after lost signal to thread 26600
I20260709 13:08:22.439229 22896 rpcz_store.cc:276] 0709 13:08:15.556828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.556963 (+   135us) service_pool.cc:224] Handling call
0709 13:08:22.107860 (+6550897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.324047 (+216187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.326480 (+  2433us) spinlock_profiling.cc:243] Waited 6.55 s on lock 0x7b54003ea650. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:22.326627 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15058729728}
W20260709 13:08:22.451617 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08001a1d80 after lost signal to thread 26666
W20260709 13:08:22.451857 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080014b980 after lost signal to thread 26667
W20260709 13:08:22.452062 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800374480 after lost signal to thread 26672
W20260709 13:08:22.452396 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003155a0 after lost signal to thread 26675
W20260709 13:08:22.452744 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08001e4720 after lost signal to thread 26684
W20260709 13:08:22.453208 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08002c38c0 after lost signal to thread 26688
W20260709 13:08:22.453630 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080004ed20 after lost signal to thread 26689
W20260709 13:08:22.453861 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003c4160 after lost signal to thread 26691
I20260709 13:08:22.452147 23445 rpcz_store.cc:276] 0709 13:08:17.660531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.375477 (+4714946us) service_pool.cc:224] Handling call
0709 13:08:22.375773 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.376247 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.376272 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.376654 (+   382us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.443868 (+ 67214us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:22.443905 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.443970 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:22.446949 (+  2979us) raft_consensus.cc:1685] finished
0709 13:08:22.447090 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.447210 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15064,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"thread_start_us":66872,"threads_started":1,"wal-append.queue_time_us":67230}
W20260709 13:08:22.454078 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080014e480 after lost signal to thread 26694
W20260709 13:08:22.454764 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080038d3a0 after lost signal to thread 26712
W20260709 13:08:22.454988 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003a5340 after lost signal to thread 26713
W20260709 13:08:22.455328 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800373fa0 after lost signal to thread 26731
W20260709 13:08:22.455546 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080023f0e0 after lost signal to thread 26734
W20260709 13:08:22.455839 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08002feda0 after lost signal to thread 26738
W20260709 13:08:22.456044 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08002227e0 after lost signal to thread 26741
W20260709 13:08:22.456310 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003d3fa0 after lost signal to thread 26755
W20260709 13:08:22.456518 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800304960 after lost signal to thread 26759
W20260709 13:08:22.456792 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800320d00 after lost signal to thread 26765
W20260709 13:08:22.457098 22926 debug-util.cc:398] Leaking SignalData structure 0x7b0800125740 after lost signal to thread 26775
W20260709 13:08:22.457401 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003c74a0 after lost signal to thread 26781
W20260709 13:08:22.457685 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003f19c0 after lost signal to thread 26782
W20260709 13:08:22.457998 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6580 after lost signal to thread 26793
W20260709 13:08:22.447141 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 128958us
I20260709 13:08:22.459102 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:22.459440 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080031f200 after lost signal to thread 26794
W20260709 13:08:22.460048 22926 debug-util.cc:398] Leaking SignalData structure 0x7b080037a560 after lost signal to thread 26801
W20260709 13:08:22.460295 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08002bda60 after lost signal to thread 26804
W20260709 13:08:22.462857 23416 leader_election.cc:336] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 2.254s (SENT)
W20260709 13:08:22.463721 23416 connection.cc:256] RPC call timeout handler was delayed by -2.47916249299999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29900000000000004s for a response
W20260709 13:08:22.464306 23416 connection.cc:256] RPC call timeout handler was delayed by -2.37635831700004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2983s for a response
W20260709 13:08:22.464682 23416 connection.cc:256] RPC call timeout handler was delayed by -2.37635831700004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2983s for a response
W20260709 13:08:22.465487 23416 connection.cc:256] RPC call timeout handler was delayed by -1.9331249230000367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3371s for a response
I20260709 13:08:22.467803 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.468859 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.469401 22750 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:22.469483 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.470062 22831 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:22.470337 22755 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:22.470932 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.471104 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.471482 22831 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:22.472882 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.481534 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:22.481405 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 159267us
I20260709 13:08:22.482347 22825 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 6: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:22.482609 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:22.483112 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.487438 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.487507 22747 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:22.483136 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:22.473778 26472 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:22.448499 26672 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:22.438433 26734 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:08:22.498899 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 138303us
I20260709 13:08:22.503544 22753 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:22.467417 26813 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:22.510767 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.512017 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.513134 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:22.514223 22719 leader_election.cc:336] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.040s (SENT)
I20260709 13:08:22.514832 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:22.514237 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.516531 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:22.520745 22756 raft_consensus.cc:2393] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:22.521369 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.491982 26472 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.458564 26591 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.526402 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.495975 26672 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.530074 22755 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:22.526898 22749 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:22.467307 26765 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:08:22.537591 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.540398 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:22.541134 22751 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:22.542670 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.489528 26737 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:08:22.546753 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:22.597705 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51008us
I20260709 13:08:22.598523 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:22.508536 26794 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:08:22.548707 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:22.550956 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51080.9us
I20260709 13:08:22.607331 22871 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:22.562688 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:22.571250 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.593492 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.493191 26775 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:08:22.496137 26819 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:08:22.492151 26497 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:08:22.514624 26738 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:22.525684 26712 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:22.525946 26472 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:22.526702 26721 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:22.596323 22824 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:22.547480 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.540125 26672 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:22.613179 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:22.550082 26693 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:22.523008 26825 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
W20260709 13:08:22.622416 22719 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.084s (SENT)
I20260709 13:08:22.622298 22751 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:22.610411 26811 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:22.615186 26696 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:08:22.658174 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 125890us
I20260709 13:08:22.621109 26741 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:22.673144 22749 raft_consensus.cc:2410] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:08:22.710597 26691 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:08:22.686463 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 73337.7us
W20260709 13:08:22.692171 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.715929 22824 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:22.716845 26591 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:22.721846 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:22.740053 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:22.692584 26693 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.682962 22898 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
W20260709 13:08:22.715974 26842 log.cc:927] Time spent T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:08:22.745712 26696 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:22.758179 26844 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 13:08:22.758951 26849 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:08:22.755686 26741 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.793567 26741 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:22.764537 22901 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:22.797025 26472 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.768492 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.797976 23453 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:22.657650 26824 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.799300 23447 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 15. Ops: []
I20260709 13:08:22.799538 26472 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.800350 26824 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.800639 26755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 135e2f7520a34b53aaf4ddb360a8a029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:22.800750 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 736) took 5118 ms. Trace:
I20260709 13:08:22.801182 23447 rpcz_store.cc:276] 0709 13:08:17.682986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.394281 (+4711295us) service_pool.cc:224] Handling call
0709 13:08:22.394590 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.799657 (+405067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.800523 (+   866us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b540055a854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:22.800688 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":930302336}
I20260709 13:08:22.767722 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:22.773350 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.766558 26691 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.805647 26691 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:22.806360 26843 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:08:22.807555 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1631) took 5264 ms. Trace:
I20260709 13:08:22.808008 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:22.756034 26604 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:08:22.810760 26604 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:22.811192 26604 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:22.811653 22830 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:22.811985 26604 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.812422 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.812937 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.775543 26696 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:22.813956 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:22.807878 22900 rpcz_store.cc:276] 0709 13:08:17.543308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.345041 (+4801733us) service_pool.cc:224] Handling call
0709 13:08:22.345352 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.345895 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.345921 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.346364 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.496894 (+150530us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:22.496931 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.497131 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:22.807353 (+310222us) raft_consensus.cc:1685] finished
0709 13:08:22.807372 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.807504 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":150184,"threads_started":1,"wal-append.queue_time_us":276252}
I20260709 13:08:22.814685 22824 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:22.815857 26804 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:22.817718 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.818717 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.819607 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.820233 22830 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:22.820858 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.809120 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:22.821375 22824 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:22.822150 26594 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:22.822510 26594 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:22.822616 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:22.822834 26824 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:22.823288 26824 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:22.823746 26824 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:22.823395 26321 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:22.756220 26688 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:08:22.771672 26689 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:22.824571 26792 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:22.824973 26792 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:22.822588 22824 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:22.825796 26792 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.826942 26792 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:22.827296 26827 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:22.817246 26804 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:22.827845 26827 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:22.829157 26827 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.827960 26853 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:22.830770 26853 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:22.831487 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 282810us
I20260709 13:08:22.831609 26853 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.832792 26853 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:22.832644 22797 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:22.834174 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 908) took 5296 ms. Trace:
I20260709 13:08:22.834508 22827 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:22.834551 22897 rpcz_store.cc:276] 0709 13:08:17.538280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.323668 (+4785388us) service_pool.cc:224] Handling call
0709 13:08:22.323989 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.324309 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.324336 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.324761 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.497443 (+172682us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:22.497481 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.497611 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:22.833994 (+336383us) raft_consensus.cc:1685] finished
0709 13:08:22.834014 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:22.834123 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":61,"thread_start_us":170751,"threads_started":1,"wal-append.queue_time_us":224732}
I20260709 13:08:22.836645 26497 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:22.837455 26712 raft_consensus.cc:697] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:22.840720 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:22.773743 26693 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:22.841939 26794 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:22.842355 26794 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:22.843353 26794 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:22.844609 26794 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:22.846037 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.835984 22827 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:22.843811 26775 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:22.848892 22830 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:22.848950 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.852952 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.853641 22903 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
W20260709 13:08:22.863896 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 241500us
I20260709 13:08:22.865167 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.865751 22900 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:22.864857 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2179) took 3184 ms. Trace:
I20260709 13:08:22.858326 22829 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Committing config change with OpId 8.7: config changed from index -1 to 7, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:08:22.869155 22901 rpcz_store.cc:276] 0709 13:08:19.680340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.359917 (+2679577us) service_pool.cc:224] Handling call
0709 13:08:22.360198 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.797464 (+437266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.859918 (+ 62454us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54002dd7d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:22.864787 (+  4869us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":920031616}
I20260709 13:08:22.879688 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.881081 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.842221 26770 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:22.882387 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.882668 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.883085 22896 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:22.883527 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.883939 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:22.884078 22904 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:22.884472 22905 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:22.837201 26497 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.885366 23447 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:22.900913 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.882411 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:22.883213 22897 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:22.884075 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.891299 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.902629 22827 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
W20260709 13:08:22.900046 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53106us
I20260709 13:08:22.899766 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:22.904136 22719 leader_election.cc:336] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.160s (SENT)
I20260709 13:08:22.902032 23446 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:22.906411 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:22.908900 26721 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:08:22.910060 22719 connection.cc:256] RPC call timeout handler was delayed by -1.2231475000000955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2493s for a response
W20260709 13:08:22.912168 22719 connection.cc:256] RPC call timeout handler was delayed by -1.2060475000000679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25120000000000003s for a response
W20260709 13:08:22.913932 22719 connection.cc:256] RPC call timeout handler was delayed by -1.1025475000001279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2627s for a response
I20260709 13:08:22.917532 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:22.918133 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:22.915642 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.920594 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.921526 23445 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:22.922034 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.922703 23451 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 6.
I20260709 13:08:22.904238 22899 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
W20260709 13:08:22.922291 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:22.916507 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.930006 22827 raft_consensus.cc:1217] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Deduplicated request from leader. Original: 9.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 13:08:22.930987 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.931996 22824 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:22.915557 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:22.934198 22827 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:08:22.869035 26830 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:22.935488 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:22.938582 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.940784 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:22.946538 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.947492 22824 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:22.948074 22830 raft_consensus.cc:2410] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:22.823354 26594 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:22.949812 26594 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:22.950740 26860 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:22.951301 26860 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:22.951706 26860 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:22.953411 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:22.899415 26770 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.824587 26321 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:08:22.882694 26775 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:22.959872 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:22.960927 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:22.962023 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.903216 26497 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:22.966779 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:22.967617 22824 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:22.972036 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.977921 22827 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:22.979574 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:22.980638 22750 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:22.979993 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:22.980633 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.983047 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:22.982462 22751 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:22.988420 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:22.989266 22898 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:22.991351 22748 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:22.978756 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.964462 26770 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:22.978613 26810 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:22.974234 26830 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:23.012032 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:22.992482 22827 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:22.976702 26734 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:22.978747 26461 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:23.030686 22751 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:23.021631 22824 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:23.046952 22750 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:23.019816 26810 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:23.057757 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.061669 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:08:23.064409 22868 connection.cc:256] RPC call timeout handler was delayed by -3.6912300550001191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
I20260709 13:08:23.033797 26461 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:23.064818 22868 connection.cc:256] RPC call timeout handler was delayed by -3.5877300550000655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
I20260709 13:08:23.025520 26734 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:23.065158 22868 connection.cc:256] RPC call timeout handler was delayed by -3.5679300550000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
I20260709 13:08:23.067087 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:23.047370 26782 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:23.069012 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:23.109992 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:08:23.091620 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:23.111231 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44236.2us
I20260709 13:08:23.012426 26672 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:23.159636 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 198789us
I20260709 13:08:23.160522 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:23.162102 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 52153.9us
I20260709 13:08:23.163234 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:22.995126 26712 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:23.166411 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:23.169867 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:23.059307 26810 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:23.171886 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:23.174866 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.068181 26832 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:23.068332 26688 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.069793 26461 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:23.139390 26734 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:23.112428 26782 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:23.192159 23452 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:08:23.193658 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 240343us
W20260709 13:08:23.198391 22825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.030s	sys 0.039s
I20260709 13:08:23.198359 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:08:23.199263 22825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.031s	sys 0.040s
I20260709 13:08:23.166075 26591 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:23.207254 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:23.207448 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:23.208030 22830 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:23.205454 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.108718 26689 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:08:23.221501 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.184253 26738 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:23.054903 26775 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:23.190541 26832 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:23.230201 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 73361.6us
W20260709 13:08:23.229852 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.231467 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:23.202956 26782 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:23.204737 26830 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:23.166567 26684 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 13:08:23.248327 26862 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:08:23.251039 26472 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:23.266355 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:23.299856 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:23.231942 26813 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:23.315816 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.205612 26688 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:23.314193 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:23.228749 26720 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:23.235229 26832 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:23.329658 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.237776 26873 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:08:23.331771 26854 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:08:23.331900 26672 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:23.236449 26738 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:23.359575 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:23.253937 26830 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:23.311870 26591 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:23.287333 26472 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:23.372115 26459 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:23.406126 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:23.373565 26852 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.348s	user 0.000s	sys 0.001s
I20260709 13:08:23.374696 26873 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:23.406100 26472 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:23.379503 22643 maintenance_manager.cc:419] P 359166ef74e042839c4e58133432df83: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033397
W20260709 13:08:23.409055 26870 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:08:23.423683 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2705) took 1040 ms. Trace:
I20260709 13:08:23.424073 22754 rpcz_store.cc:276] 0709 13:08:22.382954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.438546 (+ 55592us) service_pool.cc:224] Handling call
0709 13:08:22.438846 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.474040 (+ 35194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.474074 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.474505 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.626019 (+151514us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:22.626055 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.626131 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.423486 (+797355us) raft_consensus.cc:1685] finished
0709 13:08:23.423508 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.423630 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":175,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":13456,"thread_start_us":114774,"threads_started":1,"wal-append.queue_time_us":453472}
W20260709 13:08:23.426581 26824 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.000s
W20260709 13:08:23.420375 26863 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:08:23.406805 26591 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:23.406651 26830 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:23.333478 26720 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:23.406440 26864 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
W20260709 13:08:23.449549 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 247864us
W20260709 13:08:23.416326 26862 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:08:23.414522 26672 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:23.414016 26876 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:08:23.433820 22905 consensus_meta.cc:287] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.007s
I20260709 13:08:23.497879 22905 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:23.449375 26591 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:23.455317 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2471) took 1021 ms. Trace:
I20260709 13:08:23.497514 26876 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:23.506841 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.525120 22898 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
W20260709 13:08:23.502931 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 238347us
I20260709 13:08:23.454003 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:23.441687 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:23.526846 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72933.7us
I20260709 13:08:23.528131 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:23.520833 26860 consensus_meta.cc:287] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.013s
I20260709 13:08:23.527456 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:23.531019 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:23.531714 22905 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:23.532249 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:23.532632 23419 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.009s	sys 0.025s
W20260709 13:08:23.534348 23419 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.009s	sys 0.025s
I20260709 13:08:23.531729 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2470) took 1097 ms. Trace:
I20260709 13:08:23.505831 22822 rpcz_store.cc:276] 0709 13:08:22.434252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.434395 (+   143us) service_pool.cc:224] Handling call
0709 13:08:22.473421 (+ 39026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.493587 (+ 20166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.493613 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.506218 (+ 12605us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.920457 (+414239us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:23.454664 (+534207us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.454986 (+   322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.455075 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.455123 (+    48us) raft_consensus.cc:1685] finished
0709 13:08:23.455139 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.455265 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24593,"prepare.queue_time_us":56433,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":22869,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":57041,"thread_start_us":853004,"threads_started":2,"wal-append.queue_time_us":507025}
W20260709 13:08:23.534637 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 550512us
I20260709 13:08:23.535423 22829 rpcz_store.cc:276] 0709 13:08:22.434003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.434652 (+   649us) service_pool.cc:224] Handling call
0709 13:08:22.469182 (+ 34530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.470248 (+  1066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.470274 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.482520 (+ 12246us) log.cc:844] Serialized 499 byte log entry
0709 13:08:22.858217 (+375697us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:22.967114 (+108897us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.967460 (+   346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.967536 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.531533 (+563997us) raft_consensus.cc:1685] finished
0709 13:08:23.531552 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.531667 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":176,"thread_start_us":339166,"threads_started":1,"wal-append.queue_time_us":487477}
I20260709 13:08:23.543041 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:23.544878 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:23.449191 26824 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:23.554189 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.496304 26672 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:23.551800 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.542893 26860 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:23.622771 26860 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:23.625746 23453 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:08:23.635644 23417 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.017s	sys 0.054s
W20260709 13:08:23.636152 23417 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.017s	sys 0.054s
I20260709 13:08:23.531890 26876 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:23.538324 26871 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:08:23.571913 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.640195 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.659384 23451 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:08:23.555877 26857 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:08:23.615808 26696 consensus_meta.cc:287] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.802s	user 0.010s	sys 0.007s
I20260709 13:08:23.551000 26879 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:23.553764 22905 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:23.658098 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.548822 26713 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:23.626806 26672 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:23.674114 26691 consensus_meta.cc:287] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.868s	user 0.011s	sys 0.007s
W20260709 13:08:23.707108 26741 consensus_meta.cc:287] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.913s	user 0.013s	sys 0.006s
I20260709 13:08:23.707242 26720 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:23.669808 26696 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:23.706233 26853 consensus_meta.cc:287] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.873s	user 0.008s	sys 0.010s
I20260709 13:08:23.703682 26738 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:23.718560 26874 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 13:08:23.733400 26875 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:08:23.735298 26879 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:23.737229 26876 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:23.739706 26876 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:23.775053 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 452031us
I20260709 13:08:23.781431 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:23.553133 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:23.788916 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:23.547899 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:23.794263 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 246287us
I20260709 13:08:23.796653 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:23.798677 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 712) took 1271 ms. Trace:
I20260709 13:08:23.790798 22754 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:23.799233 22823 rpcz_store.cc:276] 0709 13:08:22.526795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.527169 (+   374us) service_pool.cc:224] Handling call
0709 13:08:22.527460 (+   291us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:22.527796 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.527824 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:22.550248 (+ 22424us) log.cc:844] Serialized 52 byte log entry
0709 13:08:23.054410 (+504162us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:23.166421 (+112011us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.497386 (+330965us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.500331 (+  2945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:23.500431 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.798005 (+297574us) raft_consensus.cc:1685] finished
0709 13:08:23.798054 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.798416 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":310434,"prepare.queue_time_us":136488,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":62,"raft.queue_time_us":75446,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":57080,"spinlock_wait_cycles":45916416,"thread_start_us":663759,"threads_started":2,"wal-append.queue_time_us":806682}
I20260709 13:08:23.659874 22898 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:23.640466 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 97466.7us
I20260709 13:08:23.826992 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:23.830901 26910 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:23.828860 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:23.839349 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.842655 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:23.843483 22754 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:23.845427 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:23.846509 22747 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:08:23.850407 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69354.1us
I20260709 13:08:23.832914 26910 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:23.852177 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:23.710858 26741 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:23.862298 23444 raft_consensus.cc:2410] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:08:23.874300 26909 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:23.874929 26909 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:23.875382 26909 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:23.859301 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:23.881691 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:23.882292 22902 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:23.883178 22830 consensus_meta.cc:287] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.663s	user 0.006s	sys 0.024s
I20260709 13:08:23.885104 22830 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
W20260709 13:08:23.886353 26594 consensus_meta.cc:287] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.936s	user 0.007s	sys 0.005s
W20260709 13:08:23.889812 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 337233us
I20260709 13:08:23.890115 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:23.891172 22905 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:23.893764 26900 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:23.894529 26900 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:23.895176 26900 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:23.897701 26869 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:23.899134 26869 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:23.899668 26869 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:23.901806 26675 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:23.909713 26824 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:23.917959 26794 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.073s	user 0.011s	sys 0.000s
W20260709 13:08:23.918776 26904 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:08:23.698182 26691 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:23.919276 26693 consensus_meta.cc:287] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.077s	user 0.009s	sys 0.002s
I20260709 13:08:23.919831 26693 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:23.894867 26804 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:23.917874 26741 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:23.921332 22899 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:23.948206 26912 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:08:23.923223 26770 consensus_meta.cc:287] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.898s	user 0.018s	sys 0.000s
I20260709 13:08:23.953722 26770 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:23.956725 26770 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:23.929069 26869 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:23.959105 26869 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:23.962052 26804 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:23.962500 26804 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:23.963001 26696 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:23.963896 26804 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:23.965517 26804 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:23.977573 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 148962us
I20260709 13:08:23.982683 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:23.991110 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:23.921238 26691 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:23.993713 26819 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:08:23.997498 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.013010 26688 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:24.001647 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1133) took 1121 ms. Trace:
I20260709 13:08:24.001591 26910 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:24.004168 26912 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.004753 26794 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:24.007278 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:24.008069 26713 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:24.018705 22828 raft_consensus.cc:2410] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:08:23.926368 26831 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:24.029083 26880 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:24.031117 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:24.000239 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1136) took 1060 ms. Trace:
I20260709 13:08:24.035392 23452 rpcz_store.cc:276] 0709 13:08:22.879652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.880854 (+  1202us) service_pool.cc:224] Handling call
0709 13:08:24.001185 (+1120331us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54006329d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:24.001463 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713309440}
I20260709 13:08:24.042207 23453 rpcz_store.cc:276] 0709 13:08:22.939636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.939829 (+   193us) service_pool.cc:224] Handling call
0709 13:08:23.999654 (+1059825us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b5400920254. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:23.999926 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1501180800}
I20260709 13:08:24.039372 26819 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:24.041790 26831 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:24.056926 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:24.060014 26831 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:24.070448 26672 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:24.071664 23444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1131) took 1192 ms (client timeout 1578 ms). Trace:
W20260709 13:08:24.072165 23444 rpcz_store.cc:269] 0709 13:08:22.879082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.879189 (+   107us) service_pool.cc:224] Handling call
0709 13:08:24.071339 (+1192150us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b5400583754. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:24.071589 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2120737024}
I20260709 13:08:24.072649 26672 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.077409 26713 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.078639 26459 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:24.079156 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:24.080056 26713 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:24.080129 26459 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:24.068572 26794 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:24.084308 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:24.100672 26693 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:24.105587 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 141648us
I20260709 13:08:24.092301 26794 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.115761 26882 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:08:24.118294 26782 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.851s	user 0.006s	sys 0.006s
W20260709 13:08:24.118460 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.119060 26782 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.121707 26892 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:08:24.122516 26734 consensus_meta.cc:287] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.883s	user 0.012s	sys 0.015s
I20260709 13:08:24.123306 26734 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:24.123462 26819 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.129277 26900 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:24.130050 26811 consensus_meta.cc:287] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.374s	user 0.011s	sys 0.003s
I20260709 13:08:24.130498 26811 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:24.131491 26900 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.091794 26880 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.066042 26912 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.161018 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:24.161257 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:24.162300 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.160552 26919 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.122646 26892 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:24.171538 26830 consensus_meta.cc:287] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.716s	user 0.008s	sys 0.007s
I20260709 13:08:24.172549 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.175185 26912 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:24.178081 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1138) took 1223 ms. Trace:
I20260709 13:08:24.168725 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:24.184659 23451 rpcz_store.cc:276] 0709 13:08:22.954246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.954541 (+   295us) service_pool.cc:224] Handling call
0709 13:08:24.177674 (+1223133us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b5400620f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:24.177888 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553441536}
W20260709 13:08:24.177101 26461 consensus_meta.cc:287] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.975s	user 0.015s	sys 0.009s
I20260709 13:08:24.191268 23445 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:24.191917 26919 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.192727 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:24.193970 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:24.194051 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 342046us
I20260709 13:08:24.194885 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:24.201241 26782 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:24.202111 26792 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.375s	user 0.011s	sys 0.006s
W20260709 13:08:24.201383 22901 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.783s	user 0.020s	sys 0.002s
W20260709 13:08:24.203241 26591 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.000s
I20260709 13:08:24.196796 26830 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:24.201984 26461 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.203186 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:24.222254 26926 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:08:24.243546 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47005us
W20260709 13:08:24.199965 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 118739us
I20260709 13:08:24.210212 26892 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.185969 26734 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:24.307962 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 123758us
I20260709 13:08:24.312157 22901 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:24.312314 26321 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:24.312973 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.318037 23452 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:08:24.320116 23452 pending_rounds.cc:85] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 7
I20260709 13:08:24.324138 22905 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:24.322362 23452 pending_rounds.cc:107] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:08:24.329757 23452 raft_consensus.cc:2889] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:24.329188 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2200) took 1078 ms. Trace:
I20260709 13:08:24.331902 22901 rpcz_store.cc:276] 0709 13:08:23.249745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.405806 (+156061us) service_pool.cc:224] Handling call
0709 13:08:24.324085 (+918279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.333081 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.333673 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2716) took 1520 ms. Trace:
I20260709 13:08:24.334040 22753 rpcz_store.cc:276] 0709 13:08:22.813626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.815235 (+  1609us) service_pool.cc:224] Handling call
0709 13:08:22.815512 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.869932 (+ 54420us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:22.933049 (+ 63117us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.178900 (+1245851us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b180049bca0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b23a 00007fcb74e05a2a
0709 13:08:24.187666 (+  8766us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:24.188243 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:08:24.188397 (+   154us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:24.188428 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.188541 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.333248 (+144707us) raft_consensus.cc:1685] finished
0709 13:08:24.333378 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.333491 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54065,"prepare.queue_time_us":67236,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":71,"raft.queue_time_us":67456,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":2461,"spinlock_wait_cycles":7131904,"thread_start_us":8423,"threads_started":1,"wal-append.queue_time_us":1372420}
I20260709 13:08:24.335873 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:24.336475 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:24.336824 23449 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:24.335371 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.328738 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2196) took 1077 ms. Trace:
I20260709 13:08:24.338552 22905 rpcz_store.cc:276] 0709 13:08:23.248223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.871491 (+623268us) service_pool.cc:224] Handling call
0709 13:08:24.325621 (+454130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.343384 26878 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:24.335196 22752 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:24.326586 26321 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:24.363617 22753 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:24.364265 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:24.365260 26880 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:24.366366 26919 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.367115 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:24.368696 26321 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.369349 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:24.370038 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.370393 26909 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:24.371428 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:24.371570 23446 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:24.372726 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2192) took 1132 ms. Trace:
I20260709 13:08:24.373148 23447 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:24.320264 26591 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.376233 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:24.366951 26832 consensus_meta.cc:287] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.003s	user 0.000s	sys 0.019s
I20260709 13:08:24.376807 23445 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:24.376478 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.383695 22750 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:24.353631 26830 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:24.382684 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 58108.4us
I20260709 13:08:24.377966 22897 rpcz_store.cc:276] 0709 13:08:23.240150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.853145 (+612995us) service_pool.cc:224] Handling call
0709 13:08:23.853678 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.162891 (+309213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.162935 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.162988 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:24.163039 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.372392 (+209353us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54001b6a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:24.372433 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.372540 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588721536}
I20260709 13:08:24.377537 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:24.386144 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 66065.8us
W20260709 13:08:24.390452 26921 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:08:24.391177 26927 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:24.391592 23450 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:24.391810 26927 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:24.392877 26927 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.394577 26721 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:24.394995 26721 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:24.395953 26721 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.396824 26765 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:24.390452 26924 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:24.411108 26919 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:24.407438 26832 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.391646 26878 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:24.414377 26832 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:24.414366 26878 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:24.415630 22899 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 938) took 1227 ms (client timeout 1610 ms). Trace:
W20260709 13:08:24.416085 22899 rpcz_store.cc:269] 0709 13:08:23.188622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.189082 (+   460us) service_pool.cc:224] Handling call
0709 13:08:24.415289 (+1226207us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b540035ded4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:24.415515 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635375360}
I20260709 13:08:24.418141 23452 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:08:24.418944 22748 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:24.424947 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:24.426951 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:24.428061 23450 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:08:24.434492 23445 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:24.437850 22900 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:08:24.434805 26919 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:08:24.438896 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:24.439980 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:24.450672 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:24.451402 22829 raft_consensus.cc:2376] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11. Re-sending same reply.
I20260709 13:08:24.451742 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:24.452178 26675 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.452397 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:24.452699 26675 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.454278 26591 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:24.454700 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:24.455644 26765 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:24.457581 26461 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.455119 26591 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:24.460386 26591 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:24.457598 26819 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:24.462497 26819 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:24.463598 26819 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.467833 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:24.460434 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2186) took 1221 ms. Trace:
I20260709 13:08:24.461869 26591 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.474326 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1123) took 1444 ms. Trace:
I20260709 13:08:24.474848 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:24.474700 22749 rpcz_store.cc:276] 0709 13:08:22.995538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.020966 (+ 25428us) service_pool.cc:224] Handling call
0709 13:08:23.021262 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:24.081990 (+1060728us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b18003b1220. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb749a34bc 00007fcb71cf148b 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0b455 00007fcb74df3060 00007fcb74df3bfb 00007fcb74dfe64b
0709 13:08:24.082200 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.082226 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:24.082812 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:08:24.095089 (+ 12277us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:24.095136 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.095267 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.423422 (+328155us) raft_consensus.cc:1685] finished
0709 13:08:24.423438 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.440035 (+ 16597us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89568,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":10486,"spinlock_wait_cycles":34853120,"thread_start_us":157259,"threads_started":2,"wal-append.queue_time_us":12325}
I20260709 13:08:24.477577 22903 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:24.469244 22896 rpcz_store.cc:276] 0709 13:08:23.238480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.240584 (+  2104us) service_pool.cc:224] Handling call
0709 13:08:23.247826 (+  7242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:23.501955 (+254129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:23.501993 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.502034 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:23.502072 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.459997 (+957925us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b5400207e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:24.460068 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.460240 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583993088}
I20260709 13:08:24.478464 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:24.479130 22897 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:24.479298 26831 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:24.479740 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:24.480849 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 711) took 1971 ms. Trace:
I20260709 13:08:24.481037 26912 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:24.481233 22902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 934) took 1628 ms (client timeout 1524 ms). Trace:
I20260709 13:08:24.481642 26912 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:24.482204 26882 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.452997 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:24.482688 26882 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:24.584650 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.525666 23445 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:24.482772 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2484) took 1553 ms. Trace:
I20260709 13:08:24.547696 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:24.752117 22828 rpcz_store.cc:276] 0709 13:08:22.929249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.313886 (+384637us) service_pool.cc:224] Handling call
0709 13:08:24.482562 (+1168676us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54002314d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:24.482706 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1391494016}
W20260709 13:08:24.557863 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:24.517956 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 81122.6us
I20260709 13:08:24.706784 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.526489 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:24.599435 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:24.481693 22902 rpcz_store.cc:269] 0709 13:08:22.852889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.853004 (+   115us) service_pool.cc:224] Handling call
0709 13:08:24.481019 (+1628015us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b540035cad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:24.481160 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2234388608}
I20260709 13:08:24.548933 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:24.513571 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 105645us
W20260709 13:08:24.676201 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.607456 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2184) took 1371 ms. Trace:
I20260709 13:08:24.663383 23449 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:24.718752 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:24.541631 26804 consensus_meta.cc:287] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.576s	user 0.014s	sys 0.000s
I20260709 13:08:24.772859 22755 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:08:24.556039 26810 consensus_meta.cc:287] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.361s	user 0.006s	sys 0.020s
W20260709 13:08:24.513645 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 77802.6us
W20260709 13:08:24.774780 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:24.624961 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 200097us
W20260709 13:08:24.701496 26931 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.317s	user 0.000s	sys 0.001s
I20260709 13:08:24.586192 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:24.751719 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:24.776564 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.786410 23450 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:24.777221 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:24.778038 26913 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:08:24.778400 26472 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.168s	user 0.013s	sys 0.005s
I20260709 13:08:24.557626 26712 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:24.525424 26461 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:24.781935 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2191) took 1542 ms. Trace:
W20260709 13:08:24.611210 26497 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.626s	user 0.014s	sys 0.011s
I20260709 13:08:24.587394 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2201) took 1337 ms. Trace:
I20260709 13:08:24.719683 26909 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.783978 22751 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:24.482987 26880 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.788961 22754 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:24.630761 26831 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:24.548501 26869 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.006s
I20260709 13:08:24.543143 26928 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:24.514310 26770 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:24.454804 26675 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:24.623440 26912 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.766674 22904 rpcz_store.cc:276] 0709 13:08:23.235772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.235907 (+   135us) service_pool.cc:224] Handling call
0709 13:08:23.269020 (+ 33113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:23.415410 (+146390us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:23.415728 (+   318us) log.cc:844] Serialized 14 byte log entry
0709 13:08:23.874443 (+458715us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:23.874558 (+   115us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:23.874686 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.606992 (+732306us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540005a5d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:24.607051 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.607276 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2363,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":60875,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":177,"spinlock_wait_cycles":335978880,"thread_start_us":458095,"threads_started":2,"wal-append.queue_time_us":314245,"wal-append.run_cpu_time_us":1313,"wal-append.run_wall_time_us":1019513}
W20260709 13:08:24.798166 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.804481 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:24.802637 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:24.807512 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:24.799304 22900 rpcz_store.cc:276] 0709 13:08:23.250046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.854491 (+604445us) service_pool.cc:224] Handling call
0709 13:08:24.418545 (+564054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:24.438254 (+ 19709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.587142 (+148888us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b540035ded0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:24.587322 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1125416832}
I20260709 13:08:24.797204 26712 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:24.481249 22831 rpcz_store.cc:276] 0709 13:08:22.509108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.510040 (+   932us) service_pool.cc:224] Handling call
0709 13:08:22.512338 (+  2298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:22.527604 (+ 15266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.527635 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:22.592622 (+ 64987us) log.cc:844] Serialized 52 byte log entry
0709 13:08:22.867963 (+275341us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:22.904295 (+ 36332us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.904636 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 13:08:22.917157 (+ 12521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.917249 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.425818 (+508569us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b1800301a20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:24.015926 (+590108us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b1800301a20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:24.463386 (+447460us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b1800301a20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:24.480508 (+ 17122us) raft_consensus.cc:1685] finished
0709 13:08:24.480527 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.480685 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":366505,"prepare.queue_time_us":153058,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":7008,"raft.queue_time_us":43264,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":13322,"spinlock_wait_cycles":61927680,"thread_start_us":209841,"threads_started":1,"wal-append.queue_time_us":344515}
I20260709 13:08:24.808123 23450 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:24.797470 22898 rpcz_store.cc:276] 0709 13:08:23.239887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.828684 (+588797us) service_pool.cc:224] Handling call
0709 13:08:23.829482 (+   798us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:23.830392 (+   910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:23.830422 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:23.831601 (+  1179us) log.cc:844] Serialized 52 byte log entry
0709 13:08:23.992706 (+161105us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:24.034127 (+ 41421us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.597346 (+563219us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b180046c540. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb749b1d70 00007fcb749a5293 00007fcb7499acea 00007fcb7499b23a 00007fcb74e05a2a 00007fcb74e150bf 00007fcb74e15038
0709 13:08:24.636423 (+ 39077us) log.cc:844] Serialized 14 byte log entry
0709 13:08:24.766440 (+130017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.766514 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.781735 (+ 15221us) raft_consensus.cc:1685] finished
0709 13:08:24.781753 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.781884 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":661608,"prepare.queue_time_us":134328,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":1172,"raft.queue_time_us":207463,"raft.run_cpu_time_us":537,"raft.run_wall_time_us":12062,"spinlock_wait_cycles":2352512,"thread_start_us":144916,"threads_started":1,"wal-append.queue_time_us":161028}
I20260709 13:08:24.799012 26472 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.816836 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.801258 26461 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:24.818022 23451 raft_consensus.cc:2368] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:24.801105 26810 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.815747 26712 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:24.817797 26721 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:24.836673 23446 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:24.805358 26497 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:24.841041 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:24.845160 26755 consensus_meta.cc:287] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.769s	user 0.018s	sys 0.019s
I20260709 13:08:24.821806 26880 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:24.855654 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.854698 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:24.858533 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:24.860497 23450 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:24.861931 26912 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:24.862391 26912 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:24.865695 23446 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:24.867911 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.864553 26912 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:24.887843 22831 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:24.898981 23449 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:24.823064 26928 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:24.892230 26472 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.910281 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:24.911768 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:24.917500 23449 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:24.919436 26882 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.920912 22747 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:24.936046 23450 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:24.893594 26461 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.943913 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:24.944747 22748 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:24.946447 26928 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:24.946969 26878 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
W20260709 13:08:24.951859 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 174736us
I20260709 13:08:24.931047 26824 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:24.952206 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:24.956763 22755 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:24.934865 26810 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.958969 22831 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:08:24.959684 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:24.960942 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 122384us
I20260709 13:08:24.966251 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:24.968606 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:24.969120 26860 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.954136 26824 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:24.970813 22754 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:24.953261 26497 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.969594 22747 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
W20260709 13:08:24.971606 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 112077us
I20260709 13:08:24.973724 23445 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:24.951593 26878 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:24.975119 26824 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:24.991091 22748 raft_consensus.cc:2410] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:08:24.974507 26882 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:24.978555 26880 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:24.993587 26755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38064 (request call id 1113) took 9012 ms. Trace:
I20260709 13:08:24.999935 26755 rpcz_store.cc:276] 0709 13:08:15.977371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:15.977740 (+   369us) service_pool.cc:224] Handling call
0709 13:08:16.404349 (+426609us) ts_tablet_manager.cc:934] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
0709 13:08:22.491889 (+6087540us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:08:23.994002 (+1502113us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:23.994355 (+   353us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:08:24.874577 (+880222us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:08:24.932952 (+ 58375us) tablet_replica.cc:206] Creating consensus instance
0709 13:08:24.964432 (+ 31480us) ts_tablet_manager.cc:989] Replica registered.
0709 13:08:24.978200 (+ 13768us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":275,"tablet-copy.queue_time_us":329876,"thread_start_us":328283,"threads_started":2}
I20260709 13:08:25.001533 26755 tablet_copy_client.cc:806] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:25.002225 26755 tablet_copy_client.cc:670] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
W20260709 13:08:25.003235 26938 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:08:25.004514 23453 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:25.012437 23446 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:25.013259 23447 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:08:25.014144 26936 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 13:08:25.016773 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.018684 22747 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:25.028717 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:24.980419 26860 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.044488 26675 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:25.045718 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:25.047457 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 229973us
I20260709 13:08:25.051105 26912 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.051684 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.052055 26813 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:25.053824 26604 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:25.054222 26604 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:25.052995 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:25.042120 26934 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:08:25.057701 22748 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:25.063374 22748 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:25.053236 26813 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:08:25.062083 26878 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:25.065855 26880 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:25.066669 26892 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:25.068394 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:25.070065 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.070513 23451 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:25.072046 26811 raft_consensus.cc:2764] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:08:25.073416 22750 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:25.073680 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.053212 26912 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:25.074343 26712 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.074867 26880 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:25.076649 26712 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:25.076820 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.055034 26604 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:25.078574 26880 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:25.078394 22748 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:25.079298 26880 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:25.079965 22748 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:25.080021 26928 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:25.081568 26819 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:25.081964 26819 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:25.083154 26770 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:25.083062 26819 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.084195 26819 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:25.083585 26770 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:25.055401 26860 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:25.014585 26882 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:25.058492 26675 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:25.090059 26912 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:25.090400 22747 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:25.092206 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:25.093043 26738 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:24.901994 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:25.093937 26831 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:25.105665 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:25.108858 22905 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:25.111066 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.095379 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.101148 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:25.111856 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.101863 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:25.131553 22897 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:25.100819 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:25.103562 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:25.131824 23448 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:25.102592 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:25.121693 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 89183us
I20260709 13:08:25.121053 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.109871 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:25.127736 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.125289 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.114111 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.114369 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.135663 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:25.135455 22748 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:08:25.136376 22898 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:25.143191 23444 raft_consensus.cc:2368] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6.
W20260709 13:08:25.143682 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48448.2us
W20260709 13:08:25.105290 26942 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:08:25.110280 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.146003 22899 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:25.107865 26909 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:25.148752 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.108398 26927 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:25.154141 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:25.124799 26831 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:25.150735 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.116048 26882 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:25.168972 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1187) took 1005 ms. Trace:
I20260709 13:08:25.179493 23453 rpcz_store.cc:276] 0709 13:08:24.164002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.168502 (+  4500us) service_pool.cc:224] Handling call
0709 13:08:25.168810 (+1000308us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b5400671454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:25.168913 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370887936}
I20260709 13:08:25.179932 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.170117 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.171244 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.121299 26864 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:25.184341 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.185715 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.186450 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:25.109010 26813 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:08:25.119297 26869 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.158938 26927 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.188131 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.192734 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:25.182322 26864 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.191396 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.191339 22905 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:25.155606 26591 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:25.192462 22904 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:25.189747 23451 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:25.193249 22831 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:25.156400 26831 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:25.197492 23447 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:08:25.198895 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:25.200301 26824 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.195477 26869 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.203416 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.203722 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.203964 26882 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:25.125790 26770 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.209978 26864 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:25.236657 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:25.158882 26738 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:08:25.208513 26813 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:25.177759 26909 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.243803 26461 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:25.255308 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:25.256039 22747 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:25.246236 26928 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:08:25.258927 26824 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:25.259799 26775 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:08:25.260811 26684 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:08:25.263214 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:25.263861 22754 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:25.267120 26813 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:25.267557 26813 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:25.272727 23419 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:25.273128 26604 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:25.273993 26461 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:25.274415 26461 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.274755 26461 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:25.277452 26912 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:25.277655 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:25.278314 26878 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:25.278052 26912 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.281126 22650 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 9 to 10, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 10 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:25.282068 26775 raft_consensus.cc:3055] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:25.245445 26591 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:25.295339 22719 connection.cc:256] RPC call timeout handler was delayed by -6.3492957899999283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
I20260709 13:08:25.295166 22899 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
W20260709 13:08:25.297499 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 60922.6us
I20260709 13:08:25.297657 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:25.302785 22755 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:25.302963 22896 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:08:25.283981 22898 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:25.302804 26819 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:25.299751 26775 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:25.305228 26819 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:25.301890 22719 connection.cc:256] RPC call timeout handler was delayed by -6.2700957900000276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 13:08:25.306797 22719 connection.cc:256] RPC call timeout handler was delayed by -6.1926957900000161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
W20260709 13:08:25.307188 22719 connection.cc:256] RPC call timeout handler was delayed by -6.1881957900000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
W20260709 13:08:25.307663 22719 connection.cc:256] RPC call timeout handler was delayed by -6.1854957900000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20179999999999998s for a response
W20260709 13:08:25.307986 22719 connection.cc:256] RPC call timeout handler was delayed by -6.1602957899999637s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
I20260709 13:08:25.306241 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:25.312575 23446 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:25.305557 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:25.314843 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:25.316778 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 118015us
I20260709 13:08:25.294685 26878 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:25.325215 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:25.316107 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.330613 26591 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:25.268493 26813 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:08:25.337777 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.305554 26770 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:25.301499 26892 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:25.343134 22826 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:25.352315 22747 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:25.304922 26775 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.346741 26892 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.383610 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:25.389721 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 268745us
I20260709 13:08:25.354470 26591 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:25.372635 26688 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:08:25.391952 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.393088 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:25.401532 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.387110 26892 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:25.407114 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:25.407944 22748 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:25.406987 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:25.439472 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 57566.8us
I20260709 13:08:25.443545 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:25.346892 26831 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.394217 26688 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:25.462435 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73769.8us
I20260709 13:08:25.463639 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.463435 26688 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:25.464506 22825 raft_consensus.cc:2410] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:08:25.465616 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.465763 26591 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.466579 22824 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:25.468818 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:25.470229 26831 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:25.471201 26804 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:25.465570 26712 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:25.478994 26712 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:25.480157 26712 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.411386 26860 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:25.446237 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.448676 22896 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:25.454222 22650 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:25.483086 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.458150 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.485852 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.486418 22830 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 6.
I20260709 13:08:25.487579 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.488224 22830 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
W20260709 13:08:25.489109 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 354208us
W20260709 13:08:25.461930 26949 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:08:25.462169 23448 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:08:25.463907 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.455298 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:25.503398 23447 raft_consensus.cc:2368] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:25.458629 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:25.490032 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.448022 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.505072 22898 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:08:25.495970 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:25.499871 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:25.506616 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:25.506994 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.507731 22755 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:25.465035 26688 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:25.516410 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:25.520690 23445 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:25.523352 22755 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:25.526199 23446 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:25.526078 26954 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:25.523130 26811 raft_consensus.cc:2764] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:08:25.514003 26951 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.528165 26951 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:25.529445 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.529667 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.530295 22827 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:25.530393 26951 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:25.530496 26912 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:25.531711 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:25.532310 26909 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:25.517096 26712 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:25.534296 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.535076 22830 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:25.535758 22797 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:25.535949 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.536619 22830 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:25.516386 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46132.5us
I20260709 13:08:25.538193 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:25.543943 23447 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:25.543601 26892 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.412319 26819 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.503543 26927 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:25.397512 26878 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:25.548470 26878 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:25.553762 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.554347 22823 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:25.556084 26892 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:25.559382 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.560295 22900 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:25.537075 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:25.548712 26461 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.562399 22826 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:25.564093 23447 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:25.580551 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.581542 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.582203 22826 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:25.583989 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.584805 22905 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:25.585244 22830 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:25.586041 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.590925 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.593685 22896 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:08:25.596575 22826 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:25.598472 26775 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:25.601694 22827 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:25.603590 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.604329 22825 raft_consensus.cc:2368] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:25.579067 26684 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:25.605655 26684 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:25.606190 26684 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:25.607040 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:25.607997 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:25.608839 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:25.605129 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.609496 26927 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:25.610031 22901 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:25.605685 22830 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
W20260709 13:08:25.605685 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 198499us
I20260709 13:08:25.616735 23447 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:08:25.617344 22904 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:25.618654 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.620306 22823 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:25.621301 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.622429 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.623152 22897 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:25.631246 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:25.642089 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51846.2us
I20260709 13:08:25.643059 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.644176 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.642247 22896 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:25.621492 23416 connection.cc:256] RPC call timeout handler was delayed by -1.3942296300000407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
I20260709 13:08:25.609993 26927 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:25.645171 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.673062 22901 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:25.673588 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 83689.9us
I20260709 13:08:25.673846 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.673471 26927 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:25.674603 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.674866 22901 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:25.630861 26909 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:25.678252 23447 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:25.679919 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:25.681957 26813 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:25.687264 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.684386 22747 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:08:25.631639 26461 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:25.647166 26951 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:25.688467 22823 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:25.690420 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:25.694639 22828 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:25.690584 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:25.699882 22824 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:25.700371 22829 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:25.697407 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:25.704016 23451 raft_consensus.cc:2368] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:25.704666 22831 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:25.704650 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.706144 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.706064 26864 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.707008 22901 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:25.707321 22902 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:25.708065 26864 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:25.708020 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.709254 22901 raft_consensus.cc:2410] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:08:25.710254 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.711803 26721 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:25.703684 26882 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:25.713791 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:25.714813 22755 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:08:25.716025 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.691579 26909 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:25.701843 26765 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:25.751277 26684 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:25.752076 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.752655 22902 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:25.753422 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.753696 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:25.754132 22902 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:25.754451 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.754923 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.755259 22905 raft_consensus.cc:2410] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:08:25.756258 22905 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:25.756510 26497 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:25.756639 26770 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:25.757038 26497 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:25.757520 26497 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:25.687433 26968 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:25.759198 26968 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:25.759552 26968 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:25.748713 26321 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:25.761013 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:25.761629 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:25.749958 26869 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:25.702030 26951 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:25.758723 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:25.773833 22747 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:25.722317 22896 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
W20260709 13:08:25.725410 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 94294.2us
W20260709 13:08:25.763170 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.756484 22902 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:25.753209 26684 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:25.784566 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:25.785985 22755 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:25.786211 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.770416 26321 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:25.795516 22747 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:25.796769 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:25.798148 26775 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:25.797840 26951 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.798727 26775 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:25.799578 26775 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:25.780923 26869 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:25.801402 26869 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:25.802265 26869 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.781275 26831 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:08:25.806825 26831 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:25.807204 26831 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:25.808007 26831 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.806849 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:25.809168 26831 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:25.831372 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:25.833252 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 51633us
W20260709 13:08:25.830273 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 127242us
I20260709 13:08:25.835462 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:25.836071 22750 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:08:25.812084 26878 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:25.838374 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:25.840991 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:25.827319 26321 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:25.843703 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.844440 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:25.835866 26819 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:25.845470 26770 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:25.850620 26497 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:25.851645 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44859.5us
I20260709 13:08:25.857823 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:25.866226 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:25.864959 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.867220 26810 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:25.873544 26712 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:25.846092 26472 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:25.874197 26591 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:08:25.877348 26591 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:25.876981 26321 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.877692 26591 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:25.878451 26591 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.879524 26591 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:25.880908 22898 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:25.865376 26880 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:25.897974 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:25.902181 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 61634.9us
I20260709 13:08:25.889891 26765 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:25.839694 26909 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:25.912843 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:25.939262 22903 raft_consensus.cc:606] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.939666 22903 raft_consensus.cc:614] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:25.913105 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:25.913604 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:25.918212 26869 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:25.922552 26882 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.899971 26880 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:08:25.925024 26497 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:25.932236 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.890769 26738 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:25.934298 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:25.935396 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 59337.1us
I20260709 13:08:25.867187 26684 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:08:25.943657 22750 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:25.948661 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.945837 26882 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:25.954744 22902 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:25.955341 22902 raft_consensus.cc:606] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:25.956873 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 62541.6us
I20260709 13:08:25.901758 26472 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:08:25.958259 22902 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:25.958791 22902 raft_consensus.cc:606] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.959079 22902 raft_consensus.cc:614] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:25.961397 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.962585 22824 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:08:25.963881 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.965551 26807 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:25.966403 22902 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:25.966905 22902 raft_consensus.cc:606] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.967219 22902 raft_consensus.cc:614] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:25.946228 26892 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:25.967159 26953 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:25.972438 22902 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
W20260709 13:08:25.972913 22654 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 337) took 24129 ms (client timeout 15000 ms). Trace:
W20260709 13:08:25.989681 22654 rpcz_store.cc:269] 0709 13:08:01.843272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.875166 (+ 31894us) service_pool.cc:224] Handling call
0709 13:08:25.972665 (+24097499us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:22.768929 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:22.777609 (+  8680us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:22.777642 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:08:22.777662 (+    20us) tablet.cc:662] Decoding operations
0709 13:08:22.778449 (+   787us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:22.778519 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:22.778554 (+    35us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:08:22.779062 (+   508us) tablet.cc:701] Row locks acquired
0709 13:08:22.779089 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:08:22.779224 (+   135us) write_op.cc:270] Start()
0709 13:08:22.779333 (+   109us) write_op.cc:276] Timestamp: P: 1783602502779179 usec, L: 0
0709 13:08:22.779355 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:08:22.779979 (+   624us) log.cc:844] Serialized 2459 byte log entry
0709 13:08:23.740736 (+960757us) op_driver.cc:464] REPLICATION: finished
0709 13:08:23.741929 (+  1193us) write_op.cc:301] APPLY: starting
0709 13:08:23.742016 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:23.742220 (+   204us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:23.742328 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:23.743068 (+   740us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:23.743322 (+   254us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:08:23.743384 (+    62us) write_op.cc:312] APPLY: finished
0709 13:08:23.859423 (+116039us) log.cc:844] Serialized 53 byte log entry
0709 13:08:23.920596 (+ 61173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:23.920845 (+   249us) write_op.cc:454] Released schema lock
0709 13:08:23.920908 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:23.922211 (+  1303us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:23.922255 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":919500,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":956,"apply.run_cpu_time_us":3770,"apply.run_wall_time_us":180029,"num_ops":6,"prepare.queue_time_us":221738,"prepare.run_cpu_time_us":50022,"prepare.run_wall_time_us":317508,"raft.queue_time_us":189168,"raft.run_cpu_time_us":1154,"raft.run_wall_time_us":1154,"replication_time_us":961217,"thread_start_us":504305,"threads_started":4,"wal-append.queue_time_us":410305,"wal-append.run_cpu_time_us":29590,"wal-append.run_wall_time_us":1695511}]]}
I20260709 13:08:25.973603 22650 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 7 to 9, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 9 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:25.993678 22650 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 3 to 8, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 8 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:25.995282 22650 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 9 to 12, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 12 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:25.975755 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.976210 26813 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:25.997740 26813 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:25.983625 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.972985 22902 raft_consensus.cc:606] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:08:25.999164 22902 raft_consensus.cc:614] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:25.997130 22650 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 12 to 13, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 13 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:26.004487 22829 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:26.007334 22719 leader_election.cc:400] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6.
I20260709 13:08:26.007647 22719 leader_election.cc:403] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:08:26.015923 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 68834.5us
I20260709 13:08:26.016860 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.016786 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:26.014053 22650 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:26.015406 26721 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 5. Current term is 6.
I20260709 13:08:26.024129 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:25.996803 26968 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:26.025683 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.025934 22829 raft_consensus.cc:2410] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:08:26.025540 22650 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:26.027699 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:26.031443 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.025338 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:26.039878 26953 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:26.087548 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:26.065933 26461 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:26.036309 26927 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:26.088243 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:26.074568 26976 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:08:26.125514 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:26.077811 26813 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:26.148249 26882 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:26.072875 22650 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 10 to 11, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 11 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:26.065723 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.087234 22827 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:26.098182 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 60272us
I20260709 13:08:26.088063 22794 leader_election.cc:400] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 8. Current term is 9.
I20260709 13:08:26.100652 22828 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:26.103140 26755 tablet_copy_client.cc:538] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:26.082921 26968 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:26.117792 22825 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:26.064953 26957 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:26.099992 26951 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:26.133095 22823 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:26.133683 23445 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:26.133625 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 190072us
W20260709 13:08:26.133545 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:26.133385 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 241815us
W20260709 13:08:26.159565 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.066390 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:26.156159 22650 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 11 to 13, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 13 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:26.161710 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 73807us
I20260709 13:08:26.162621 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:26.164904 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 98610.5us
I20260709 13:08:26.169046 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:26.169950 22903 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:26.145162 26819 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:26.173848 22902 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:26.176766 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:26.182641 22831 raft_consensus.cc:2410] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:26.190172 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.193102 22823 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:26.173359 26804 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:26.194357 22650 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:26.198330 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 47619.5us
W20260709 13:08:26.207249 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44713.7us
I20260709 13:08:26.208606 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.208835 22719 leader_election.cc:400] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:26.173517 26807 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:08:26.209257 22719 leader_election.cc:403] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:26.210865 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:26.202139 26953 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:26.212961 22650 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 9 to 11, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 11 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:26.201900 26831 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:26.215116 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.218677 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.220569 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:26.220867 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.211949 26804 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:26.227428 26804 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.228358 23453 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:26.213507 26591 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:26.231537 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:26.212045 26927 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:08:26.234356 26927 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:26.234694 26927 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:26.216984 26974 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.235540 26927 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.235806 26974 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:26.236764 26927 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:26.215514 26675 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:26.215809 26909 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:26.240983 23445 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:26.245148 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:26.230541 22650 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:26.246150 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:26.212585 26775 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:08:26.249827 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.237393 26721 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:26.250730 22650 catalog_manager.cc:5697] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:26.215410 26770 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 13:08:26.270358 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49574.2us
I20260709 13:08:26.281286 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.271339 22898 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:26.282418 22898 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:26.286454 22650 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 9 to 12, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 12 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:26.302309 26869 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:26.303571 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 77243.9us
W20260709 13:08:26.303891 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57814.7us
W20260709 13:08:26.304270 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54502.5us
I20260709 13:08:26.249188 26977 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:26.271064 26721 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:26.308588 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.285857 26675 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:26.334131 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.305150 26919 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:26.343811 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:26.351907 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:26.352708 22902 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:08:26.357924 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.359419 26684 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:26.361281 26684 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:26.327800 26721 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:26.329847 26882 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:26.368623 26984 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:08:26.375348 26974 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:26.375267 22901 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:26.375669 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.377285 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:26.378481 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 97331.6us
I20260709 13:08:26.323304 26591 raft_consensus.cc:2764] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:08:26.379765 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:26.346089 26953 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:26.390080 23449 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:26.391618 26775 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:26.394140 26991 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:26.403059 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.383649 26882 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:26.388043 26985 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:26.385193 26810 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:08:26.422703 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 114196us
I20260709 13:08:26.428610 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.428936 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.399675 26991 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:26.413882 26974 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:08:26.422313 26983 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:08:26.433243 26985 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:26.437988 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.440845 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.426669 26882 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:26.446756 26882 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.447492 22903 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:26.431617 26807 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:08:26.439182 26977 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:26.431274 26675 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.423946 26996 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:26.443832 26909 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:26.471494 26987 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 13:08:26.444772 26988 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:08:26.441506 26824 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:08:26.463454 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.445178 26991 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.479715 26991 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:26.484093 22902 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:26.434010 26974 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:26.432541 26990 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:08:26.429747 26813 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:08:26.486096 26892 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:26.486836 26738 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:26.492520 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:26.495726 22750 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:26.499140 26712 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.499615 26712 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:08:26.495008 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 68521.3us
I20260709 13:08:26.501403 22747 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:08:26.491925 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:26.465863 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:26.504271 26738 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.506323 26738 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:26.506968 26951 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:26.507272 26985 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.507674 22898 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.508637 26864 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:26.509055 26864 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:26.510046 26864 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.510712 22750 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:26.509001 22898 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:26.518427 27002 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:26.493201 26974 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:26.530543 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.537850 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:26.532011 22902 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:26.541817 22902 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:26.542918 22898 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:26.511184 26864 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:26.552980 22902 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:26.530148 26990 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:26.581012 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.538946 26819 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:26.519749 26951 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:26.531188 26461 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:26.554769 26968 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:08:26.581905 26990 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:26.601058 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 87145.3us
I20260709 13:08:26.596177 26738 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:26.612236 22902 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:26.613547 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:26.616703 26951 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:26.624202 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.599697 26712 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:26.598963 26819 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:26.649791 22902 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:08:26.650925 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.590687 26461 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.606662 26968 consensus_queue.cc:1243] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:08:26.651360 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:08:26.687579 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:26.699496 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:26.713146 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.616623 26721 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:26.722067 27001 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:08:26.721805 26977 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:26.749441 26472 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:26.749898 26472 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:26.751245 26755 tablet_bootstrap.cc:492] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:08:26.750883 26472 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.756325 22755 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:26.745121 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 285344us
W20260709 13:08:26.764743 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 183797us
I20260709 13:08:26.765861 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:26.768744 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 147734us
I20260709 13:08:26.788067 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:26.789429 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 176191us
W20260709 13:08:26.764614 26927 consensus_meta.cc:287] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.527s	user 0.014s	sys 0.000s
I20260709 13:08:26.756055 26675 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.798300 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.796640 26977 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:08:26.834090 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.835815 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.820442 26813 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:26.739444 26990 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.828948 26927 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:08:26.855052 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.856065 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:26.828253 26968 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:26.866652 22903 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:08:26.806229 26675 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.867506 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.849908 26951 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:26.827566 26892 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:08:26.806773 26819 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:26.858256 26927 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:26.888274 26951 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:26.896632 26951 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:26.887079 26993 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:08:26.885304 26962 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:26.900609 27000 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:26.868570 26968 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:26.907326 26968 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:26.907159 27000 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:26.893954 26813 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:26.909190 27000 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:26.909978 27018 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:08:26.911288 26461 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:26.914247 27030 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:08:26.915032 26953 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.000s
I20260709 13:08:26.915419 26953 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:26.915813 26891 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.916383 26891 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:26.917730 27014 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:08:26.918516 26891 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:26.918705 26721 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:26.891487 26909 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:26.925657 26909 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.893428 26927 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:26.926891 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.927629 26909 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:26.929184 22829 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:26.929932 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:26.930092 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:26.909969 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.933310 22719 leader_election.cc:400] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
W20260709 13:08:26.911548 27023 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:08:26.940199 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:26.940793 22748 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:08:26.944985 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 95663.4us
I20260709 13:08:26.946537 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:26.950167 26997 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:26.950667 26997 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:26.956244 26997 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:26.944300 26909 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:26.958827 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:26.960748 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.931406 26962 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:26.961704 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.962177 26962 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:26.965592 26775 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:26.969905 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:26.972044 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:26.973719 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:26.974720 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:26.975401 23446 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:26.985423 27021 raft_consensus.cc:2764] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:08:26.992080 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:26.992762 23448 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:08:26.992817 22756 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:26.996734 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:27.001142 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.977414 26977 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:26.979085 26721 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:26.979262 26968 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:08:26.978343 26909 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:27.007171 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:26.981653 26738 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:27.008893 22899 pending_rounds.cc:85] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 8
I20260709 13:08:27.012012 22899 pending_rounds.cc:107] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:08:27.012297 22899 raft_consensus.cc:2889] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:27.012444 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:27.003353 27021 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:27.018518 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:27.019086 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:27.022589 22901 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
W20260709 13:08:27.020303 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45237.4us
I20260709 13:08:27.022827 22797 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:27.011258 26591 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.028383 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:27.030333 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:27.030550 22904 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:27.016512 26880 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.992274 26927 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.011313 26977 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:27.035724 22719 leader_election.cc:400] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
I20260709 13:08:27.037537 22719 leader_election.cc:403] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:27.036504 26953 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:27.020481 26824 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:08:27.033702 26591 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.038252 26880 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:27.058223 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 47822.4us
I20260709 13:08:27.077673 22755 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:27.053612 26721 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12.
W20260709 13:08:27.080901 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 52836us
I20260709 13:08:27.085628 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:27.086272 22756 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:27.053781 26984 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:08:27.060061 26977 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:08:27.093111 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63535us
I20260709 13:08:27.068958 26991 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:08:27.071223 26985 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:08:27.102850 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:27.094372 26721 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.096333 26675 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.122004 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:27.122813 22830 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:08:27.123358 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:27.122850 22831 raft_consensus.cc:2410] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:27.123950 22823 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:27.097185 26765 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 13:08:27.101027 26864 consensus_meta.cc:287] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 13:08:27.128746 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:27.131347 23448 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:27.130925 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.131841 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.131171 22829 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:27.133594 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:27.134330 22823 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:27.134373 22897 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:27.135166 23446 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:27.131050 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:27.135874 23445 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:27.137742 22824 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:27.123370 27003 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:27.123996 26813 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:27.117918 26721 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.112288 26996 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:08:27.169129 22755 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:08:27.169524 22755 pending_rounds.cc:85] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:08:27.169731 22755 pending_rounds.cc:107] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:08:27.169929 22755 raft_consensus.cc:2889] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:27.171252 26927 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.181726 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:27.144601 27003 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:27.195855 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:27.166482 26864 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.199167 27024 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:27.168819 26813 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:27.215728 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:27.239919 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 150074us
I20260709 13:08:27.261430 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:27.276398 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:27.195036 26738 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:27.204107 27003 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:27.278851 27045 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:08:27.289878 27019 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.498s	user 0.002s	sys 0.002s
I20260709 13:08:27.291683 23451 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:27.291954 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:27.293221 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.301674 23450 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:27.293869 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:27.294361 26721 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.294530 26591 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:27.295296 27050 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:08:27.296119 27042 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 13:08:27.310700 26721 raft_consensus.cc:2764] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:08:27.273679 27048 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:27.311820 27048 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:27.312239 27048 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:27.303279 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:27.317615 22823 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:27.304001 22831 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:27.305073 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:27.319460 22828 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 7: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:27.305876 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:27.319546 27048 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.320407 22829 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:27.203091 26882 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:08:27.209508 26864 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:27.331379 26990 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:27.343676 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.345110 26880 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:27.347604 27052 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:08:27.321376 27053 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:08:27.358428 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.360349 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:27.287456 26738 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.370224 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:27.375396 22902 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:27.378333 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:27.378958 22900 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:27.380784 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:27.384984 22830 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:08:27.387753 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:27.391741 23453 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:27.397677 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.398404 23447 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
W20260709 13:08:27.398310 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 202505us
I20260709 13:08:27.395046 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.402256 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.374413 26997 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:27.408466 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.409017 23453 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:27.404376 22904 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:27.401671 23452 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:27.374940 27053 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:27.385604 27004 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.402894 23451 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:27.385470 26860 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.401520 26591 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.373553 26951 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.408097 26997 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.429227 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:27.435073 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:27.440349 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:27.417912 27004 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.441632 27005 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
W20260709 13:08:27.446117 26974 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.890s	user 0.012s	sys 0.000s
I20260709 13:08:27.419945 26860 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.446506 26974 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:27.447999 26860 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:27.449605 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:27.450183 22755 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:27.451490 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:27.452860 27004 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.453697 26860 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:27.454109 26860 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.454447 26860 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:27.457639 26721 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:27.459365 26721 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:27.388779 26813 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:27.461583 26860 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.462040 26864 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:27.462911 27053 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:27.464198 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:27.464696 22755 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:27.468396 26909 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:27.431175 26591 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.469806 26738 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.444065 26996 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:27.474501 22748 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:27.470889 22747 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:27.473384 22824 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:27.527563 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:27.606271 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 187514us
I20260709 13:08:27.432962 26997 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:27.621358 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:27.636245 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:27.665084 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:27.665114 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:27.668802 22831 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:27.639736 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 199474us
W20260709 13:08:27.689728 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 284027us
I20260709 13:08:27.701180 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:27.702240 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:27.671306 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:27.674589 22749 raft_consensus.cc:2410] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:27.588547 26497 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:27.688407 22900 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:27.637404 22901 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:27.636617 27000 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:27.706902 27059 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:27.708142 26864 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:27.672163 26860 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:27.614096 22902 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:08:27.480417 22755 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:27.714886 22754 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:08:27.713258 22831 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:27.608409 22903 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:27.708357 27000 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.717593 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:27.718384 26860 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:27.718631 27000 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.719455 22901 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:08:27.720698 26860 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:27.522704 26831 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.441673 27022 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:27.668268 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:27.637806 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:27.605850 26591 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:27.722342 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 51122.3us
I20260709 13:08:27.740924 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:27.493006 27003 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:27.724833 22899 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
W20260709 13:08:27.725562 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 57347.1us
I20260709 13:08:27.727555 26860 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.732806 26957 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:27.754835 26957 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:27.755628 26957 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.756757 26957 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:27.757488 27062 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:27.758206 27062 consensus_queue.cc:1243] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:08:27.758612 27068 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:27.759004 27068 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:27.745540 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:27.638312 26882 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.700006 26951 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.748910 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.748987 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:27.766112 22824 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:27.766341 23450 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:27.645265 26968 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:08:27.751356 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:27.750720 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:27.759819 27068 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.705730 27004 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:27.771142 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:27.774457 27005 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:27.765309 26882 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:27.750802 26996 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:27.777120 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:27.749471 27048 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.778083 22830 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:27.738746 27000 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:27.797894 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:27.799068 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:27.796900 26951 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:27.721870 26497 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.743188 26831 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:27.802457 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.815539 26959 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:27.821624 27054 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.547s	user 0.004s	sys 0.003s
I20260709 13:08:27.901067 26959 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.825409 27068 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:27.900271 23448 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:08:27.906065 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.795809 27022 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:27.907729 27022 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:27.803593 22831 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:27.908633 27022 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.900205 22827 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:27.903190 26959 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.907145 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:27.954421 23448 raft_consensus.cc:2410] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:27.824434 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2236) took 1309 ms. Trace:
I20260709 13:08:27.795693 27062 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:27.907078 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:27.952703 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 154895us
I20260709 13:08:27.954782 22897 rpcz_store.cc:276] 0709 13:08:26.515537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.515705 (+   168us) service_pool.cc:224] Handling call
0709 13:08:26.529392 (+ 13687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:27.134767 (+605375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:27.824200 (+689433us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b5400206a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:27.824378 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390686080}
I20260709 13:08:27.955444 23447 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:27.798823 27005 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.963551 26909 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:27.964098 26472 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:27.976538 26864 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:27.955991 27062 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.977558 27062 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:27.950242 26882 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:27.982779 26997 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.984889 26997 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.985666 27003 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:27.985834 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:27.957957 27053 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:27.803963 26974 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:27.990763 27004 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.991235 27004 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:27.955034 26951 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:27.995395 26951 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:27.815073 27048 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.997826 27048 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:27.999934 27048 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:27.999725 27062 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.002301 26831 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:28.005347 26497 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.007040 27062 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:27.965927 27005 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:27.925402 26891 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:28.017728 26891 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.023177 23444 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 7.
I20260709 13:08:28.026374 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:28.078666 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 49654.8us
I20260709 13:08:28.082976 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:27.993099 27004 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.150388 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:28.147497 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:28.153885 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:28.148147 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:28.094545 26959 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:28.093148 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:28.162258 26959 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.164450 26959 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.166275 26997 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:28.167366 26968 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:28.167742 26968 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:28.168521 26968 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.177860 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:28.179713 22749 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:08:28.180480 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:28.182437 22831 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:08:28.183490 23452 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:28.183990 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:28.175560 27013 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.171707 26891 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.188621 26738 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:28.187350 27059 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:28.082425 27048 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:28.214679 26738 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.200137 27021 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:08:28.248929 22827 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:28.212954 27013 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.250620 26864 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.246133 26738 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:28.254580 27058 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.221386 27048 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:28.282035 26968 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:28.286538 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 330037us
I20260709 13:08:28.287413 26977 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:28.288793 27062 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:28.291451 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:28.292027 22905 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:28.294487 22825 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:28.277428 27048 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:28.298070 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:28.298708 22899 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:28.299742 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:28.300305 23447 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:28.306639 27048 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.255168 26953 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:08:28.313648 22905 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:28.309299 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:28.314661 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:28.316785 27039 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:28.317304 22900 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:28.317129 26472 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:28.321066 22899 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:28.323592 22903 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:08:28.265257 26864 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.348227 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:28.270845 27058 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:28.351430 26721 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:28.356489 26721 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:28.360743 22826 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:08:28.369997 27013 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.359026 26721 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.375128 26927 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:28.375566 26927 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:28.376585 27068 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:28.379410 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:28.343411 27016 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:08:28.357589 27061 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:08:28.369140 27048 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.391407 26990 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.428467 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 757) took 1135 ms. Trace:
I20260709 13:08:28.432585 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:28.433137 23447 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:28.433851 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:28.428839 23444 rpcz_store.cc:276] 0709 13:08:27.292863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.293672 (+   809us) service_pool.cc:224] Handling call
0709 13:08:28.428307 (+1134635us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b540018c4d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:28.428410 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1625996032}
I20260709 13:08:28.439790 23447 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:28.440804 27048 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.441259 27048 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:28.442494 27066 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.732s	user 0.004s	sys 0.000s
I20260709 13:08:28.417713 27013 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:28.465773 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:28.472667 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:28.435568 26864 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.493255 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:28.496421 27073 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:08:28.477859 26321 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:08:28.502297 27058 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:28.505625 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:28.506675 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:28.507565 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:28.507994 22902 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:28.466593 27048 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:28.510177 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:28.510689 23445 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:28.514955 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:28.521262 22902 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:28.523537 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:28.523976 22902 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:28.515687 22826 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:28.426461 26990 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.480248 27076 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:28.527645 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:28.528633 22827 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:28.531888 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:28.526051 27016 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:28.533536 22902 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:08:28.479486 27013 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.477398 27005 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.552822 23445 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:28.555634 27005 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.529970 26959 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:28.534811 27058 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:28.566215 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2511) took 1152 ms. Trace:
I20260709 13:08:28.572598 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:28.533520 27059 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:28.566952 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:28.568686 27013 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.569895 26738 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:28.575076 27072 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 13:08:28.572326 22830 rpcz_store.cc:276] 0709 13:08:27.413544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.413672 (+   128us) service_pool.cc:224] Handling call
0709 13:08:27.413908 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:27.801217 (+387309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.565952 (+764735us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b540043add4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:28.566151 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837147392}
I20260709 13:08:28.586401 22825 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:28.578799 22756 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:08:28.578104 26959 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:28.588928 26959 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:28.547248 26864 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:28.589570 26864 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.589955 26864 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:28.591897 22752 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:08:28.592242 22752 pending_rounds.cc:85] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 7
I20260709 13:08:28.592459 22752 pending_rounds.cc:107] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:08:28.592666 22752 raft_consensus.cc:2889] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:28.593380 26765 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:28.593658 26738 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:28.594343 22897 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:28.594396 26765 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:28.594947 26765 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:28.595201 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:28.595795 26765 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.597003 26765 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:28.545889 27024 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:28.535864 27076 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:28.607890 22756 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:28.609725 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:28.611096 22904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2252) took 1239 ms (client timeout 1545 ms). Trace:
W20260709 13:08:28.613521 22904 rpcz_store.cc:269] 0709 13:08:27.371429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.371600 (+   171us) service_pool.cc:224] Handling call
0709 13:08:28.610902 (+1239302us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b54003bfbd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:28.611030 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36319488}
I20260709 13:08:28.566362 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:28.607668 27053 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.612398 27013 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.631981 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:28.629242 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:28.541483 27016 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.610724 26721 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:28.633370 23448 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 8: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:28.634044 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:28.617276 27048 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:28.584686 27059 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.612751 26738 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 13:08:28.652912 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 74869.3us
I20260709 13:08:28.631822 26990 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:28.655551 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:28.614933 27024 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:28.654847 26991 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:08:28.645480 27084 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.661284 23448 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:28.647701 26721 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:28.779116 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.635954 27013 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.634311 27053 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:28.651808 27048 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:28.921139 27097 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.921882 27097 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:28.933987 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 278500us
I20260709 13:08:28.935307 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:28.952518 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:28.953229 22825 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:28.877856 26962 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:28.879527 27099 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:28.977622 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:28.991216 22825 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:28.666594 27084 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:28.971813 26738 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:28.975337 27099 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:28.974473 27039 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.005513 27059 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.007462 26755 tablet_bootstrap.cc:492] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:29.007660 27013 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.002759 27099 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:29.008402 26755 tablet_bootstrap.cc:492] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:08:28.993307 26721 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.979007 26962 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.010533 26755 ts_tablet_manager.cc:1403] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 2.260s	user 0.030s	sys 0.025s
I20260709 13:08:29.027069 26755 raft_consensus.cc:359] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 8 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:29.028075 26755 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 8 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:08:29.033612 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.034314 22748 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:29.040073 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.040741 22748 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:29.042666 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:29.044848 26962 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:29.050235 22642 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.452s	user 0.011s	sys 0.032s
I20260709 13:08:29.013463 27022 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:29.020530 26738 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.089797 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:29.091231 27048 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.092164 27053 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:29.092978 27103 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:08:29.078832 27092 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:08:29.012189 27039 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.089746 26951 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.120167 26813 raft_consensus.cc:993] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:08:29.120615 26813 raft_consensus.cc:1081] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
W20260709 13:08:29.115247 26984 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.017s	sys 0.025s
W20260709 13:08:29.121716 26984 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.017s	sys 0.025s
I20260709 13:08:29.103543 27048 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.094264 26738 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.088855 27022 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.028851 26755 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:29.114665 27059 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:29.109694 26953 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:08:29.145933 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.146605 22899 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:29.150681 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.135372 26996 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:29.136508 26813 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.114804 26951 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:29.138765 27024 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:29.168618 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.168764 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.173283 27016 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:29.174047 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.191466 22752 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:29.175755 26738 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.140439 27084 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:29.133442 27048 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.146684 27076 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:29.166868 27093 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
W20260709 13:08:29.193051 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 103325us
W20260709 13:08:29.168581 27100 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:08:29.168464 27094 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:08:29.166841 26996 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:29.191123 27013 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:29.163556 26927 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:29.152496 27059 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:29.198944 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1252) took 1401 ms. Trace:
I20260709 13:08:29.166673 27095 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:29.199378 26864 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.002s
I20260709 13:08:29.180395 22750 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
W20260709 13:08:29.207239 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 64194.9us
W20260709 13:08:29.208273 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.211272 22750 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:29.180557 26813 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:29.211630 22750 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:29.210089 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:29.208761 23450 rpcz_store.cc:276] 0709 13:08:27.797093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.198381 (+1401288us) service_pool.cc:224] Handling call
0709 13:08:29.198617 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:29.198739 (+   122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:29.198756 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:29.198781 (+    25us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:29.198805 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.198834 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.198897 (+    63us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:29.217571 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.220528 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.207949 27097 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.224421 27048 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.230141 27096 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:08:29.235957 22719 consensus_peers.cc:597] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:29.263298 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 753010us
I20260709 13:08:29.263892 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:29.265017 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:29.238483 27090 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.607s	user 0.003s	sys 0.001s
I20260709 13:08:29.214464 26864 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:29.242558 26755 ts_tablet_manager.cc:1434] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.219s	user 0.007s	sys 0.001s
I20260709 13:08:29.242987 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2514) took 1822 ms. Trace:
I20260709 13:08:29.292501 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.243566 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2363) took 1061 ms. Trace:
I20260709 13:08:29.243616 26951 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:29.246055 27039 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:29.246918 26765 consensus_meta.cc:287] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.632s	user 0.006s	sys 0.002s
I20260709 13:08:29.246840 27059 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:29.248090 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.270256 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.218180 26953 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:29.217736 26927 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:29.280866 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.291618 27122 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:29.295642 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 139734us
I20260709 13:08:29.294040 26721 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:29.293280 22747 rpcz_store.cc:276] 0709 13:08:28.181869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.182121 (+   252us) service_pool.cc:224] Handling call
0709 13:08:28.195302 (+ 13181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:28.195653 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:28.556588 (+360935us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b18000ff640. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0b455 00007fcb787f33c7 00007fcb74e059a7 00007fcb74e150bf 00007fcb74e15038
0709 13:08:28.851250 (+294662us) log.cc:844] Serialized 14 byte log entry
0709 13:08:29.137190 (+285940us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:29.137955 (+   765us) log.cc:844] Serialized 26 byte log entry
0709 13:08:29.138193 (+   238us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:29.139506 (+  1313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.139623 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:29.243299 (+103676us) raft_consensus.cc:1685] finished
0709 13:08:29.243398 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.243515 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":116280,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":5503,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":14309,"spinlock_wait_cycles":2705792,"thread_start_us":449658,"threads_started":2,"wal-append.queue_time_us":256777}
I20260709 13:08:29.210568 27013 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:29.302947 26951 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:29.303467 27087 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.304168 27099 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.305621 26974 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:29.307729 26909 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:29.310082 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.310802 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.335261 22825 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:08:29.292465 22828 rpcz_store.cc:276] 0709 13:08:27.420658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.420865 (+   207us) service_pool.cc:224] Handling call
0709 13:08:27.436342 (+ 15477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:27.436746 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:27.436786 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:27.474931 (+ 38145us) log.cc:844] Serialized 26 byte log entry
0709 13:08:27.606721 (+131790us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:27.644805 (+ 38084us) log.cc:844] Serialized 14 byte log entry
0709 13:08:27.645016 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:27.645089 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:29.219440 (+1574351us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b1800309be0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:29.242728 (+ 23288us) raft_consensus.cc:1685] finished
0709 13:08:29.242816 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.242936 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38091,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":1092,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":7874,"spinlock_wait_cycles":10719360,"thread_start_us":117894,"threads_started":1,"wal-append.queue_time_us":186962}
I20260709 13:08:29.314035 22829 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:29.213840 27095 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.271000 27096 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:29.279532 27060 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:29.262627 27005 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:29.323724 23450 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:29.324965 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:29.292339 22750 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.308328 22642 maintenance_manager.cc:643] P 359166ef74e042839c4e58133432df83: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 5.538s	user 0.182s	sys 0.105s Metrics: {"bytes_written":29968,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":320693,"dirs.run_cpu_time_us":484,"dirs.run_wall_time_us":32658,"drs_written":1,"lbm_read_time_us":381,"lbm_reads_lt_1ms":4,"lbm_write_time_us":227026,"lbm_writes_1-10_ms":4,"lbm_writes_10-100_ms":8,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":373820,"threads_started":2,"update_count":15,"wal-append.queue_time_us":451364}
I20260709 13:08:29.329578 23453 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 8 LEARNER]: Advancing to term 9
I20260709 13:08:29.330559 22831 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:08:29.331154 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.339363 22829 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:29.349193 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.344985 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.326145 27088 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.332000 26953 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:29.354990 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.362066 22750 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.341666 27039 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.328886 26996 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:29.399961 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 46400.3us
I20260709 13:08:29.405855 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1670) took 1051 ms. Trace:
I20260709 13:08:29.358512 27048 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:29.406270 22901 rpcz_store.cc:276] 0709 13:08:28.354775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.357408 (+  2633us) service_pool.cc:224] Handling call
0709 13:08:28.364041 (+  6633us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:28.364398 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:28.364424 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:28.368331 (+  3907us) log.cc:844] Serialized 52 byte log entry
0709 13:08:28.479027 (+110696us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:28.479065 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.479127 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:29.237980 (+758853us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:29.363423 (+125443us) raft_consensus.cc:1685] finished
0709 13:08:29.363458 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.405786 (+ 42328us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1581,"raft.queue_time_us":2366,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":20810,"spinlock_wait_cycles":5173632,"thread_start_us":61971,"threads_started":1,"wal-append.queue_time_us":189584}
I20260709 13:08:29.334668 26951 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.338598 26974 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.419473 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.420369 22905 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:08:29.432149 23447 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:29.332379 27122 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.408192 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:29.363009 26813 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:29.438330 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.439965 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.441663 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:29.442185 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.441682 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.443037 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:29.443899 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.449638 22896 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:29.450165 23448 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:29.448091 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2255) took 1003 ms. Trace:
I20260709 13:08:29.451356 23451 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:29.450999 22900 rpcz_store.cc:276] 0709 13:08:28.444383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.585079 (+140696us) service_pool.cc:224] Handling call
0709 13:08:28.585364 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:28.624158 (+ 38794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:28.624185 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:28.630632 (+  6447us) log.cc:844] Serialized 26 byte log entry
0709 13:08:28.982505 (+351873us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:28.982558 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.430805 (+448247us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b5400184554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:29.430850 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:29.430894 (+    44us) raft_consensus.cc:1685] finished
0709 13:08:29.430908 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.448019 (+ 17111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10997,"raft.queue_time_us":51822,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":63123840,"thread_start_us":328915,"threads_started":1,"wal-append.queue_time_us":446692}
I20260709 13:08:29.409150 27039 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.457105 22901 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:29.343467 27087 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:29.459468 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 51351.5us
I20260709 13:08:29.356159 27005 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.460251 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:29.461036 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66406.8us
I20260709 13:08:29.425366 27048 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.442301 27060 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:29.461344 26953 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:29.469004 27060 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.470173 26951 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.470798 26951 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:29.441867 27096 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:29.475682 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.441653 27028 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.490548 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.464474 26813 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:29.449838 26721 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:29.500653 23450 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:29.502820 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.503515 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.364967 27088 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.504333 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.473673 27048 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:29.499240 27115 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:08:29.512192 22643 maintenance_manager.cc:419] P 359166ef74e042839c4e58133432df83: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 11548 bytes on disk
I20260709 13:08:29.520843 22901 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:29.522162 26864 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:29.503854 26997 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.499169 27130 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:29.540298 27130 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:29.540787 27128 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.541746 27128 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.522157 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.542824 22752 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:29.543084 22642 maintenance_manager.cc:643] P 359166ef74e042839c4e58133432df83: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":306,"lbm_reads_lt_1ms":4}
I20260709 13:08:29.498489 27119 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:29.539606 26997 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.544234 27128 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.541430 27130 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:29.498780 27062 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.557107 27062 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:29.564102 27120 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:08:29.565234 27062 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:29.565591 27095 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.566344 26721 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.557107 27013 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:29.562340 27117 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 13:08:29.549737 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 87261.4us
I20260709 13:08:29.525149 27107 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:08:29.506248 27028 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.575472 27028 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:29.577816 27122 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.578778 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1671) took 1216 ms. Trace:
I20260709 13:08:29.579281 22903 rpcz_store.cc:276] 0709 13:08:28.362331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.362451 (+   120us) service_pool.cc:224] Handling call
0709 13:08:28.362749 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:28.363048 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:28.363071 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:28.363465 (+   394us) log.cc:844] Serialized 26 byte log entry
0709 13:08:29.226194 (+862729us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b180041c440. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:08:29.348996 (+122802us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:29.349028 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.349085 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:29.578447 (+229362us) raft_consensus.cc:1685] finished
0709 13:08:29.578477 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.578710 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":294,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":4232,"spinlock_wait_cycles":4172672,"thread_start_us":122644,"threads_started":1,"wal-append.queue_time_us":1212524}
I20260709 13:08:29.581674 27087 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.582442 27099 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.584316 27088 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.585775 26974 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.589905 26996 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.591289 27039 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:29.599339 23451 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
W20260709 13:08:29.604490 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 144300us
I20260709 13:08:29.512627 27076 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.605688 22871 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:29.611966 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 261244us
I20260709 13:08:29.612764 22871 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:29.613649 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:29.614193 23451 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:29.607491 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:29.622586 23451 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
W20260709 13:08:29.624423 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 161211us
I20260709 13:08:29.627158 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.625176 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:29.646106 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:29.654129 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.655856 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:29.655453 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:29.655347 22747 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:29.658524 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.659090 23445 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:29.660185 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.632484 27099 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:29.661386 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.613448 27076 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.663092 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.664142 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.654332 27039 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:29.671941 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.665086 23449 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:29.666354 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.654332 27119 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:29.644677 27135 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:08:29.673763 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.653398 26927 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:08:29.674880 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.797449 22826 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:29.775139 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:29.782871 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:29.798527 22824 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:29.775995 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.787662 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.782516 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.775471 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.792601 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.800323 22905 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:29.800371 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:29.783784 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.788975 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.801965 22747 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:29.700526 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.803119 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:29.803344 22831 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:29.803669 22823 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:29.662465 26996 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:29.788126 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:29.807039 22898 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:29.794480 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.784911 22825 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:29.702759 22901 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:08:29.795529 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:29.687446 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:29.804101 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:29.809731 22823 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:29.809963 22900 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:29.810295 22900 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:29.810380 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.688495 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:29.697768 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 193876us
I20260709 13:08:29.811681 22828 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:29.805936 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:29.811031 22900 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.812913 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:29.816677 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:29.817242 22755 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:29.818969 27126 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:29.819552 27126 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:29.818151 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:29.829692 27087 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:29.830106 27087 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:29.830477 27087 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:29.833118 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:29.834162 27138 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:29.835422 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:29.836133 27138 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:29.812481 27100 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:08:29.838299 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:29.841503 23449 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:29.671823 27039 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.845608 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:29.807413 27107 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:29.817219 27095 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:29.854030 27138 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:29.843703 27135 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:29.937048 27135 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:29.940004 27135 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:29.942677 27076 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:28.322563 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:08:29.910043 27146 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:08:29.910614 26909 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:29.911077 27143 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:29.971477 27143 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.971863 27143 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:29.857702 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:29.973685 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 116071us
I20260709 13:08:29.858610 27039 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:29.974639 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:29.976110 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:29.982661 27124 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:29.983166 27124 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:29.984833 27139 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:29.985620 27139 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.985988 27139 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:29.954384 27095 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:29.915912 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.844822 27126 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:29.912253 27148 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:08:29.816870 27088 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:29.669092 26974 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.157s
I20260709 13:08:30.100117 27088 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.100603 27088 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:29.933691 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:30.101214 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 167736us
I20260709 13:08:30.102581 22650 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33010 (request call id 225) took 7879 ms. Trace:
I20260709 13:08:29.817364 26996 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:29.984390 27048 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:29.924665 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.105410 27130 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:30.109120 26996 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.111205 27048 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:30.111868 22902 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:30.112507 26974 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.105710 27129 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:08:30.113013 26974 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.113878 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.114986 26974 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.105168 27095 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:30.125165 27039 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.127843 27039 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:30.091506 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 115457us
W20260709 13:08:30.033025 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.020367 27135 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:30.144709 27135 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:30.077085 27110 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.145787 27110 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.146788 27135 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:30.147796 27110 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.000088 27013 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:29.983526 27124 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:29.991696 26721 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.153584 26721 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:30.155632 26721 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:30.156786 27005 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.159842 27143 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.166549 22827 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:30.105399 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:30.105997 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.170092 22828 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:30.130045 27139 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.108541 27123 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:29.991473 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:30.020133 27024 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:30.174968 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 183545us
I20260709 13:08:30.175634 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:30.176800 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.182379 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:30.174688 26721 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.180063 27130 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:08:30.183753 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 78429.1us
I20260709 13:08:30.184118 26977 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:30.184408 27110 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:30.102975 22650 rpcz_store.cc:276] 0709 13:08:22.222714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.222877 (+   163us) service_pool.cc:224] Handling call
0709 13:08:30.102486 (+7879609us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:26.499381 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:26.499948 (+   567us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:26.499994 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:08:26.500012 (+    18us) tablet.cc:662] Decoding operations
0709 13:08:26.504345 (+  4333us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:26.504440 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:26.504469 (+    29us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:08:26.505651 (+  1182us) tablet.cc:701] Row locks acquired
0709 13:08:26.505700 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:08:26.505856 (+   156us) write_op.cc:270] Start()
0709 13:08:26.505968 (+   112us) write_op.cc:276] Timestamp: P: 1783602506505796 usec, L: 0
0709 13:08:26.505987 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:08:26.506662 (+   675us) log.cc:844] Serialized 5916 byte log entry
0709 13:08:26.891433 (+384771us) op_driver.cc:464] REPLICATION: finished
0709 13:08:27.145030 (+253597us) write_op.cc:301] APPLY: starting
0709 13:08:27.148190 (+  3160us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:27.353969 (+205779us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:27.354095 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:27.488789 (+134694us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:27.489176 (+   387us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=30
0709 13:08:27.489223 (+    47us) write_op.cc:312] APPLY: finished
0709 13:08:27.489582 (+   359us) log.cc:844] Serialized 107 byte log entry
0709 13:08:27.489752 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:27.489950 (+   198us) write_op.cc:454] Released schema lock
0709 13:08:27.490015 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:27.490470 (+   455us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:27.490508 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":693457,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":218436,"apply.run_cpu_time_us":5857,"apply.run_wall_time_us":359448,"num_ops":15,"prepare.queue_time_us":179355,"prepare.run_cpu_time_us":20301,"prepare.run_wall_time_us":100717,"raft.queue_time_us":96756,"raft.run_cpu_time_us":60560,"raft.run_wall_time_us":220453,"replication_time_us":360383,"thread_start_us":382459,"threads_started":4,"wal-append.queue_time_us":135508,"wal-append.run_cpu_time_us":6742,"wal-append.run_wall_time_us":1839414}]]}
I20260709 13:08:30.174913 27123 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.189873 27143 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.169344 27095 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:29.988346 27087 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.208068 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:30.191625 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 52537.3us
I20260709 13:08:30.216179 22902 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:30.206643 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.218930 27110 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:30.219866 27110 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:30.222472 27124 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.224436 22830 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:30.203150 26909 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:30.225797 26909 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:30.225934 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:30.226918 26909 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.227777 27087 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.228096 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.228386 27087 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.207981 27123 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:30.201364 26721 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.230746 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.231700 27138 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.231876 26721 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.203364 26977 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.232213 27138 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.232914 26977 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:30.200510 26974 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:30.215071 27053 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:08:30.234341 27138 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.215348 27095 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:30.235915 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.236488 22749 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:30.236968 27095 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:30.238236 22750 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:30.239045 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.240217 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:30.241705 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.242487 22823 raft_consensus.cc:2410] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:30.243052 27139 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.228844 27087 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.245478 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.246815 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.200335 27130 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.260268 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:30.252367 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:30.185232 27024 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.272218 22899 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:30.252192 22752 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:30.264541 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.225046 27124 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.279325 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:30.285449 22871 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:30.257432 27127 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.289381 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:30.271706 27130 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:30.246124 27084 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.290472 22897 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:30.271764 27021 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.297804 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.297544 27099 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:30.293987 27118 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:30.295447 27076 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:30.306756 27021 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:30.309645 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.309965 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.309067 27021 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:30.284392 27138 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:30.334782 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 82432.6us
I20260709 13:08:30.282763 26974 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.285099 27053 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.286955 27110 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:30.337690 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:30.340241 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:30.293838 26927 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.287900 26996 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.342480 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.308882 22652 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:30.368713 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:30.371124 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 78516.3us
I20260709 13:08:30.370342 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:30.341385 27110 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:30.327068 27084 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.368551 22652 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index -1 to 7, term changed from 8 to 9, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to <none>, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 9 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:08:30.391580 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:30.413654 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2540) took 1131 ms. Trace:
W20260709 13:08:30.414614 27161 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:08:30.414194 22652 catalog_manager.cc:5697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:30.400049 27060 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:08:30.416190 27060 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:30.416540 27060 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:30.418928 27084 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.419518 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:30.296769 27127 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:30.372547 27013 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:30.428447 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:30.341562 26974 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.340224 27138 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.367285 26927 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.410182 22901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:30.441224 22901 raft_consensus.cc:606] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:30.415643 22652 catalog_manager.cc:5697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 9, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 9 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:30.365684 27143 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:30.428831 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:30.445159 22902 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:08:30.401739 27110 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:30.355038 27095 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:30.414000 22822 rpcz_store.cc:276] 0709 13:08:29.281853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.281982 (+   129us) service_pool.cc:224] Handling call
0709 13:08:29.298887 (+ 16905us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:29.299226 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:29.299255 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:29.299666 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:08:29.347589 (+ 47923us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:29.347619 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.347674 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:30.408552 (+1060878us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b180003b080. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:30.413383 (+  4831us) raft_consensus.cc:1685] finished
0709 13:08:30.413405 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:30.413603 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142763,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":17827,"spinlock_wait_cycles":22795648,"thread_start_us":42095,"threads_started":1,"wal-append.queue_time_us":188682}
I20260709 13:08:30.422966 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.443208 22899 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:08:30.442742 22652 catalog_manager.cc:5697] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:30.446971 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:30.452087 22825 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:30.454284 22652 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 7 to 11, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 11 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:30.456243 22652 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:30.428359 27145 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:30.458364 22830 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:30.458424 22652 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 11, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:30.368110 27133 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:30.430431 27107 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:30.464229 22896 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:30.431301 27013 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:30.355703 27053 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.463884 22652 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 7 to 8, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 8 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:30.469470 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.474277 22749 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:30.369187 26996 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.475437 22652 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 12 to 13, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 13 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:30.479403 22652 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 7 to 10, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 10 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
W20260709 13:08:30.477510 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60381.5us
I20260709 13:08:30.482194 22825 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:30.481925 22652 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 9, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 9 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:30.464131 27145 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:30.483793 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 113512us
I20260709 13:08:30.461464 27143 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:30.485455 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:30.466007 27107 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.488380 27107 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:30.488531 27095 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:30.489079 27095 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.489542 27095 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:30.491056 26977 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.492964 26977 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.493724 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:30.496017 22899 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:30.467566 27013 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:30.496655 22899 raft_consensus.cc:606] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:30.497148 22899 raft_consensus.cc:614] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:30.485596 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.429112 27060 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.508296 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:30.508757 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.487816 27145 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:30.483986 22652 catalog_manager.cc:5697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:30.511157 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:30.487123 27087 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:30.512051 27156 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:08:30.498780 27024 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.514845 22652 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 11, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 11 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:30.515457 26977 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:08:30.517728 26974 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.517899 27157 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:30.519374 26927 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:30.517765 27158 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:08:30.519819 26997 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:30.520741 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 719029us
W20260709 13:08:30.521386 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 709493us
I20260709 13:08:30.522631 23419 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:30.524020 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:30.523801 27130 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:30.526190 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.530633 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.531245 22755 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:30.526012 22652 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 11, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:30.532289 22652 catalog_manager.cc:5697] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 3 to 5, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 5 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:30.531203 22826 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:30.533485 22652 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 11, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:30.512554 27076 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:30.529649 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:30.530025 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:30.524885 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:30.535287 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:30.540041 23444 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:08:30.540297 22901 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:30.535732 27022 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.529045 22914 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-135e2f7520a34b53aaf4ddb360a8a029 received from {username='slave'} at 127.0.0.1:42490
I20260709 13:08:30.536618 22752 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:30.537213 26957 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:30.537838 27127 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:30.542541 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:30.543607 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:30.551066 22902 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:30.544327 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:30.552066 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.552798 22826 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:30.538944 22652 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:30.548472 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.549654 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:30.557554 23444 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:08:30.559023 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.561033 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.562585 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:30.538192 27005 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:30.563956 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.568104 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.536619 27028 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:30.577107 27176 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:30.541755 26804 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.579425 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:30.584568 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:30.545642 27060 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:30.586573 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 43731.5us
I20260709 13:08:30.577812 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.558923 27039 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:30.601120 22902 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:30.585225 22914 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-135e2f7520a34b53aaf4ddb360a8a029 on tablet 135e2f7520a34b53aaf4ddb360a8a029 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.612668 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.624929 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.626014 26927 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:30.630458 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.632298 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:30.604143 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.557195 26738 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:30.568012 27144 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:30.566115 26953 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:30.608011 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.550604 26957 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.634124 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:30.614166 23452 raft_consensus.cc:2410] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:08:30.635066 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.635784 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.650511 23445 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:08:30.651477 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.607009 22826 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:30.582731 27099 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:30.653986 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.654728 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:30.586369 27028 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.636929 27039 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.563865 27157 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 13:08:30.641598 26738 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.675479 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.648828 27024 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.658753 26957 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:30.693200 23451 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:30.657673 26909 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:30.662549 27028 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:30.676617 27039 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:30.552023 27143 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:30.723785 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.647183 26977 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:08:30.716289 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:30.736892 23445 raft_consensus.cc:2410] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:08:30.636729 26804 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.664295 27176 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:30.699702 26738 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:30.721911 27088 consensus_meta.cc:287] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.000s
W20260709 13:08:30.746759 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 114711us
I20260709 13:08:30.700891 27127 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:30.785818 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:30.743753 27123 consensus_meta.cc:287] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.000s
W20260709 13:08:30.746071 27162 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:08:30.788031 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:30.783931 27005 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:30.784664 27088 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.802469 26804 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:30.803951 27088 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.805660 27005 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.806519 27168 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:30.741446 27176 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.721525 27024 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:30.903753 27169 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:08:30.895359 27176 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:30.913933 26962 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:30.914535 26962 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:30.791316 26472 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.915251 26472 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.929092 27024 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:30.930521 27175 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:08:30.931454 27013 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.790584 27053 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:30.788760 27133 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:30.875126 27118 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:30.790819 27107 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.821731 27123 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:30.935039 22830 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:08:31.368645 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:31.369067 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.369413 27107 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.369629 22828 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:08:31.090377 22900 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:31.050331 27147 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:31.214334 23449 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.014s	sys 0.051s
W20260709 13:08:31.372074 23449 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.020s	sys 0.063s
I20260709 13:08:30.803886 26804 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:31.372305 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:31.176959 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:31.376817 22719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.000s	sys 0.009s
W20260709 13:08:31.380470 22719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.000s	sys 0.009s
W20260709 13:08:31.380929 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 726194us
W20260709 13:08:31.385890 22870 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.005s	sys 0.007s
W20260709 13:08:31.386731 22870 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.005s	sys 0.007s
W20260709 13:08:31.366729 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 788913us
W20260709 13:08:31.013895 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.389081 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:31.391553 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.394344 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.394611 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.394976 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.379001 22826 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:31.387145 27118 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.398718 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.403885 23451 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:31.399207 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:31.405604 22902 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:31.399574 22900 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:30.913249 26738 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.390081 27179 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:31.415529 26738 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.416162 27118 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.419710 26721 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.419503 23451 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:31.421197 27087 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:31.421705 27087 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:31.420334 26721 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.423393 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.430562 26721 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.432794 26721 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:31.434494 26721 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:31.376778 27139 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.005s	sys 0.005s
I20260709 13:08:31.423593 27087 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.371222 27147 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.442094 27157 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:31.453121 27096 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:31.443207 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.445945 26860 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:31.483536 26860 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:31.446235 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.841s (SENT) [suppressed 44 similar messages]
W20260709 13:08:31.292629 26957 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.000s
I20260709 13:08:31.058959 27028 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.450439 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.451056 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.378654 26804 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:30.945683 27130 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.452060 27138 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.443894 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2383) took 1336 ms. Trace:
I20260709 13:08:31.476200 27058 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:31.464496 22829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.011s	sys 0.023s
I20260709 13:08:31.467900 23446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
W20260709 13:08:31.511381 22829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.011s	sys 0.024s
I20260709 13:08:31.511591 23446 raft_consensus.cc:606] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:31.512060 23446 raft_consensus.cc:614] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:31.217940 27127 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:31.049055 27013 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:31.412532 22900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1677) took 1725 ms (client timeout 2102 ms). Trace:
I20260709 13:08:31.058612 26321 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:31.453004 27039 consensus_meta.cc:287] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.710s	user 0.004s	sys 0.001s
I20260709 13:08:31.467466 26721 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:31.401377 27122 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:31.477216 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:31.442607 27173 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:31.479188 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:31.512809 26990 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.021s	sys 0.026s
W20260709 13:08:31.483106 27110 consensus_meta.cc:287] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.018s	user 0.010s	sys 0.000s
W20260709 13:08:31.524996 27145 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.009s	user 0.008s	sys 0.003s
I20260709 13:08:31.487937 22899 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:31.449918 27187 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:31.475754 27139 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.005s	sys 0.005s
I20260709 13:08:31.464390 26959 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.395205 27126 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:08:31.494725 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:08:31.700608 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 205937us
I20260709 13:08:31.701395 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.497746 22826 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:31.052110 27176 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.477579 27106 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.506108 22753 rpcz_store.cc:276] 0709 13:08:30.107574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.107691 (+   117us) service_pool.cc:224] Handling call
0709 13:08:30.112306 (+  4615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:30.165791 (+ 53485us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:30.334066 (+168275us) log.cc:844] Serialized 14 byte log entry
0709 13:08:30.507220 (+173154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:30.507271 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:30.507318 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.443704 (+936386us) spinlock_profiling.cc:243] Waited 53.3 ms on lock 0x7b54003f11d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:31.443743 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:31.443846 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1322,"prepare.queue_time_us":199580,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":1041,"raft.queue_time_us":36992,"spinlock_wait_cycles":130219136,"thread_start_us":340788,"threads_started":3,"wal-append.queue_time_us":144000}
I20260709 13:08:31.498911 26962 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.509840 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:31.397137 27133 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:31.516629 22756 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:31.517736 22900 rpcz_store.cc:269] 0709 13:08:29.686600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.687197 (+   597us) service_pool.cc:224] Handling call
0709 13:08:31.090187 (+1402990us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b58008018d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d570aa 00007fcb74dd9447 00007fcb74df1fe7 00007fcb74df37de 00007fcb74dff27c 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c
0709 13:08:31.412189 (+322002us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2013019904}
I20260709 13:08:31.519364 23444 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:31.519611 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.502200 26804 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:31.499465 27119 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:31.498111 27022 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:31.498199 27107 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:31.485682 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1687) took 1057 ms. Trace:
I20260709 13:08:31.711252 22896 rpcz_store.cc:276] 0709 13:08:30.428124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.428234 (+   110us) service_pool.cc:224] Handling call
0709 13:08:31.485494 (+1057260us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b540035cad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:31.485618 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80895232}
I20260709 13:08:31.540638 23445 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:08:31.712533 27189 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:08:31.713449 23445 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:08:31.698560 26959 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:31.717680 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1809) took 1498 ms. Trace:
I20260709 13:08:31.718425 27028 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:31.718294 27107 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.603274 27173 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.719224 26959 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:31.702723 26984 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:31.600065 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 52861.9us
W20260709 13:08:31.601737 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 165147us
I20260709 13:08:31.523728 27039 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.603806 22827 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:31.498818 26957 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:31.594027 27060 consensus_meta.cc:287] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.915s	user 0.008s	sys 0.002s
W20260709 13:08:31.544327 26990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.026s	sys 0.026s
I20260709 13:08:31.497109 27178 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 13:08:31.593343 26860 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:31.674328 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 251151us
I20260709 13:08:31.752882 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.754593 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:31.675197 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 228934us
I20260709 13:08:31.592978 27016 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:31.517400 27058 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:31.543776 27076 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:31.462795 22905 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:31.519382 27127 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:31.523803 27188 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.161s
I20260709 13:08:31.706894 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.767625 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:31.768098 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 61273.5us
I20260709 13:08:31.768602 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:31.544574 27110 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:31.544348 22828 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:31.703821 27095 consensus_meta.cc:287] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.214s	user 0.011s	sys 0.000s
I20260709 13:08:31.739612 27130 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:31.721760 22825 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:31.723241 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.609858 27187 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:31.723937 27022 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.724653 27119 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.725636 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 787) took 1083 ms. Trace:
I20260709 13:08:31.719885 26765 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.601881 27145 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.545908 26321 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.726811 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.787890 22823 raft_consensus.cc:2410] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:31.718149 22829 rpcz_store.cc:276] 0709 13:08:30.219696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.223746 (+  4050us) service_pool.cc:224] Handling call
0709 13:08:30.224042 (+   296us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:30.282667 (+ 58625us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:30.282710 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:30.288598 (+  5888us) log.cc:844] Serialized 52 byte log entry
0709 13:08:30.811526 (+522928us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b1800186160. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:08:31.511848 (+700322us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:31.511896 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.511980 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:31.717379 (+205399us) raft_consensus.cc:1685] finished
0709 13:08:31.717397 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:31.717627 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58166,"raft.queue_time_us":60698,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":8951,"spinlock_wait_cycles":60604672,"thread_start_us":700110,"threads_started":1,"wal-append.queue_time_us":1196706}
I20260709 13:08:31.790059 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.708350 27021 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:31.713089 27176 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.777769 27174 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.758708 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:31.762547 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:31.804723 22824 raft_consensus.cc:2410] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:31.768778 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:31.542973 27181 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.155s
I20260709 13:08:31.770646 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:31.816642 22751 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:31.816881 27181 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:31.776113 27190 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:31.817250 27181 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:31.790637 26765 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.817857 27190 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:31.818292 27190 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:31.818701 27190 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:31.818943 27191 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:31.768519 27016 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.822561 26721 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.823153 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.823835 23452 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:31.824702 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.824723 26721 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.782904 27130 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:31.828014 27130 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:31.831666 27173 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.770699 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.833792 27145 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.834280 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.835031 22822 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:31.836067 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.771461 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.838187 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1169) took 1252 ms. Trace:
I20260709 13:08:31.838240 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.838721 23450 rpcz_store.cc:276] 0709 13:08:30.585812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.586086 (+   274us) service_pool.cc:224] Handling call
0709 13:08:30.586424 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:31.026937 (+440513us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:31.252831 (+225894us) log.cc:844] Serialized 14 byte log entry
0709 13:08:31.606450 (+353619us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:31.670892 (+ 64442us) log.cc:844] Serialized 52 byte log entry
0709 13:08:31.671035 (+   143us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:31.671239 (+   204us) log.cc:844] Serialized 14 byte log entry
0709 13:08:31.677588 (+  6349us) log.cc:844] Serialized 14 byte log entry
0709 13:08:31.699346 (+ 21758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.699420 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:31.837983 (+138563us) raft_consensus.cc:1685] finished
0709 13:08:31.838006 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:31.838128 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52340,"prepare.queue_time_us":229894,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":63,"raft.queue_time_us":174520,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":119804,"spinlock_wait_cycles":16131712,"thread_start_us":737555,"threads_started":2,"wal-append.queue_time_us":534487}
I20260709 13:08:31.840194 23445 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:31.772048 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:31.721550 22826 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:31.841670 27119 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.841908 22900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:31.842173 22826 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:31.769423 22905 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:31.777130 22753 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:31.723537 27107 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.783457 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.766561 27060 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:31.783938 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.769469 27076 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:31.710853 27106 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.849807 23451 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:31.766376 27138 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:31.781399 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 775) took 1242 ms. Trace:
I20260709 13:08:31.740157 26984 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.851799 23448 rpcz_store.cc:276] 0709 13:08:30.538660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.546873 (+  8213us) service_pool.cc:224] Handling call
0709 13:08:30.581503 (+ 34630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:30.962149 (+380646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:30.962180 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:31.366770 (+404590us) log.cc:844] Serialized 26 byte log entry
0709 13:08:31.591964 (+225194us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:31.592014 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.592102 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:31.781181 (+189079us) raft_consensus.cc:1685] finished
0709 13:08:31.781199 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:31.781329 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35469,"raft.queue_time_us":135907,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":14489,"thread_start_us":493349,"threads_started":2,"wal-append.queue_time_us":341228}
I20260709 13:08:31.853825 22825 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:08:31.854157 22825 pending_rounds.cc:85] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 8
I20260709 13:08:31.854411 22825 pending_rounds.cc:107] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:08:31.854647 22825 raft_consensus.cc:2889] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:31.855955 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:31.781993 23449 rpcz_store.cc:276] 0709 13:08:30.641969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.642233 (+   264us) service_pool.cc:224] Handling call
0709 13:08:30.642616 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:31.372463 (+729847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:31.372492 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:31.398178 (+ 25686us) log.cc:844] Serialized 52 byte log entry
0709 13:08:31.490118 (+ 91940us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:31.490175 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.490329 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:31.725381 (+235052us) raft_consensus.cc:1685] finished
0709 13:08:31.725402 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:31.725550 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109605,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":6037,"thread_start_us":779912,"threads_started":2,"wal-append.queue_time_us":239172}
I20260709 13:08:31.776636 27184 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.748379 26991 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:08:31.856704 27184 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.857164 26991 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:31.857506 27022 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.857604 26991 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:31.858469 26991 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.859354 26957 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.859668 26991 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:31.802817 27147 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.791167 26951 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:31.782382 27196 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:31.861018 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.776494 27053 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.861454 27196 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:31.788977 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.862219 27053 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:31.780512 27095 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.815264 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:31.868955 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.869581 27095 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.869654 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.870226 22898 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:31.818153 27181 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.871078 27181 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:31.871587 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.851425 27106 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.854636 26984 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.874732 27110 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:31.876266 27127 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.858901 27184 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.708595 27048 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:31.878086 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.877231 27076 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:31.878763 27123 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.879504 27133 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:31.880563 27076 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.882727 26996 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:31.840099 27058 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:31.811815 27187 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.863390 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.813997 27174 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.864426 22829 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:31.865257 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.866183 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.894496 23451 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:31.867869 27196 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:31.815995 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 47304.8us
I20260709 13:08:31.873103 27053 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.908828 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.785409 22904 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:31.909370 22900 raft_consensus.cc:606] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:31.909587 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.909734 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.910300 23447 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:31.911098 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.856020 27039 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.912371 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.912657 27174 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:31.909754 22900 raft_consensus.cc:614] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:31.915808 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.916540 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.916833 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.917528 22823 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:31.888057 22828 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:31.892577 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:31.917980 22825 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:31.907493 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.917999 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:31.918813 23452 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:31.919055 23449 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:31.918953 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:31.920797 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:31.920583 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.921576 22826 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:31.906738 23451 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:08:31.922931 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.923074 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.923588 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.924005 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:31.924149 22828 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:31.927268 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:31.815682 26804 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.929152 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.931068 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:31.931429 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:31.932798 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:31.932833 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.933616 22905 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:31.934432 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:31.935449 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.936478 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.937484 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:31.861224 26951 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:31.938424 22831 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:31.938450 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.939994 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.940848 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:31.941576 22831 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:31.940817 22828 raft_consensus.cc:2410] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:08:31.943073 23452 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:31.943710 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:31.944588 23448 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:31.946748 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:31.915361 27084 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.954973 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:31.958530 22756 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:08:31.962008 22756 pending_rounds.cc:85] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 4
I20260709 13:08:31.962240 22756 pending_rounds.cc:107] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:08:31.962529 22756 raft_consensus.cc:2889] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:31.925714 27136 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.968860 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:31.964718 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.925185 26984 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.973366 22905 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:31.970981 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.961696 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.974120 23445 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:31.939332 27110 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:31.977388 22753 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:31.939553 27107 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:31.976224 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:31.979946 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:31.980751 22753 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:31.980887 27180 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:31.949386 26991 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:32.008430 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:31.962728 27173 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:31.965754 27126 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.919828 27058 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:08:32.008466 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:32.020182 22905 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:31.951089 26962 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:08:32.023239 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.018736 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:31.982216 27123 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:31.978698 27143 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:31.986557 27095 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:31.973472 26959 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:08:32.008131 26991 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.984388 27021 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:08:32.036813 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:32.037168 22751 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:32.037509 22903 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:32.016431 27173 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.035118 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:31.984314 27028 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:08:32.033058 27095 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:32.049525 27095 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:32.049654 27028 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:32.050242 27028 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:32.051299 27028 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.052807 27028 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:32.036814 26991 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:32.025843 27126 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.058950 27095 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.061144 27095 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:32.044155 27208 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:08:32.035439 27048 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:08:32.063225 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:32.064260 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:32.064915 27048 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:32.045943 26738 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:32.046705 27119 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:31.978430 27136 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.048069 26804 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.042330 27190 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.061870 27184 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:31.972832 27084 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.067180 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.065270 27048 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:32.096171 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 116603us
I20260709 13:08:32.096585 22871 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:32.065222 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:32.100212 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 138586us
I20260709 13:08:32.097060 22902 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:32.099390 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.101403 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:31.982806 26984 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.102953 22900 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:08:32.103049 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 59560.7us
I20260709 13:08:32.103900 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.061070 27126 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:32.070680 27173 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:32.105775 22896 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:32.107759 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.068564 26321 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.070415 27016 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:32.106231 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.109218 27048 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.105598 26864 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:32.105465 27110 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:32.122107 27110 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.122610 27110 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:32.112511 27190 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:32.104650 26804 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:32.125792 27190 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:32.103257 26472 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:32.112524 27210 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:08:32.116878 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.117169 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.137346 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.118878 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.138863 22903 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:32.118340 22753 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
W20260709 13:08:32.115872 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 144994us
I20260709 13:08:32.119398 26951 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:32.104889 27184 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.142711 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:32.143460 22900 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:32.144011 27184 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.144202 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.144609 27184 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:32.144791 22900 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:32.121587 26984 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.145645 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.148685 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:32.145824 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:32.148648 27024 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:32.149608 27196 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:32.149637 26984 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:32.150220 27196 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:32.150368 26984 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:32.151075 27196 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.151209 26984 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.151708 26765 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.151934 26991 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.154345 27157 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:32.154747 27107 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:32.155416 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:32.156936 27217 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:32.157424 27217 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:32.146574 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.158236 27217 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.159511 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.161180 22903 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:32.160212 22900 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
W20260709 13:08:32.161855 23444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 784) took 1538 ms (client timeout 2000 ms). Trace:
W20260709 13:08:32.164167 23444 rpcz_store.cc:269] 0709 13:08:30.623701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.632005 (+  8304us) service_pool.cc:224] Handling call
0709 13:08:32.161659 (+1529654us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b5400671454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:32.161803 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030242688}
I20260709 13:08:32.161947 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.161986 23450 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:32.166199 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:32.166910 22900 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:32.170298 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:32.173712 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.183303 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:32.178256 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.185470 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:32.186879 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.187959 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.189481 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:32.189886 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.190789 23447 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:32.192667 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:32.193914 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.194693 22903 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:32.175858 26927 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.196115 26927 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.196487 27136 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:32.176113 27024 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:32.198048 26927 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.154795 27157 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:32.198175 27024 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:32.199352 27157 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.200680 27024 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:32.200866 27157 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:32.201571 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.202361 22824 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:32.202630 27194 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:32.191962 27118 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:32.203670 27118 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.204017 27118 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:32.205555 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:32.206547 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.207410 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.189577 27021 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:32.203091 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.208905 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:32.207805 26864 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:32.216413 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.282649 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:32.283025 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 388169us
I20260709 13:08:32.279330 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.278851 22823 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:32.276151 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.286116 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:32.247968 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:32.286903 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.228078 22900 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:32.287710 22903 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:32.203125 27194 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.288342 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.288506 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:32.280547 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:32.290216 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.182893 27095 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:32.290318 22897 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:32.292622 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.248754 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.291563 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.292033 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.252511 22902 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:08:32.293599 22824 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:32.294238 22902 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:32.294399 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.295171 22824 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:32.192613 27022 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:32.296821 22828 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:32.299872 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:32.299995 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.168642 27173 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.300884 23447 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:32.295877 22755 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:32.223004 27138 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:32.332170 22902 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:08:32.331399 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.384104 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:32.522842 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.422863 27182 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:32.377454 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1001) took 1851 ms. Trace:
I20260709 13:08:32.380960 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.524153 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:32.303838 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.530256 22899 rpcz_store.cc:276] 0709 13:08:30.525744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.529873 (+  4129us) service_pool.cc:224] Handling call
0709 13:08:31.487745 (+957872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:31.592367 (+104622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:32.377240 (+784873us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b540035cad0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:32.377392 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2202268928}
W20260709 13:08:32.364562 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 64621.1us
I20260709 13:08:32.331066 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.366946 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.369880 22747 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:32.446377 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:32.412003 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.532689 22831 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:32.436085 22897 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:32.449788 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.385181 27194 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:32.535228 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 231471us
W20260709 13:08:32.536561 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.537367 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:32.340093 27053 raft_consensus.cc:1064] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
W20260709 13:08:32.433878 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.302661 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.543891 22827 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
W20260709 13:08:32.339218 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 53948.2us
I20260709 13:08:32.275858 27096 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:32.445956 22829 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:08:32.341529 27095 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.215s
I20260709 13:08:32.545344 22829 pending_rounds.cc:85] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 8
I20260709 13:08:32.547547 22829 pending_rounds.cc:107] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:08:32.547833 22829 raft_consensus.cc:2889] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:32.546881 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.547052 27196 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:32.550454 26738 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:32.550937 26738 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:32.551992 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 102312us
I20260709 13:08:32.551815 26738 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.354341 26991 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:32.512547 27130 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.555598 27130 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.325143 26951 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:32.556524 26951 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.413921 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:32.559677 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 145848us
I20260709 13:08:32.561014 27217 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:32.531379 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:32.250272 27170 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.567087 27110 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.567943 27215 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:32.568079 22902 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:32.568470 27215 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:32.569409 27110 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:32.570098 27178 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:32.416944 27198 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.587s	user 0.003s	sys 0.000s
I20260709 13:08:32.570588 27178 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.571154 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.571882 27147 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:32.572130 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 791) took 1186 ms. Trace:
I20260709 13:08:32.572530 27178 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.572501 23453 rpcz_store.cc:276] 0709 13:08:31.385684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.385811 (+   127us) service_pool.cc:224] Handling call
0709 13:08:31.443857 (+ 58046us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:31.444220 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:31.444250 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:31.444709 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:08:31.619637 (+174928us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:31.619683 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.619766 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:32.416343 (+796577us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:32.571886 (+155543us) raft_consensus.cc:1685] finished
0709 13:08:32.571906 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:32.572077 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1942,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":80,"spinlock_wait_cycles":144035200,"thread_start_us":174541,"threads_started":1,"wal-append.queue_time_us":333531}
I20260709 13:08:32.574836 27147 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:32.576839 27107 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:32.576476 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:32.578660 23450 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:08:32.580768 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:32.439692 27048 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:32.292675 27187 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.248s
I20260709 13:08:32.593019 27053 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.405128 27118 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.542796 27173 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:32.357688 27211 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.595252 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.595489 27211 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.595896 27118 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:32.576756 27147 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:32.557507 27130 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.512518 27139 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:32.558251 26321 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:32.597718 27211 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:32.530014 27182 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:32.597784 27139 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.599893 27139 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:32.567708 27170 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:32.420851 27209 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.440s	user 0.002s	sys 0.000s
I20260709 13:08:32.600704 22824 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:32.601864 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1698) took 1127 ms. Trace:
I20260709 13:08:32.358009 27024 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.602726 27024 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.604668 26321 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:32.604956 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.597119 27147 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.341161 27084 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:32.540045 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:32.582582 23450 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:32.582614 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:32.585144 22830 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:32.302901 27136 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:32.587076 23447 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:32.625440 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.626055 22828 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:32.588222 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:32.588462 27143 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.588788 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.589069 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:32.589407 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.627192 22829 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:32.628021 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.590512 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.591851 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:32.629494 22748 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:32.629369 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:32.592132 26951 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:32.602309 22901 rpcz_store.cc:276] 0709 13:08:31.474013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.474287 (+   274us) service_pool.cc:224] Handling call
0709 13:08:31.474630 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:31.731568 (+256938us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:31.742022 (+ 10454us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:31.749020 (+  6998us) log.cc:844] Serialized 26 byte log entry
0709 13:08:31.924979 (+175959us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:31.925025 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:32.420253 (+495228us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:32.428619 (+  8366us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54003e9254. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:32.428681 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:32.601611 (+172930us) raft_consensus.cc:1685] finished
0709 13:08:32.601637 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:32.601784 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":320,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":12131,"spinlock_wait_cycles":640409728,"thread_start_us":146648,"threads_started":1,"wal-append.queue_time_us":216256}
I20260709 13:08:32.574520 27215 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.640877 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:32.641667 23453 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:32.641937 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.642939 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:32.645615 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:32.562640 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.383396 27163 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.430548 22896 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:08:32.605769 22752 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:32.599056 27182 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:32.427983 22904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1693) took 1884 ms (client timeout 1540 ms). Trace:
W20260709 13:08:32.651571 22904 rpcz_store.cc:269] 0709 13:08:30.543681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.544076 (+   395us) service_pool.cc:224] Handling call
0709 13:08:32.427788 (+1883712us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400207954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:32.427907 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2836013312}
I20260709 13:08:32.609270 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:32.652860 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 43664us
I20260709 13:08:32.653517 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:32.653985 26321 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:32.654386 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.654639 27194 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:08:32.654877 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 92315.7us
I20260709 13:08:32.655009 27194 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:32.580050 27119 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:32.622458 27187 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:32.622324 27107 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.612053 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:32.637077 22831 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:32.640079 22748 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:32.604687 27024 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:32.662603 22797 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:32.662720 26951 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:32.663784 27005 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:32.664187 27005 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.664510 27005 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:32.582506 27217 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:32.655586 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:32.626178 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 86217.2us
I20260709 13:08:32.609404 27096 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:32.659206 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:32.660527 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48565.6us
I20260709 13:08:32.655437 27194 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:32.635466 27157 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:32.667155 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.683604 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.693228 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:32.668605 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.693006 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:32.655597 27119 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.696995 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.696164 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:32.632720 27143 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.665508 27182 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:32.705649 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:32.707087 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:32.666555 27060 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:32.659998 27211 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:08:32.752772 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 45767.6us
I20260709 13:08:32.756072 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:32.667608 27084 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:32.683216 27170 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.696940 27119 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:32.695071 27173 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:32.758984 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.694823 27076 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:32.703393 26927 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:32.736510 27127 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:32.736007 26957 raft_consensus.cc:2764] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:08:32.815714 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56797.8us
I20260709 13:08:32.819247 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:32.820277 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.682862 27217 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.667245 27107 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:32.737936 27157 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:32.736989 27180 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:32.666221 27138 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.769075 27028 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:32.755788 26813 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:32.696597 27187 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:08:32.753755 27060 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:32.769090 26991 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:32.756604 27211 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:32.772990 27084 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:32.816138 27076 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:32.814154 27173 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.825174 27039 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:32.822703 27127 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.828778 26927 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:32.836488 27180 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.669018 27163 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.863494 27157 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:32.873436 27076 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:32.875797 27173 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:32.871841 27211 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:32.860262 26813 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:32.892817 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.892796 27127 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:32.865764 26991 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.871544 27084 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:32.891619 22903 raft_consensus.cc:606] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:32.912343 22903 raft_consensus.cc:614] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:32.875844 27039 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.842070 26765 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:08:32.878031 27028 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:32.814891 27096 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.892254 27060 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.911453 22754 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:32.913513 22900 raft_consensus.cc:2410] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:08:32.896617 27180 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:32.914912 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:32.917600 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 259930us
I20260709 13:08:32.918128 27140 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:32.918485 27163 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:32.918586 27183 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:32.897333 27223 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:32.908340 26813 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:32.928115 27028 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:32.920537 26991 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:32.922601 27039 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:32.912242 27107 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:32.932129 27229 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:08:32.963027 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:32.932651 27096 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:32.935482 27217 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:32.933730 27223 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.932423 27183 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:32.935055 27140 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:32.968312 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 275148us
W20260709 13:08:32.964144 27185 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.029s	sys 0.068s
I20260709 13:08:32.982185 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:32.968477 27143 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:33.009814 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.986779 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:32.981519 27183 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:32.984030 27185 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.161s	user 0.029s	sys 0.069s
I20260709 13:08:32.978384 27223 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:32.946300 26927 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:33.036484 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49764.3us
W20260709 13:08:33.052030 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.984858 27217 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:08:33.030810 27095 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:33.031922 27118 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:33.030229 27215 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:33.102720 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 397107us
I20260709 13:08:33.072950 27243 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:33.036530 27143 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:08:33.115212 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 283405us
I20260709 13:08:33.118072 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:33.118681 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.119360 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:33.120877 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:32.932197 26738 consensus_queue.cc:1175] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c45e1805f049467697a7358fb566de2f" responder_term: 14 status { last_received { term: 13 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:33.127485 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:33.122641 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:33.118072 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:33.123728 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:33.128067 22901 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:33.128607 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:33.128831 22719 connection.cc:256] RPC call timeout handler was delayed by -1.1885120079999751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:08:33.068977 27140 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.126156 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:33.126490 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:33.135290 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:33.142690 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:33.144212 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:33.145881 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:33.146637 23453 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:33.187559 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58800 (request call id 34) took 1030 ms. Trace:
I20260709 13:08:33.192694 22903 rpcz_store.cc:276] 0709 13:08:32.109054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.288391 (+179337us) service_pool.cc:224] Handling call
0709 13:08:33.139103 (+850712us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54003bfbd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74df5853 00007fcb79dc360a 00007fcb733497d4 00007fcb73349762 00007fcb73349691 00007fcb7334960c 00007fcb733488c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:33.139197 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385679616}
I20260709 13:08:33.188385 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:33.190121 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:33.199770 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:33.127170 27170 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:33.190191 23445 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:08:33.101415 27118 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:33.212790 23453 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 9 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:08:33.217128 22828 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:08:33.115736 27095 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.122612 27215 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:33.187676 27239 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.074s
W20260709 13:08:33.258882 27233 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:08:33.212246 27243 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
W20260709 13:08:33.273392 27238 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
W20260709 13:08:33.277846 27234 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
W20260709 13:08:33.288456 27228 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 13:08:33.289247 27241 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
W20260709 13:08:33.290066 27235 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 13:08:33.229434 27028 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:33.311182 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 244408us
I20260709 13:08:33.311928 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:33.343645 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:33.313616 27237 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.252s	user 0.001s	sys 0.001s
I20260709 13:08:33.354765 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1023) took 1206 ms. Trace:
I20260709 13:08:33.355284 22900 rpcz_store.cc:276] 0709 13:08:32.148242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.288142 (+139900us) service_pool.cc:224] Handling call
0709 13:08:33.354528 (+1066386us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54003bfbd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:33.354694 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435478272}
I20260709 13:08:33.359874 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:33.351271 27217 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:33.309918 26984 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.363557 27118 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:33.397176 27028 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:33.350303 27005 consensus_meta.cc:287] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.000s
W20260709 13:08:33.554250 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 435539us
W20260709 13:08:33.498927 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.323375 27157 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:33.564599 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:33.640530 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 76015.2us
I20260709 13:08:33.641606 26321 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:33.587632 27248 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:33.567894 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.644488 27248 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:33.583786 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 453319us
I20260709 13:08:33.646517 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:33.646678 27248 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:33.648758 27076 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.739s	user 0.004s	sys 0.002s
I20260709 13:08:33.649842 26321 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.595050 22824 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:08:33.570170 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.584336 27028 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.563730 26984 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.610284 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:33.619843 27107 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:08:33.624181 27244 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 13:08:33.658661 26321 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.627373 27095 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:33.628319 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2828) took 1018 ms. Trace:
I20260709 13:08:33.637552 27247 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:33.628808 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2574) took 1045 ms. Trace:
I20260709 13:08:33.650357 27147 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:33.672983 22752 rpcz_store.cc:276] 0709 13:08:32.610285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.663560 (+ 53275us) service_pool.cc:224] Handling call
0709 13:08:32.663827 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:32.664171 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:32.664199 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:32.664607 (+   408us) log.cc:844] Serialized 26 byte log entry
0709 13:08:33.121066 (+456459us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:33.121102 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:33.121164 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:33.628044 (+506880us) raft_consensus.cc:1685] finished
0709 13:08:33.628072 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:33.628240 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":272,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":456194,"threads_started":1,"wal-append.queue_time_us":692686}
I20260709 13:08:33.614449 26472 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:33.653537 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:33.567085 27122 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:08:33.673008 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2578) took 1088 ms. Trace:
I20260709 13:08:33.673583 26321 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:33.656384 26984 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:33.687032 23450 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.034s	sys 0.060s
I20260709 13:08:33.687209 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:33.674520 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:33.687510 22825 rpcz_store.cc:276] 0709 13:08:32.584735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.584946 (+   211us) service_pool.cc:224] Handling call
0709 13:08:32.587940 (+  2994us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:32.588252 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:32.588280 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:32.701476 (+113196us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b2c00000e98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:08:32.702060 (+   584us) log.cc:844] Serialized 52 byte log entry
0709 13:08:32.972312 (+270252us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:33.190145 (+217833us) log.cc:844] Serialized 14 byte log entry
0709 13:08:33.217709 (+ 27564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:33.217778 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:33.672689 (+454911us) raft_consensus.cc:1685] finished
0709 13:08:33.672797 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:33.672956 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":325254,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":71257,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":13399,"spinlock_wait_cycles":78852352,"thread_start_us":358108,"threads_started":2,"wal-append.queue_time_us":388935}
I20260709 13:08:33.693550 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:33.664109 27107 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:33.652747 27173 consensus_meta.cc:287] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.741s	user 0.004s	sys 0.001s
I20260709 13:08:33.638154 26765 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.654111 27250 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:33.673655 22827 rpcz_store.cc:276] 0709 13:08:32.583315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.587388 (+  4073us) service_pool.cc:224] Handling call
0709 13:08:32.589171 (+  1783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:32.589745 (+   574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:32.589777 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:32.702714 (+112937us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b2c00000e98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:08:32.703090 (+   376us) log.cc:844] Serialized 26 byte log entry
0709 13:08:32.932240 (+229150us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:32.932274 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:32.932364 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:33.628407 (+696043us) raft_consensus.cc:1685] finished
0709 13:08:33.628438 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:33.628727 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":975,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"spinlock_wait_cycles":75639808,"thread_start_us":176204,"threads_started":1,"wal-append.queue_time_us":280520}
W20260709 13:08:33.695369 23450 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.034s	sys 0.060s
I20260709 13:08:33.694216 22750 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
W20260709 13:08:33.688366 27138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.051s	sys 0.091s
W20260709 13:08:33.692166 27253 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:08:33.649179 27076 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.585917 27005 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:33.685628 26472 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:33.708649 26991 consensus_meta.cc:287] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.758s	user 0.006s	sys 0.001s
I20260709 13:08:33.708351 26472 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.680346 27147 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:33.709040 26991 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.709491 27147 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:33.703737 26765 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.710108 23447 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:08:33.710649 27028 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:33.711634 26765 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:33.698050 27250 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:33.712142 26991 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:33.712767 27250 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:33.683197 27157 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:33.703823 22750 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:33.713747 27250 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:33.715438 27119 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.898s	user 0.010s	sys 0.000s
I20260709 13:08:33.715926 27119 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.716521 22750 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:33.704116 27138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.051s	sys 0.091s
I20260709 13:08:33.699522 27107 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:33.755170 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:33.758774 22750 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:08:33.763303 22754 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:33.751097 26472 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:33.673483 27247 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.827944 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:33.832111 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:33.847754 22900 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:33.851358 22897 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:08:33.761045 27127 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.838s	user 0.010s	sys 0.000s
I20260709 13:08:33.852757 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:33.859880 22896 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:33.744059 27095 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:33.853582 22903 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:33.750728 27215 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:33.755632 27194 consensus_meta.cc:287] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.061s	user 0.008s	sys 0.004s
I20260709 13:08:33.861960 22900 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:33.713793 27157 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.701224 27173 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.844108 27138 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.813983 27058 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:33.845970 26721 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:33.845685 26927 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:33.751649 27179 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.113s
W20260709 13:08:33.913024 23446 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.008s	sys 0.017s
W20260709 13:08:33.913492 23446 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.008s	sys 0.017s
W20260709 13:08:33.925606 22831 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.002s	sys 0.024s
W20260709 13:08:33.875842 27096 consensus_meta.cc:287] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.905s	user 0.009s	sys 0.002s
W20260709 13:08:33.948890 22831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.002s	sys 0.024s
W20260709 13:08:33.895169 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 248701us
W20260709 13:08:33.906268 22828 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.002s	sys 0.021s
I20260709 13:08:33.885210 22750 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
W20260709 13:08:33.933390 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 273149us
I20260709 13:08:33.874931 27257 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:33.885926 27182 consensus_meta.cc:287] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.118s	user 0.009s	sys 0.002s
I20260709 13:08:33.871809 27145 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:33.885038 27084 consensus_meta.cc:287] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.959s	user 0.012s	sys 0.000s
I20260709 13:08:33.883649 26321 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:33.940115 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.010s	sys 0.017s
I20260709 13:08:33.946983 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 2392) took 1715 ms. Trace:
I20260709 13:08:33.949018 27013 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:33.863428 27127 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:33.951756 22828 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.002s	sys 0.021s
W20260709 13:08:33.954195 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.865494 27095 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:33.957788 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.010s	sys 0.017s
W20260709 13:08:33.970556 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 653035us
I20260709 13:08:33.961602 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:33.971345 22897 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:33.957051 22756 rpcz_store.cc:276] 0709 13:08:32.231077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.231604 (+   527us) service_pool.cc:224] Handling call
0709 13:08:33.946720 (+1715116us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54003f0cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:33.946889 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3350631296}
I20260709 13:08:33.866736 27194 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:33.958423 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 598598us
W20260709 13:08:33.949867 27087 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.008s	sys 0.018s
I20260709 13:08:33.975086 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:33.975989 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:33.965999 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.978125 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:33.965929 27179 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:33.883351 27058 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.980253 22896 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:33.980317 27179 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.981832 27058 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:33.953047 27119 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:33.887907 27138 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.993498 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:33.914655 26927 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:33.994349 22748 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:08:33.966650 27013 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:33.914263 26721 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:33.983870 27087 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.008s	sys 0.019s
I20260709 13:08:34.015774 27194 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:34.017887 27260 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:08:34.018072 27060 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:34.021343 27119 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:34.021579 27266 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:34.021862 27119 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.023700 27119 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:34.024420 26721 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.026111 27268 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:34.028121 27250 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:34.028486 27250 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:34.028692 27119 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:34.029191 27119 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.031052 27119 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:34.031911 26813 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.082s	user 0.008s	sys 0.000s
I20260709 13:08:34.032317 26813 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.033490 27076 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:34.034158 27211 consensus_meta.cc:287] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.113s	user 0.005s	sys 0.004s
I20260709 13:08:33.970017 26321 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.034564 27211 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.036098 26321 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.036409 27211 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.037732 27247 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:34.038934 27147 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.040802 27147 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.041592 27013 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.042317 26927 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.042588 27013 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:34.043030 27179 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.003563 27274 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:34.043781 27274 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.045786 27274 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.046231 27127 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.048207 27095 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:33.967316 27257 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.049209 27173 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.006179 26738 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:33.995301 27024 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.050428 27257 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.050347 27024 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:33.969762 27084 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:34.052388 27024 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.052706 27084 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:33.979156 27028 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:08:34.053794 27157 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:33.977797 26765 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:08:34.056030 27005 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:33.992877 27058 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:34.057873 27058 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.059749 27058 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:34.061911 27039 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.107s	user 0.009s	sys 0.001s
I20260709 13:08:34.062278 27039 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.062814 27138 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.062958 22797 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:33.970839 27145 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.065613 27145 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.066555 27145 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:34.067250 27145 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:34.068116 27145 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.085467 22897 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
W20260709 13:08:34.070602 27242 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.857s	user 0.000s	sys 0.004s
I20260709 13:08:34.070618 27170 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:34.075130 27183 consensus_meta.cc:287] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.013s	user 0.009s	sys 0.000s
I20260709 13:08:34.077780 26765 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:08:34.040715 27180 consensus_meta.cc:287] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.106s	user 0.004s	sys 0.005s
I20260709 13:08:34.081405 27107 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:34.089560 27223 consensus_meta.cc:287] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.025s	user 0.009s	sys 0.004s
I20260709 13:08:34.090461 27170 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:34.093389 27267 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.093266 26765 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:34.108902 27223 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.114149 26765 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.115501 27223 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.115715 26765 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:34.101580 27183 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.128347 27183 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.106946 27039 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.102259 27180 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.112326 27250 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.135054 27267 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.148006 26813 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:34.164345 27107 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:34.165606 27180 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.166051 27267 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.169840 27239 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:34.170490 27239 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:34.178876 27262 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:34.179404 27262 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:34.180419 27262 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.191929 27262 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:34.182453 27251 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.679s	user 0.001s	sys 0.008s
I20260709 13:08:34.183984 27217 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:34.192888 27217 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:34.193884 27217 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.171567 27239 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.195284 27239 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:34.195197 27217 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:34.196422 27250 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:34.182480 27170 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.204639 27170 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.240676 22751 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:08:34.241714 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:34.242630 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.242738 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2404) took 1661 ms. Trace:
I20260709 13:08:34.247037 22756 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:34.246457 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.248010 22750 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:34.248715 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.249692 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.250773 22752 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:34.251569 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1836) took 1060 ms. Trace:
I20260709 13:08:34.253666 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 802) took 1128 ms. Trace:
I20260709 13:08:34.254120 23446 rpcz_store.cc:276] 0709 13:08:33.124650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.124775 (+   125us) service_pool.cc:224] Handling call
0709 13:08:33.186330 (+ 61555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:33.186597 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:33.934912 (+748315us) log.cc:844] Serialized 14 byte log entry
0709 13:08:34.252799 (+317887us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:34.252927 (+   128us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:34.253083 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.253190 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:34.253418 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4114,"prepare.queue_time_us":696354,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":48,"raft.queue_time_us":25246,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":78702,"thread_start_us":1015691,"threads_started":3,"wal-append.queue_time_us":138308}
I20260709 13:08:34.252019 22831 rpcz_store.cc:276] 0709 13:08:33.190882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.191016 (+   134us) service_pool.cc:224] Handling call
0709 13:08:33.191344 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:33.191570 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:33.276231 (+ 84661us) log.cc:844] Serialized 14 byte log entry
0709 13:08:34.250679 (+974448us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:34.250809 (+   130us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:34.250954 (+   145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.251069 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:34.251301 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22884,"prepare.queue_time_us":119894,"prepare.run_cpu_time_us":8647,"prepare.run_wall_time_us":34104,"raft.queue_time_us":36349,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":7226,"thread_start_us":1027431,"threads_started":3,"wal-append.queue_time_us":732825}
W20260709 13:08:34.259160 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.260396 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:34.263850 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:34.265897 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 243953us
I20260709 13:08:34.267627 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:34.268146 22753 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:08:34.269079 27136 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.270005 27136 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:34.274189 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 285418us
I20260709 13:08:34.275424 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:34.275730 27136 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.276362 27136 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:34.277109 26984 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:34.277787 26984 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:34.278376 26984 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:34.281975 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.283365 22756 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:34.284274 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.285370 22755 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:34.286966 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:34.287293 26765 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:34.288347 27107 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:34.289788 27107 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:34.290213 27107 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:34.290644 27107 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:34.291785 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.243450 22747 rpcz_store.cc:276] 0709 13:08:32.581076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.653466 (+ 72390us) service_pool.cc:224] Handling call
0709 13:08:32.668074 (+ 14608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:32.668272 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:32.668293 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:32.668655 (+   362us) log.cc:844] Serialized 26 byte log entry
0709 13:08:33.137680 (+469025us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:33.137717 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:33.137782 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:34.181700 (+1043918us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:34.241815 (+ 60115us) raft_consensus.cc:1685] finished
0709 13:08:34.241869 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:34.242429 (+   560us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16348416,"thread_start_us":365964,"threads_started":1,"wal-append.queue_time_us":550772}
I20260709 13:08:34.293012 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1253) took 1073 ms. Trace:
I20260709 13:08:34.294188 22754 rpcz_store.cc:276] 0709 13:08:33.219687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.219836 (+   149us) service_pool.cc:224] Handling call
0709 13:08:33.220111 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:33.763708 (+543597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.292498 (+528790us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b5400045e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:34.292815 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248825216}
I20260709 13:08:34.296579 27048 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:34.283537 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.298382 22750 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:34.293617 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2830) took 1653 ms. Trace:
I20260709 13:08:34.301640 22748 rpcz_store.cc:276] 0709 13:08:32.639633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.928721 (+289088us) service_pool.cc:224] Handling call
0709 13:08:32.936314 (+  7593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:33.996071 (+1059757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.289279 (+293208us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540016b054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:34.293381 (+  4102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432480000}
I20260709 13:08:34.308914 23453 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:08:34.287335 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:34.310581 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:34.310925 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:34.312132 26957 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:34.312681 26957 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:34.313151 26957 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:34.313637 26765 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:34.314265 26765 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:34.317286 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:34.318142 22828 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:34.315047 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:34.319435 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:34.319664 22827 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:34.320819 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:34.316100 26765 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.322348 23453 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:34.323522 27145 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:34.324433 27145 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:34.326380 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.329344 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:34.330312 23446 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:34.332775 27145 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.303625 27048 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.304942 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:34.306437 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2835) took 1213 ms. Trace:
I20260709 13:08:34.344216 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:34.342483 22751 rpcz_store.cc:276] 0709 13:08:33.093386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:33.096182 (+  2796us) service_pool.cc:224] Handling call
0709 13:08:33.099243 (+  3061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:34.241995 (+1142752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.306196 (+ 64201us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540016b554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:34.306386 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2623052032}
I20260709 13:08:34.350558 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:34.353579 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48867.4us
I20260709 13:08:34.356503 27287 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:34.357028 27287 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:34.357553 27287 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:34.332958 26957 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.357771 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:34.358348 27266 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:34.358633 22828 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:34.359247 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:34.360569 22748 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:34.361442 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:34.362941 27275 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.363778 27275 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.365773 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:34.348915 27048 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.367069 22752 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:08:34.368104 27048 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.369884 27266 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.370657 27266 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.375550 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.377293 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:34.382169 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:34.382567 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:34.383230 22900 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:34.383719 22899 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:34.367657 27275 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.373027 27266 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.387553 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:34.397305 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:34.393620 27290 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:34.398954 27290 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:34.399196 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:34.399412 27290 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:34.400034 22828 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:34.402758 27263 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.403492 27263 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.404927 27226 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.395978 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:34.411005 22831 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:34.410871 27226 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.412894 22752 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:34.394130 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.407269 27267 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.365306 27107 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.410007 27263 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:34.409981 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:34.343837 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 369786us
W20260709 13:08:34.408634 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.415652 22751 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:34.414616 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:34.417919 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.415691 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.417217 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.419340 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:34.416489 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.420009 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:34.437768 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:34.433980 26984 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.439100 22754 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:34.414875 27267 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.440759 26984 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:34.413823 27272 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:34.436348 22828 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:34.443636 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:34.445355 22749 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:34.447263 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:34.447541 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:34.447955 23450 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:34.448187 22751 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:34.448819 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:34.418411 27107 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:34.413246 27226 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:34.455986 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.456167 27263 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:34.456630 27263 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:34.457087 27263 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:34.457043 27288 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:34.458570 27267 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:34.461429 27288 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:34.461836 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:34.462570 22825 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:34.446868 27272 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:34.400800 27290 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:34.931351 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.618206 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:34.573598 22831 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
W20260709 13:08:34.877462 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 262043us
I20260709 13:08:34.466357 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:34.466174 22900 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:34.567198 22751 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
W20260709 13:08:34.874696 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 294804us
W20260709 13:08:34.790062 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:35.076807 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.608701 22752 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:34.501914 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 92086.1us
I20260709 13:08:34.482833 23446 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:35.068019 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:34.465184 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:35.086422 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:34.465538 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 471902us
I20260709 13:08:35.090430 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:34.495049 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:35.011900 27250 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.453819 26957 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:35.079278 27226 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.463660 27288 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:35.101403 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:35.099948 23453 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:35.103782 23450 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:35.102890 22899 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:35.016322 27291 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.871876 27107 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.233s
I20260709 13:08:34.872901 27257 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:34.506949 27287 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.110531 22903 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:35.118335 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:35.117460 27288 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:35.120688 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.131470 27272 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:35.132597 27272 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:08:35.133782 27262 consensus_meta.cc:287] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.856s	user 0.010s	sys 0.007s
I20260709 13:08:35.135102 23447 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:35.136344 22827 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:35.138082 27107 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:35.146622 27288 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.150470 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:35.110103 27226 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.123256 27250 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.133083 27272 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:35.169441 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 59965.4us
I20260709 13:08:35.187347 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.188006 27174 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.188655 27174 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.189127 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:35.190804 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.169623 27272 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:35.193073 22903 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:35.196067 27174 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:35.205395 27296 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:35.206224 27296 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:35.212411 27302 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:08:35.161569 27291 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.201650 27290 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:35.224625 22899 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:35.230082 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:35.185076 27257 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:35.234192 22830 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:35.223584 27250 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:35.232467 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:35.233140 27288 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:35.246793 27250 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.248718 27296 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.275321 27226 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:35.275852 27299 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.173s	user 0.004s	sys 0.000s
W20260709 13:08:35.278774 27301 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.194s	user 0.000s	sys 0.002s
I20260709 13:08:35.264518 27287 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.269914 27174 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:08:35.283239 27170 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:35.290512 27226 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:35.297816 27226 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:35.298015 27263 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.840s	user 0.006s	sys 0.006s
I20260709 13:08:35.298422 27263 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.298799 27226 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.271538 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:08:35.300153 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 100501us
I20260709 13:08:35.292455 27170 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:35.308992 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:35.311156 27296 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:35.313516 26957 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:35.293649 27287 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.317307 22752 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:35.316272 27170 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:35.305716 27250 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.319151 26957 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:35.328953 27250 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.323266 27291 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.330374 26957 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:35.369874 27263 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:35.370496 27263 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:35.371078 27263 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:35.373399 26813 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.098s	sys 0.109s
W20260709 13:08:35.373781 26813 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.098s	sys 0.110s
I20260709 13:08:35.375334 26813 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:35.375730 26813 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:35.385460 27263 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.376717 26813 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.386451 26813 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:35.396113 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 132710us
I20260709 13:08:35.364625 27287 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.384852 27272 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.415565 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:35.421995 27272 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:35.414213 27076 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:35.442413 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 351928us
W20260709 13:08:35.451084 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48445us
I20260709 13:08:35.442255 27288 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:35.459697 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:35.451964 27118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.005s	sys 0.040s
I20260709 13:08:35.459223 27263 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:35.466547 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:35.473063 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.476432 22748 raft_consensus.cc:2410] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:08:35.476879 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:35.486773 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.480368 22750 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:08:35.469347 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:35.466542 27118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.006s	sys 0.040s
I20260709 13:08:35.491884 27039 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:35.492358 27039 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:35.493566 27039 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.495190 27039 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:35.499147 27226 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:35.503082 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 310981us
W20260709 13:08:35.505563 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 49214.5us
W20260709 13:08:35.506572 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:35.508958 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:35.526650 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.505156 27287 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.517504 26765 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:35.487025 27263 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.520090 27306 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:35.538030 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 229148us
W20260709 13:08:35.540239 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 461153us
I20260709 13:08:35.533922 26765 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:35.536525 27118 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:08:35.548552 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.496665 26957 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.549371 26765 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:35.548918 27118 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:35.556874 22797 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:35.558355 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:35.559674 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:35.561002 27287 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:35.561417 27287 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:35.561621 27272 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:35.561796 27287 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:35.563454 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:35.564111 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:35.566540 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:35.567322 27076 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:35.567723 27076 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:35.568074 27076 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:35.573427 27308 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:35.574079 27308 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:35.575172 27308 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:35.579169 27303 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:08:35.581465 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:35.582247 27174 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:35.582759 27174 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:35.583770 22752 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:35.583526 27174 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:35.584084 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.585995 27118 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.587005 27118 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:35.588361 22750 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:35.542872 27170 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.590236 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.590890 22752 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:35.590924 27170 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.591643 27170 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:35.553354 27306 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.594297 27306 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.594807 27313 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.595377 27313 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.596566 22752 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:35.596909 22752 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:35.597816 22752 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.598956 22752 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:35.600255 27257 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
W20260709 13:08:35.601115 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 82945.6us
I20260709 13:08:35.602697 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:35.604530 27310 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.071s	user 0.002s	sys 0.000s
I20260709 13:08:35.605600 27250 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:35.605760 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1056) took 1183 ms. Trace:
I20260709 13:08:35.606060 27250 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.606524 27288 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:08:35.607522 27305 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:08:35.607638 27275 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:35.608273 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:35.614861 27313 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:35.618489 27300 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.536s	user 0.004s	sys 0.000s
I20260709 13:08:35.618685 27291 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:35.606534 27250 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:35.606106 22897 rpcz_store.cc:276] 0709 13:08:34.422582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.425040 (+  2458us) service_pool.cc:224] Handling call
0709 13:08:34.443502 (+ 18462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:34.471234 (+ 27732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:34.471274 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:35.113433 (+642159us) log.cc:844] Serialized 26 byte log entry
0709 13:08:35.440701 (+327268us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:35.440752 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:35.440839 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:35.604181 (+163342us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b240023dc08. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71b949c4 00007fcb71b92acb 00007fcb71d4ddc9 00007fcb71d4da55 00007fcb71d4d09d 00007fcb7497f10e 00007fcb7499688c 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221
0709 13:08:35.605368 (+  1187us) raft_consensus.cc:1685] finished
0709 13:08:35.605386 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:35.605574 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1195,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":141,"spinlock_wait_cycles":86402944,"thread_start_us":294748,"threads_started":1,"wal-append.queue_time_us":387434}
W20260709 13:08:35.620237 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58965.6us
I20260709 13:08:35.620776 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:35.608083 27275 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:35.617442 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:35.624894 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1049) took 1243 ms. Trace:
I20260709 13:08:35.625247 22896 rpcz_store.cc:276] 0709 13:08:34.379568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.379793 (+   225us) service_pool.cc:224] Handling call
0709 13:08:34.380282 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:34.380876 (+   594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:34.380907 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:34.381533 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:08:34.601958 (+220425us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:34.602012 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:34.602095 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:35.611339 (+1009244us) spinlock_profiling.cc:243] Waited 79.8 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:35.619586 (+  8247us) raft_consensus.cc:1685] finished
0709 13:08:35.619609 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:35.623241 (+  3632us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13520,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":165,"spinlock_wait_cycles":183616768,"thread_start_us":219920,"threads_started":1,"wal-append.queue_time_us":700263}
I20260709 13:08:35.630012 22752 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:08:35.630764 22752 pending_rounds.cc:85] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:08:35.629477 26813 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:35.637770 26813 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.638296 26813 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:35.617537 27039 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:08:35.642670 26984 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:35.643159 26984 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:35.639555 26957 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:35.644408 27314 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:35.630968 22752 pending_rounds.cc:107] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:08:35.647285 22752 raft_consensus.cc:2889] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:35.581251 27076 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:35.650866 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 87522.7us
I20260709 13:08:35.654920 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:35.656678 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:35.657402 27107 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:35.652320 27309 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:35.682637 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:35.683665 27323 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:35.684028 27323 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.684340 27323 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:35.684423 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:35.689006 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:35.691838 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:35.657105 27263 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.644302 26984 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.658314 27272 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:08:35.714766 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 253562us
I20260709 13:08:35.690831 27325 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.019s
W20260709 13:08:35.716917 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 107454us
I20260709 13:08:35.718717 27319 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:35.719067 27325 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.719767 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:35.719633 27325 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.724249 27325 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.724756 27325 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:08:35.726177 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 112689us
I20260709 13:08:35.726877 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:35.733227 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.734071 22752 raft_consensus.cc:2410] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:08:35.736685 26984 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:35.739390 22752 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:35.742910 27288 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:35.716166 27272 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.727284 23419 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:35.746929 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:35.749270 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:35.714026 27275 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.749676 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.750413 27275 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:35.746682 27272 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:35.754227 27291 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.754756 27291 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.756721 27291 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:35.740935 27263 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:35.757817 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 45550.6us
I20260709 13:08:35.759569 27076 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.761653 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:35.762331 23447 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:35.762435 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:35.763768 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:35.750415 22748 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:35.765115 27314 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:35.744333 27226 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:35.765726 27314 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:35.766458 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:35.766865 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.761485 27250 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.770190 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.779117 22751 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:08:35.779264 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.768805 22756 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:35.780293 22755 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:35.781610 22797 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:35.783839 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.785051 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:35.785663 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:35.784586 22747 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:35.768705 27290 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:35.770202 27296 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:35.786039 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:35.774845 27076 raft_consensus.cc:2764] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:08:35.792681 27290 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.796767 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:35.797116 27290 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:35.797535 22903 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:35.777155 27306 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:35.801506 27320 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:35.801949 27320 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.802249 27288 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:35.802960 27288 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:35.804486 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:35.804096 27288 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.796924 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:35.766759 27314 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.802361 27320 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:35.806433 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:35.807052 22896 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:35.774840 27226 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:35.802124 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 54424.7us
I20260709 13:08:35.805670 22899 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:35.786958 27322 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:08:35.814991 22897 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:35.744122 26765 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.761235 27287 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.816943 23447 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:35.808259 26813 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.817973 27226 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.816192 27118 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:35.813962 22903 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:35.829092 22900 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:35.831036 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:35.835193 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 68878us
W20260709 13:08:35.837170 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 145385us
I20260709 13:08:35.845238 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:35.847276 22899 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:35.831243 27107 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:35.831043 27250 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:35.849428 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.849196 27107 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:35.850780 26813 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.851492 27118 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.857161 27118 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.866858 22899 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:08:35.867226 22899 pending_rounds.cc:85] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 7
I20260709 13:08:35.869881 22899 pending_rounds.cc:107] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:08:35.870146 22899 raft_consensus.cc:2889] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:35.871769 22749 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:35.870018 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:35.873018 22904 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:35.874652 23446 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:35.875286 27323 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.873085 27320 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:35.876704 27174 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:35.888371 22904 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:08:35.888651 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:35.889519 27306 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:35.890362 27306 raft_consensus.cc:3055] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:35.890668 27306 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:35.890714 26765 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:35.891860 27324 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:35.891721 27306 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.892627 27324 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.893604 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:35.894817 27324 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.897392 22896 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:35.897846 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:35.893139 27306 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:35.907861 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:35.902660 23416 leader_election.cc:336] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.542s (SENT)
I20260709 13:08:35.922323 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:35.921268 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:35.924923 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:35.924929 22900 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:35.921283 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:35.931092 22896 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:35.867988 27290 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.925530 27324 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:08:35.940328 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.943145 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:35.949997 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:35.952543 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 66923.8us
I20260709 13:08:35.953127 26813 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:35.953567 26813 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.953909 26813 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:35.957654 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.959743 27272 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.963033 27320 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:35.965107 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:35.965165 27327 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:35.965816 27314 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:35.965905 27327 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.966372 27314 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.966492 27327 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:35.966804 27314 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:35.965849 22751 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:08:35.945839 27118 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.969724 27118 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:35.971827 27118 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.946573 27324 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.977844 27324 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:35.979228 22896 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:35.960079 27287 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:35.931278 27325 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:35.985407 22755 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:35.980871 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:35.965551 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 47062.3us
I20260709 13:08:35.988626 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:35.986755 22751 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:35.978772 27272 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.003342 22899 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:36.004252 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:36.011719 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:36.016743 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.942512 27288 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:36.020401 22897 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
W20260709 13:08:36.019003 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 67218.6us
I20260709 13:08:36.021916 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.024375 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.026736 22825 raft_consensus.cc:2410] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:08:36.027107 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.001457 27325 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:36.003290 27118 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:36.031589 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.029597 27322 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:36.028465 26984 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:36.036311 27076 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:36.040609 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.036738 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.037346 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.042063 22828 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:36.039139 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.042201 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:36.049259 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.057648 22830 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:36.057605 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.058161 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.059780 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.032290 26765 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:36.033842 27272 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:36.063760 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.063879 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.087285 22826 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:36.064442 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.064960 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.065852 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.067166 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.070128 27290 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:36.071067 26813 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.073683 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.082502 27107 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.084329 22899 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:36.084408 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.084960 22900 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:08:36.085407 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.088963 22823 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:36.064972 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.038698 27323 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.034382 27325 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:36.043730 27322 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.088351 23444 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:36.125224 22904 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:36.127079 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.129005 22896 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:36.130475 22897 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:36.123210 22824 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:36.132730 23450 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:36.134578 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.135988 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:36.089986 27310 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:36.138659 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.139286 23445 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:36.140259 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.142542 22825 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:36.142949 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 75860.3us
I20260709 13:08:36.134684 27076 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:36.147354 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.147898 22827 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:08:36.141427 27250 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.148811 27250 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:36.150744 27250 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:36.122624 26765 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:36.152661 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.152447 26765 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.153236 23446 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:36.097997 27118 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:36.064445 22653 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 9 to 8, leader changed from <none> to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 8 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:36.155867 22901 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:36.100939 27272 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.050462 26984 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.159351 26984 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:36.160954 22823 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:08:36.162856 26813 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.163514 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:36.141479 27107 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.145473 22822 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:36.168035 27272 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.168126 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:36.169370 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:36.169167 27314 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.171053 22753 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:36.171113 27314 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.129166 27322 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:36.176525 27118 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:36.179355 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 173621us
I20260709 13:08:36.197815 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.206599 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.207281 23453 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:36.213130 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.213927 22822 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:36.214846 23448 raft_consensus.cc:2410] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:08:36.180975 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.216051 22897 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:36.221522 22827 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:36.238847 22825 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
W20260709 13:08:36.245519 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:36.256784 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.258431 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.229722 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.258008 22797 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:36.264293 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:36.267354 22828 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:36.273648 22896 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:36.275499 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:36.281771 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:36.273514 27118 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:36.284621 22751 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:36.285418 27290 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:36.279271 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:36.288098 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.286551 27290 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:36.290511 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:36.295846 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:36.268508 27320 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:36.297834 27339 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:36.304476 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.300752 23452 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:36.299861 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:36.291752 22797 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:36.310652 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.311292 23449 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:36.300168 26984 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:08:36.313730 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.314590 23453 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:08:36.293920 27323 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:36.302060 27319 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:36.298400 27290 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:36.327688 27336 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:08:36.314977 27320 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:36.327263 22653 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 6 to 7, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 7 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:36.321470 26984 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:36.328823 27320 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:36.321619 27339 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:36.330389 27339 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.331952 27319 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.332849 27319 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.315146 27226 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:36.336596 27226 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.322283 27323 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:36.342422 27323 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:36.344144 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.344944 23453 raft_consensus.cc:2410] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:08:36.336975 27226 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:36.345672 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.346320 23444 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:36.346812 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.347393 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.348127 23452 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:08:36.348744 27290 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.349946 27290 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:36.352270 23451 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:36.353302 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.354094 22897 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:36.315707 27250 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:36.361953 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.357167 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:36.363878 23453 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:36.346320 22826 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:36.357613 27250 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.372290 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.375191 23449 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:36.380473 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.364815 27039 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.369457 27275 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:36.326284 27314 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:08:36.372431 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76679us
I20260709 13:08:36.386554 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:36.386953 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.389621 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.390296 23446 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:36.385423 27174 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:08:36.375556 27250 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:36.364794 27076 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:08:36.373636 27306 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.365341 27327 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.382741 27275 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.400883 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:36.401984 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:36.404726 26813 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:36.408525 26813 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.409543 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.409641 26813 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.410971 27308 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:36.411990 22828 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:36.412679 27296 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:36.413111 27296 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:36.414165 27296 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.415568 27296 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:36.405881 27324 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.416137 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:36.418267 27272 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:36.418823 27272 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.418927 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:36.430493 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.430835 23446 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:36.426574 27272 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:36.393559 27347 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:36.384647 27039 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.402618 27275 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:36.451588 22825 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 10. Ops: []
I20260709 13:08:36.439558 27306 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.457595 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.458163 23445 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:36.459375 27322 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:36.460258 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.460942 23447 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:36.461426 27322 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:36.467523 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:36.468106 22753 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:36.473517 23453 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:36.473670 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:36.474604 27039 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:36.445432 27326 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:36.475760 27326 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.476567 27326 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:36.476580 27327 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.446956 27347 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.483608 23453 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:36.485771 27325 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:36.485975 27347 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.488099 27325 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:36.459714 27324 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.494962 22822 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
W20260709 13:08:36.497713 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.520148 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.521687 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.522686 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.523669 22753 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:36.520936 22897 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:36.526222 22829 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 14. Ops: [12.12-12.12]
I20260709 13:08:36.526973 23445 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:36.499321 22755 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:08:36.497879 27250 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:36.528029 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 143174us
W20260709 13:08:36.532428 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:36.537799 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 118939us
I20260709 13:08:36.538863 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:36.543011 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:36.542147 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 153385us
I20260709 13:08:36.542953 22822 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:08:36.545004 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.546509 23419 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:36.549542 22755 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:08:36.551628 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:36.551777 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.553045 22831 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:36.553900 22904 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:08:36.553027 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.554769 22749 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:36.553728 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:36.549368 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.588s (SENT) [suppressed 14 similar messages]
I20260709 13:08:36.558876 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.559000 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.558825 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:36.559697 22751 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:36.563014 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.563915 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.563915 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.564859 23445 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:36.564628 23453 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 9 LEARNER]: Advancing to term 10
I20260709 13:08:36.566367 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:36.566895 22753 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:36.547186 27263 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:36.579214 22831 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:36.562729 22653 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 10, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 10 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:36.573470 22824 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:36.573576 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.580251 23446 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:36.574081 22827 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:36.581506 22827 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:36.537849 27226 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.585000 22827 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:36.543201 27324 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:08:36.588940 22753 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:36.590799 23446 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:36.592821 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.595115 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.596467 22899 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:36.597332 22831 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:36.598241 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.598883 23445 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:36.599058 22824 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:36.596407 22751 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:36.598347 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.601279 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.601475 27308 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:36.601550 22749 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:36.602782 22749 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:36.602222 22827 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:36.603585 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.607748 23453 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 10 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:36.588739 27174 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:36.612524 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.613842 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.587695 27263 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.609485 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.568344 27323 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.618660 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:36.593899 27306 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.621734 27287 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:36.621663 22749 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.602396 27272 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:36.623016 22749 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:36.596668 27319 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:36.621134 27308 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:36.619534 27324 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:36.596302 27107 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:36.627743 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 69251.9us
I20260709 13:08:36.631517 22827 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:08:36.634714 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:36.637686 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.637717 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.638898 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.641010 22826 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:36.640909 27309 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:36.642006 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.642715 27309 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:36.643062 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.643850 22823 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:08:36.632455 27118 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:36.648015 22896 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:08:36.648800 27325 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:36.649291 27325 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:36.649696 27325 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:36.649760 27250 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:36.653452 22824 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:36.638271 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.654479 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:36.655480 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.627436 27174 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:36.648697 27275 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.601382 27355 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:08:36.669972 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:36.690845 23453 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:08:36.670960 23447 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:36.671164 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.671411 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.671705 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.673544 27339 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:36.674378 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:36.692281 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.677690 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.693022 23451 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:36.693305 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:36.677421 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.693910 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.694010 22822 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:36.694815 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.695359 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:36.666095 27287 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:36.703032 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.702346 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.705885 22831 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:36.706218 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:36.670715 27118 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:36.699467 22829 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:36.678563 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.709044 23449 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:36.711694 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.712462 22825 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:08:36.713763 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.716886 22824 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:36.698974 27347 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:36.718001 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.698096 27356 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:36.720752 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.721436 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.721630 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.703603 27272 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:36.744544 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.723515 22822 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:36.720176 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.723977 22827 raft_consensus.cc:2376] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9. Re-sending same reply.
I20260709 13:08:36.725564 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.726611 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:36.727056 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.727540 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.727380 27174 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.729135 27323 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.721870 27107 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:08:36.731860 27325 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.734421 27275 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:36.736267 22719 connection.cc:256] RPC call timeout handler was delayed by -1.908100826000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
I20260709 13:08:36.738431 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:36.770948 22719 connection.cc:256] RPC call timeout handler was delayed by -1.6939008260000037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 13:08:36.771315 22719 connection.cc:256] RPC call timeout handler was delayed by -1.5733008260000361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21200000000000002s for a response
I20260709 13:08:36.771878 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:36.772212 22829 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:36.709463 27118 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.772595 27296 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:36.772784 27118 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:36.740830 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.712325 27319 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:36.773787 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.766593 23445 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:36.719390 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.712561 27324 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:36.776222 22905 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:36.767392 22902 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:36.767342 27347 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.782256 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.782621 27347 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.783411 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:36.784017 22822 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:36.715245 27263 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.789542 27226 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.790619 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.791285 23452 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:36.792721 23452 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 10 LEARNER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:08:36.793491 27319 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.794658 27325 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:36.794462 27319 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:36.767584 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.768409 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.798228 22825 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:36.799813 23451 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:36.800966 23448 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:36.772604 22829 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:36.761093 22749 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:08:36.805290 22749 pending_rounds.cc:85] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:08:36.773001 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:36.804641 22822 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:36.808593 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:36.825775 22824 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:36.829046 22905 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:36.834766 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.835825 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:36.836110 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.836534 22754 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:36.836225 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.837212 23453 raft_consensus.cc:2368] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11.
I20260709 13:08:36.838747 23444 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:36.805598 22749 pending_rounds.cc:107] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 9.9
I20260709 13:08:36.883445 22900 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:36.883203 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.884094 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.886708 22749 raft_consensus.cc:2917] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Aborting config change with OpId 9.9: Aborted: Op aborted by new leader
I20260709 13:08:36.887049 22904 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:36.887153 22749 raft_consensus.cc:2889] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:36.814354 27324 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.891660 22904 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:36.890363 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:36.890782 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 117861us
I20260709 13:08:36.890661 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.901118 22898 raft_consensus.cc:2410] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:08:36.838356 27296 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:36.888075 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.906543 22903 raft_consensus.cc:2410] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:08:36.907253 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 351097us
I20260709 13:08:36.888413 22905 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:08:36.911001 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:36.912447 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:36.900965 22829 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:36.922233 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:36.923166 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 125290us
I20260709 13:08:36.926690 22756 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
W20260709 13:08:36.930380 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.930718 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.935297 22898 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:08:36.936617 27325 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:36.940686 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:36.943579 22750 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:36.949659 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:36.950052 22829 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:36.950218 22823 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:36.933964 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:36.951184 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:36.951574 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.944756 23450 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:36.956239 23452 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:36.956570 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:36.958734 22900 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
W20260709 13:08:37.015775 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.911370 27287 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:36.961203 23450 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:08:36.912233 27306 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:37.025403 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:37.031076 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 81254.9us
I20260709 13:08:37.031381 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:37.026980 22750 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:37.026001 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:37.030107 22751 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:37.031961 22750 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:37.034045 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.034335 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:37.034986 22905 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:37.035105 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:36.950819 27345 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.035565 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.031909 22830 raft_consensus.cc:2376] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13. Re-sending same reply.
I20260709 13:08:37.056861 22824 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:37.033565 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:37.037490 27320 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:37.037665 27287 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.038725 22750 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:37.039110 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:37.061050 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.040706 22822 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:37.061710 22829 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:08:37.041297 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:37.043478 27357 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.350s	user 0.014s	sys 0.012s
W20260709 13:08:37.043910 27378 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:08:37.041301 27306 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:36.908219 27263 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
W20260709 13:08:37.050565 27361 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.345s	user 0.000s	sys 0.008s
I20260709 13:08:37.050087 27324 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:37.054551 27351 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:08:37.058239 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 103993us
I20260709 13:08:37.062551 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.086007 22829 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:37.063064 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.065112 22897 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:36.926839 27353 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.110s
I20260709 13:08:37.070530 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.071416 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:37.083480 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48437us
I20260709 13:08:37.026130 26984 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:37.036725 27296 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:37.090268 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:37.092574 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 180320us
I20260709 13:08:37.091239 22750 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:37.093834 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:37.094482 22824 raft_consensus.cc:2368] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 14.
I20260709 13:08:37.102319 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:37.104071 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:37.110140 27365 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:08:37.111721 27364 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:37.115160 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:37.116387 27360 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:37.118475 26813 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.118948 26813 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.120900 26813 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:37.121860 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.124186 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:37.129523 27381 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:08:37.091624 26984 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:37.141028 22829 raft_consensus.cc:2410] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:08:37.147320 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.148769 22829 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:37.102092 27174 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:08:37.162798 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:37.168150 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 71689.5us
W20260709 13:08:37.169255 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45135.4us
I20260709 13:08:37.169195 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.170699 22826 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:37.172199 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.174706 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.157706 27320 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:37.176298 27263 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:37.176621 27320 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:37.177143 27320 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:37.177487 27379 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:08:37.178965 22747 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:37.179297 22747 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:37.180152 22747 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.182255 27118 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:37.183581 27371 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:37.185684 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:37.186306 22756 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:37.187004 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:37.187525 22905 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:37.177202 22755 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:37.189615 22755 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:37.190685 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:37.191215 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.184443 27118 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:37.161464 27296 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:37.199376 22829 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:37.189780 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:37.200644 22830 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:37.069649 27345 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:37.195099 27347 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:37.204118 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:37.200700 22755 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.205125 22747 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:37.205878 22897 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:37.211481 22905 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
W20260709 13:08:37.213157 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 111938us
I20260709 13:08:37.218351 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:37.222574 22831 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:37.226269 22826 raft_consensus.cc:2410] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:08:37.228653 22755 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:37.230504 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 197009us
I20260709 13:08:37.214277 27371 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:37.206739 27347 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:37.236713 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:37.232118 27371 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:37.227078 27226 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.232897 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:37.247915 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:37.252996 22829 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:37.254220 22829 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:08:37.256104 22829 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:37.257117 22829 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:08:37.258615 27347 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.249243 27226 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.260066 27347 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:37.241052 27296 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:37.253454 27353 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:37.278362 26984 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:37.235716 27364 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:08:37.281937 27327 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:08:37.291450 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 43611.9us
I20260709 13:08:37.292155 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:37.293092 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48989.4us
W20260709 13:08:37.296717 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50680.4us
I20260709 13:08:37.301749 22825 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:08:37.300768 22829 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:08:37.304215 23416 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.645s (SENT)
I20260709 13:08:37.304790 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:37.309206 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.241616 27371 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:37.373567 22822 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:37.374212 22901 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:37.378343 22755 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:08:37.387647 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 84374.5us
W20260709 13:08:37.388300 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.391371 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:37.391930 23448 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:37.404284 23451 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:37.404897 22898 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:08:37.450161 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 59773.6us
W20260709 13:08:37.450667 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60166us
I20260709 13:08:37.481775 22825 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:37.254259 27250 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.132s
I20260709 13:08:37.485124 27345 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:37.487279 27368 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.783s	user 0.000s	sys 0.014s
I20260709 13:08:37.378916 27326 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:37.333015 27353 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:37.483112 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:37.507467 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:37.508920 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:37.519415 27382 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.346s	user 0.009s	sys 0.000s
W20260709 13:08:37.524573 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:37.527760 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.529304 27383 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:37.529851 27383 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:37.534684 27388 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:37.535544 27388 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:37.540557 27306 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:37.541265 27306 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:37.530954 27383 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.330333 27365 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.159s
I20260709 13:08:37.482996 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:37.549528 27226 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:37.576243 23448 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
W20260709 13:08:37.550573 27373 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:08:37.376897 27308 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.173s
I20260709 13:08:37.505769 27288 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:08:37.590324 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 99790.3us
I20260709 13:08:37.537267 27388 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:08:37.602305 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 298094us
W20260709 13:08:37.605934 23416 leader_election.cc:336] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.782s (SENT)
I20260709 13:08:37.606432 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:37.603113 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.505539 27275 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:08:37.615003 27287 raft_consensus.cc:1064] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
W20260709 13:08:37.615499 27287 raft_consensus.cc:1068] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Unable to promote non-voter c45e1805f049467697a7358fb566de2f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:37.614831 22653 catalog_manager.cc:5697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 8 to 9, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 9 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:37.617394 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:37.617082 22653 catalog_manager.cc:5697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 4 to 6, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 6 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:37.617970 22748 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:37.620450 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:37.620164 22653 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 8 to 9, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 9 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:37.612874 27263 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:08:37.621874 22830 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:37.622511 27263 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:37.623060 22653 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:37.637689 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.625448 27353 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:37.613157 27356 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:37.582998 27314 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:37.640264 22750 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.547s	user 0.006s	sys 0.003s
I20260709 13:08:37.405910 27386 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.222s
I20260709 13:08:37.642407 22653 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:37.646230 22653 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 11 to 12, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 12 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:37.622970 27263 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:37.644572 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.644727 22747 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:08:37.644259 22750 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:08:37.653786 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.612560 27383 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:37.652971 27263 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:37.664317 27388 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.666482 27388 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:37.668396 27388 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:37.643685 27356 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:37.669591 22719 leader_election.cc:400] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 14.
I20260709 13:08:37.670006 22719 leader_election.cc:403] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:37.671658 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 123848us
I20260709 13:08:37.674646 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:37.641072 27320 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.685585 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:37.687536 22747 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:37.653985 27174 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:37.688824 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:37.690580 22749 raft_consensus.cc:2410] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:08:37.703792 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 97884.2us
I20260709 13:08:37.705267 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:37.688341 27391 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 13:08:37.706344 23416 leader_election.cc:336] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 2.028s (SENT)
I20260709 13:08:37.648789 27314 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:37.686218 27296 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:37.685622 27360 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.664937 27326 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:08:37.723093 27359 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:08:37.746894 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 56426.7us
I20260709 13:08:37.770035 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:37.770867 22830 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:37.718067 27360 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.790560 22828 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:08:37.791446 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:37.826253 22652 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 334) took 12426 ms (client timeout 15000 ms). Trace:
W20260709 13:08:37.829838 27392 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:08:37.830521 27410 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:08:37.838573 22901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:37.839469 22901 raft_consensus.cc:606] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:37.839696 27410 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:37.834546 27360 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:37.723769 27356 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:37.841358 27356 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:37.841939 22751 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:37.846331 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:37.826709 22652 rpcz_store.cc:269] 0709 13:08:25.280992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.281261 (+   269us) service_pool.cc:224] Handling call
0709 13:08:37.702261 (+12421000us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:31.505227 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:31.718607 (+213380us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:31.718643 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:08:31.718659 (+    16us) tablet.cc:662] Decoding operations
0709 13:08:31.723366 (+  4707us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:31.723417 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:31.723443 (+    26us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:08:31.821613 (+ 98170us) tablet.cc:701] Row locks acquired
0709 13:08:31.821630 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:08:31.821766 (+   136us) write_op.cc:270] Start()
0709 13:08:31.821879 (+   113us) write_op.cc:276] Timestamp: P: 1783602511821721 usec, L: 0
0709 13:08:31.821897 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:08:31.822556 (+   659us) log.cc:844] Serialized 7074 byte log entry
0709 13:08:32.207943 (+385387us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb71cf10ac 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b
0709 13:08:34.074978 (+1867035us) op_driver.cc:464] REPLICATION: finished
0709 13:08:34.187855 (+112877us) write_op.cc:301] APPLY: starting
0709 13:08:34.187957 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:34.490357 (+302400us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:34.490414 (+    57us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:35.312367 (+821953us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:35.312901 (+   534us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:08:35.312965 (+    64us) write_op.cc:312] APPLY: finished
0709 13:08:35.365812 (+ 52847us) log.cc:844] Serialized 164 byte log entry
0709 13:08:35.552526 (+186714us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.658536 (+106010us) write_op.cc:454] Released schema lock
0709 13:08:35.713544 (+ 55008us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:35.753287 (+ 39743us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.753332 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9477,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":112613,"apply.run_cpu_time_us":70513,"apply.run_wall_time_us":1565375,"cfile_cache_hit":70,"cfile_cache_hit_bytes":73150,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6217,"cfile_init":2,"lbm_read_time_us":19674,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":11,"num_ops":18,"prepare.queue_time_us":727140,"prepare.run_cpu_time_us":103401,"prepare.run_wall_time_us":1573190,"raft.queue_time_us":508232,"raft.run_cpu_time_us":5983,"raft.run_wall_time_us":88772,"replication_time_us":2252938,"spinlock_wait_cycles":93673728,"thread_start_us":1965525,"threads_started":5,"wal-append.queue_time_us":1389054,"wal-append.run_cpu_time_us":35976,"wal-append.run_wall_time_us":2982695}]]}
I20260709 13:08:37.836985 27039 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:37.837975 27411 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:08:37.858476 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 349587us
I20260709 13:08:37.852792 27263 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.757995 27371 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:37.828997 27320 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:37.874531 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:37.875878 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:37.882375 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 130359us
I20260709 13:08:37.785199 27319 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 13:08:37.797361 27326 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:08:37.899706 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:37.902940 22828 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:08:37.899260 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 129968us
W20260709 13:08:37.904350 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:37.881021 27411 consensus_queue.cc:1243] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:37.905880 23448 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:37.896847 27320 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:37.922264 23447 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:37.880122 27386 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:08:37.923436 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:37.882141 27308 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:08:37.988502 27407 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:08:37.991065 27226 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:37.995328 27345 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:38.008065 27381 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:38.010442 27296 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 14.
I20260709 13:08:37.997382 27314 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:37.997560 27287 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 13:08:38.015290 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 122472us
I20260709 13:08:37.985838 22825 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:37.987265 22753 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:08:37.917981 27359 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:37.922251 27323 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:08:38.043668 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:38.025591 27398 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:38.023036 27345 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:38.046536 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 126494us
I20260709 13:08:38.027216 27381 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:38.039093 27314 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:38.071091 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.073228 27401 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:08:38.074878 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:38.076030 27347 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.815s	user 0.017s	sys 0.007s
I20260709 13:08:38.076257 27423 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:38.076762 27423 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.055573 27296 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.078395 27296 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:38.080132 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44516.2us
I20260709 13:08:38.080804 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.080932 27296 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.078758 27423 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.081816 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:38.049551 27398 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:38.069646 27314 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:38.094909 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.101040 22901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:38.101758 22901 raft_consensus.cc:606] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:38.069386 27320 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:08:38.105427 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 336671us
I20260709 13:08:38.114707 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:38.123581 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.072177 27371 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:38.125195 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:38.103091 27404 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:08:38.097406 27388 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:38.127868 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.127875 22827 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:38.129915 22825 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:38.135072 22825 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:38.130440 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:38.126006 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 51174.2us
I20260709 13:08:38.133251 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:38.136366 22822 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:38.137307 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:38.138856 23416 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.702s (SENT)
I20260709 13:08:38.139273 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.114447 27287 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:38.100589 27398 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:38.093806 27359 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:38.154662 22750 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:08:38.125589 27386 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:38.081633 27308 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 13:08:38.158113 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 76350.3us
I20260709 13:08:38.161689 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:38.155344 22825 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.177002 22825 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:38.177095 27288 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.177655 27288 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.180907 27324 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:38.181530 22822 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:38.181442 27324 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.182701 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:38.183468 27309 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.184005 27309 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:38.184648 27324 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.185603 27416 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:08:38.186105 27309 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:38.186949 27413 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:38.187809 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48973.6us
I20260709 13:08:38.186143 27263 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:38.188441 23416 leader_election.cc:336] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.799s (SENT)
I20260709 13:08:38.188727 27263 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.189131 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:38.188923 23416 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.189627 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:38.189895 22822 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.190244 22830 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:38.190603 27339 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:38.191221 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:38.191385 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:38.192066 22830 raft_consensus.cc:2410] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:08:38.192155 27308 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:38.192665 27308 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:38.192999 27308 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:38.189229 27263 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.196604 27347 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:08:38.199147 26984 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:38.205888 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44310.1us
I20260709 13:08:38.206595 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:38.207095 27263 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.215801 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:38.222730 22750 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:38.210335 22825 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:08:38.224658 22825 pending_rounds.cc:85] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 7
I20260709 13:08:38.225608 22825 pending_rounds.cc:107] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:08:38.226555 22825 raft_consensus.cc:2889] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:38.227622 22822 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:08:38.229704 22755 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:38.228864 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:38.127729 27408 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 13:08:38.254035 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 80137.7us
I20260709 13:08:38.160082 27386 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:38.159535 27287 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:38.185209 27324 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:08:38.257275 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.164042 27118 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:38.211349 27039 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:38.267159 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.131134 27345 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.252496 27250 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:38.252700 27351 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:38.275895 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:38.275959 22750 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:38.258441 27386 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:38.256079 27107 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:08:38.292755 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:38.299456 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.311283 22753 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:38.274238 27250 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:38.277127 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 83476us
I20260709 13:08:38.291651 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:38.271731 27416 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:38.291595 27288 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:38.303678 27370 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.273859 27324 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:38.321869 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 115366us
W20260709 13:08:38.330226 23416 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.588s (SENT)
I20260709 13:08:38.318617 27250 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:38.331002 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.271869 27309 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:08:38.275007 27351 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:38.344885 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.320497 27345 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:38.348692 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49463.1us
I20260709 13:08:38.324656 27415 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:08:38.342024 27385 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:08:38.342805 27416 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:38.324084 27308 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.354941 27351 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:38.351708 27441 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:38.384482 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54362.8us
I20260709 13:08:38.357414 27314 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:38.372434 27288 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:38.345777 27287 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.390162 27441 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.391985 27416 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:38.393672 27383 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:38.395273 27383 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:38.401489 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.401361 27413 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:38.401782 27423 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:38.402647 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.402771 27413 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:38.403486 27413 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:38.403844 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.403957 27413 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:38.404657 27296 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:38.405861 22900 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:38.343855 27370 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.412523 27433 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:38.413216 22749 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.420521 22749 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:08:38.423051 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.423868 27433 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:38.424326 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:38.426007 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 81190.5us
I20260709 13:08:38.429678 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.430409 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.430825 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.431478 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.431526 27296 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:38.432437 27296 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:38.433248 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.433939 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.434566 27452 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:38.399187 27324 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:38.435496 27447 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:38.435974 27383 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:38.437512 23451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:08:38.404860 27413 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.438890 27396 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:38.462229 23451 raft_consensus.cc:606] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.462327 27396 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:38.462643 23451 raft_consensus.cc:614] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:38.463263 27396 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.439411 27370 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.441200 27320 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:38.464534 27320 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:38.466400 27347 raft_consensus.cc:993] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:08:38.466871 27347 raft_consensus.cc:1081] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:08:38.428462 27433 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:38.443723 27411 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:38.444056 27308 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.446784 27386 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.407055 27415 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:08:38.460084 27250 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.448642 23444 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:08:38.480399 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 46545.2us
I20260709 13:08:38.469635 27433 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:38.484222 22719 leader_election.cc:400] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11.
I20260709 13:08:38.484771 22719 leader_election.cc:403] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:08:38.484447 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:38.489297 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.490774 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:38.470096 27421 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:38.486465 22900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:38.495992 22900 raft_consensus.cc:606] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.496490 22900 raft_consensus.cc:614] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:38.497076 23416 leader_election.cc:336] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.735s (SENT)
W20260709 13:08:38.505649 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.510002 22899 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:38.494993 27314 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:38.510627 22897 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:38.513639 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:38.514046 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:38.514303 22827 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:38.495438 27296 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11.
W20260709 13:08:38.501973 23416 leader_election.cc:336] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.792s (SENT)
I20260709 13:08:38.517266 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.517349 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:38.504833 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:38.553821 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:38.554450 22828 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:38.663276 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:38.663710 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:38.663831 22828 raft_consensus.cc:2376] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13. Re-sending same reply.
I20260709 13:08:38.664839 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:38.665789 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:38.666036 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:38.484063 27411 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.666321 22900 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:08:38.666879 27437 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:38.667984 27411 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:38.499332 27415 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:38.668356 27437 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:38.668720 27415 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:38.669611 27415 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.554759 22823 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:38.556571 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:38.674604 23447 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:38.556864 22898 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:08:38.501833 27421 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:38.561244 27287 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:38.675645 27287 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:38.675951 27421 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.676649 22902 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:38.677829 27287 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:38.560231 27398 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.679977 27398 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.562705 27250 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.512522 27452 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:08:38.565800 27433 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.567122 26984 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:08:38.567183 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 65230.3us
I20260709 13:08:38.681478 27250 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:38.681859 27250 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:38.681978 27296 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:38.682500 27296 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:38.682709 27250 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.683923 27250 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:38.684283 27296 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:38.575528 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:38.684942 22750 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:38.578313 27386 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.578369 27441 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.499639 27454 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:38.516010 27419 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:08:38.579528 27381 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.581477 27351 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:38.582620 27390 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.012s
I20260709 13:08:38.591709 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:38.592327 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:38.592605 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:38.594452 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:38.594794 27431 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.597644 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:38.606540 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.689553 22830 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:38.689702 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.689467 27431 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.690570 23448 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:38.691593 27431 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:38.692085 27431 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.613286 27450 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:38.692473 27431 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.693002 27450 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.693809 27339 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:38.694782 27351 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.695456 27381 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:38.695601 27450 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:38.613652 27410 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:38.616824 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:38.577188 27320 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.620424 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1235) took 1233 ms. Trace:
I20260709 13:08:38.635376 27396 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:38.640307 27360 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.648778 22754 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:08:38.650377 27323 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:38.516379 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:38.669902 22900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:38.674929 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:38.730901 22900 raft_consensus.cc:606] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:38.681056 22824 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
W20260709 13:08:38.555109 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:38.683146 23416 leader_election.cc:336] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.985s (SENT)
W20260709 13:08:38.731696 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 215372us
I20260709 13:08:38.731815 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:38.732578 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:38.574043 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:38.684966 22902 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:08:38.666252 22828 raft_consensus.cc:2376] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13. Re-sending same reply.
I20260709 13:08:38.576990 22752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:08:38.738008 22752 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:38.738452 22752 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:38.686533 27454 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:38.700042 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44889.7us
I20260709 13:08:38.700889 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:38.742187 22823 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:38.742120 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:38.701507 22830 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:38.703306 23451 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:08:38.706928 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:38.709947 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:38.709945 22903 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:38.685865 27398 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.711978 23446 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:08:38.710126 23444 rpcz_store.cc:276] 0709 13:08:37.387220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.389249 (+  2029us) service_pool.cc:224] Handling call
0709 13:08:37.489287 (+100038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:38.483700 (+994413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.620202 (+136502us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b5400555d54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:38.620367 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2205276928}
I20260709 13:08:38.719604 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:38.745342 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:38.749225 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:38.754487 23447 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:38.746402 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:38.755002 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:38.709604 27326 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:38.755811 22823 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:08:38.753888 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:38.756287 23448 raft_consensus.cc:2410] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:38.755882 22897 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:38.760288 23447 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:08:38.761924 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:38.762663 23447 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:38.763052 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:38.763758 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:38.764230 22749 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:38.764379 23446 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:38.765868 22826 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:38.766640 23449 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:08:38.702499 27410 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:38.796188 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:38.800698 22899 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:38.802795 22830 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:08:38.804001 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:38.804687 22830 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
W20260709 13:08:38.807245 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43694.3us
I20260709 13:08:38.696640 27413 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:08:38.808605 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:38.809077 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 125949us
I20260709 13:08:38.794476 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:38.795349 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:38.817248 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:38.817656 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:38.743328 27411 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.751545 27454 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:38.819645 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:38.763229 27355 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:38.753055 27296 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:38.825806 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:38.795794 27422 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:38.807937 27288 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.754771 27314 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:38.838999 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:38.813794 26984 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:38.892078 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 44594.5us
I20260709 13:08:38.751706 27360 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.796723 27433 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:08:38.797840 27326 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.763100 27398 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.946772 22903 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:38.949131 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:38.858009 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62741.7us
W20260709 13:08:38.954610 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 157701us
W20260709 13:08:38.959327 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:38.929658 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 62964.7us
I20260709 13:08:38.816960 27450 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:08:38.960449 22752 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:38.993404 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 44360.3us
I20260709 13:08:38.992614 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:38.822152 27107 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:08:38.892369 27039 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:39.004081 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:38.999804 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.970180 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:39.034938 26984 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:38.960788 27314 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:38.893059 27422 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:38.924253 27370 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:39.006819 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:39.045214 22905 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:08:39.070314 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 70664.4us
I20260709 13:08:39.089015 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:39.089322 22752 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:39.090430 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:39.036417 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 43875.8us
I20260709 13:08:39.094395 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:38.830564 27411 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:39.094905 23452 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:39.099382 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.945744 27353 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.105126 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:39.105993 22830 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 15: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:39.109416 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:38.960831 27359 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.142s
I20260709 13:08:39.007989 27351 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.050356 27308 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:39.038273 27039 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:39.069543 27355 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:39.120553 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.088241 27314 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:39.123107 23449 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:39.097946 27422 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:39.085119 27296 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:39.122880 27370 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:39.097481 27415 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:39.107211 27388 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:38.960561 27288 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:39.106871 27448 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:39.110971 27401 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:38.709205 27437 consensus_queue.cc:1175] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3b626c41600d4737a178ebab8a7ebfc2" responder_term: 17 status { last_received { term: 12 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: [14.12-16.13]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:39.068698 27433 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:08:39.119836 27383 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:39.096221 27452 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:39.123085 27431 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:39.168308 27456 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:39.236545 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.229540 27415 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:39.115500 27353 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:39.203140 27039 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:39.122292 27351 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:39.309721 27472 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 13:08:39.310472 27325 raft_consensus.cc:993] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:08:39.311290 27351 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:39.215556 27296 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:39.313248 27450 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.313771 27450 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:39.314234 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.314345 27371 raft_consensus.cc:1081] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:08:39.315645 27450 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:39.318320 27465 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:08:39.295786 27415 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:39.321808 27463 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.343s	user 0.003s	sys 0.000s
I20260709 13:08:39.322997 22828 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:39.323860 27424 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.085s	user 0.002s	sys 0.004s
W20260709 13:08:39.240039 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 263820us
W20260709 13:08:39.317116 27477 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
W20260709 13:08:39.307097 27462 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 13:08:39.306228 27469 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:08:39.202147 27355 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:39.359326 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 314375us
I20260709 13:08:39.361354 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:39.364950 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:39.309203 27320 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:39.337131 27174 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.374421 23446 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
W20260709 13:08:39.404387 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43822.2us
W20260709 13:08:39.368026 23416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.009s	sys 0.002s
W20260709 13:08:39.406404 23416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.009s	sys 0.007s
W20260709 13:08:39.406795 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 591690us
I20260709 13:08:39.374936 27263 raft_consensus.cc:1064] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:08:39.337520 27308 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:08:39.325892 27456 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:39.438571 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:39.460719 23451 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:39.399246 27272 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.437332 27351 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.400274 27327 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:39.479280 27456 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:39.366449 27324 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:08:39.505743 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:39.525108 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:39.432924 27360 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:39.509073 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45805.3us
I20260709 13:08:39.440371 27359 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.436549 27398 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:39.441164 27326 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:39.525285 22897 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:39.539006 22897 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:39.472163 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:39.530203 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:39.534042 22905 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: [14.8-15.9]
I20260709 13:08:39.402218 27174 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:39.516783 27327 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:39.528280 27444 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:39.614514 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:39.532950 27439 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:39.379328 27370 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:39.473294 27371 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:08:39.605455 22897 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:39.618153 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88050.6us
I20260709 13:08:39.528825 27353 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:39.539369 27429 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:08:39.610128 26813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.019s	sys 0.031s
I20260709 13:08:39.646111 22751 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:08:39.645213 22897 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:39.649508 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:39.638500 22753 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:39.509492 27272 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:39.652374 23452 consensus_meta.cc:287] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.001s
I20260709 13:08:39.652917 23452 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
W20260709 13:08:39.645658 27485 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.428s	user 0.001s	sys 0.000s
I20260709 13:08:39.614010 27327 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:39.525909 27351 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:39.711596 22755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:08:39.720398 22755 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:39.720839 22755 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:39.528532 27492 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.131s
W20260709 13:08:39.729015 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 273854us
I20260709 13:08:39.619855 27411 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:39.538789 27359 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:39.739580 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 95812us
W20260709 13:08:39.769785 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:39.800560 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69779.8us
I20260709 13:08:39.721776 27456 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:39.722224 27433 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:39.818280 23452 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:08:39.825857 22749 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:08:39.663396 27353 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.131s
I20260709 13:08:39.833407 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 829) took 1327 ms. Trace:
I20260709 13:08:39.627624 27410 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:39.836632 23453 rpcz_store.cc:276] 0709 13:08:38.505820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.506033 (+   213us) service_pool.cc:224] Handling call
0709 13:08:38.745113 (+239080us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:38.758213 (+ 13100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.758258 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:38.812173 (+ 53915us) log.cc:844] Serialized 26 byte log entry
0709 13:08:38.962634 (+150461us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:38.962683 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.962764 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:39.833177 (+870413us) raft_consensus.cc:1685] finished
0709 13:08:39.833206 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.833349 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12739,"raft.queue_time_us":19997,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":17722,"thread_start_us":68021,"threads_started":1,"wal-append.queue_time_us":254101}
I20260709 13:08:39.841917 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:39.846612 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 86689.7us
I20260709 13:08:39.802317 27508 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:39.810331 27487 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:39.878613 22755 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:39.845566 27355 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:39.814105 27324 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.060s
W20260709 13:08:39.881063 27475 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
W20260709 13:08:39.881659 27471 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 13:08:39.865252 27411 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:39.985217 22748 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:39.869297 27410 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:39.834163 27456 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:39.884390 27508 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:39.844130 27433 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:39.908767 27326 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:39.883175 27353 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.107s
W20260709 13:08:40.072727 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 199436us
I20260709 13:08:40.074537 22755 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.075304 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:39.911571 27339 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.168s
I20260709 13:08:39.891870 27487 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:39.988950 27431 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.101s
W20260709 13:08:40.112417 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.113183 27359 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.113854 27359 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:40.114284 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.114907 22898 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:08:40.115056 27480 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:40.115175 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.115545 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.115763 22902 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:40.116222 27288 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:40.116321 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
W20260709 13:08:40.116618 27490 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.458s	user 0.000s	sys 0.001s
I20260709 13:08:40.116914 22903 raft_consensus.cc:606] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:40.117159 27478 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.536s	user 0.001s	sys 0.000s
I20260709 13:08:40.117288 22903 raft_consensus.cc:614] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:40.121793 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:40.122277 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:40.081342 27326 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:40.129024 27039 consensus_meta.cc:287] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.828s	user 0.010s	sys 0.000s
I20260709 13:08:40.129415 27039 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.129983 23444 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:40.130561 27326 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.131070 27326 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.131394 27039 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.131494 27326 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:40.132401 27483 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.476s	user 0.001s	sys 0.000s
I20260709 13:08:40.133473 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:40.134088 27287 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.134547 27287 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.135215 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:40.136420 27287 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.136911 27287 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:40.143025 27491 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.253s	user 0.000s	sys 0.000s
W20260709 13:08:40.144111 27484 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.252s	user 0.000s	sys 0.000s
I20260709 13:08:40.145188 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1244) took 1447 ms. Trace:
I20260709 13:08:40.146323 27437 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:40.147333 27511 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:08:40.146957 27437 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.149025 27326 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:40.149524 27494 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:40.150095 26984 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.151708 27456 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.153954 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 830) took 1061 ms. Trace:
I20260709 13:08:40.154323 23447 rpcz_store.cc:276] 0709 13:08:39.092836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.092980 (+   144us) service_pool.cc:224] Handling call
0709 13:08:39.093266 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:39.462461 (+369195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.462916 (+   455us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:39.464177 (+  1261us) log.cc:844] Serialized 26 byte log entry
0709 13:08:39.812420 (+348243us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:39.812480 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.812600 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:40.153760 (+341160us) raft_consensus.cc:1685] finished
0709 13:08:40.153782 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.153899 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":204312,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":17326,"thread_start_us":620111,"threads_started":2,"wal-append.queue_time_us":603248}
I20260709 13:08:40.156256 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:40.095386 27470 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.567s	user 0.001s	sys 0.000s
I20260709 13:08:40.156383 27272 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.122493 23453 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:40.145550 23445 rpcz_store.cc:276] 0709 13:08:38.697452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.697715 (+   263us) service_pool.cc:224] Handling call
0709 13:08:38.698077 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:38.712580 (+ 14503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.712622 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:38.755437 (+ 42815us) log.cc:844] Serialized 26 byte log entry
0709 13:08:38.999918 (+244481us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:38.999957 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.000016 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:40.144990 (+1144974us) raft_consensus.cc:1685] finished
0709 13:08:40.145009 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.145133 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46391,"raft.queue_time_us":14666,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":20017,"thread_start_us":162821,"threads_started":1,"wal-append.queue_time_us":950142}
W20260709 13:08:40.109761 27415 consensus_meta.cc:287] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.783s	user 0.010s	sys 0.001s
W20260709 13:08:40.171375 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.147006 27327 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:40.157747 27296 consensus_meta.cc:287] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.005s
I20260709 13:08:40.190825 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2325) took 1519 ms. Trace:
I20260709 13:08:40.197212 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:40.119040 27174 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:40.191226 22904 rpcz_store.cc:276] 0709 13:08:38.671314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.671433 (+   119us) service_pool.cc:224] Handling call
0709 13:08:38.671744 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:38.676409 (+  4665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.676441 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:38.676859 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:08:38.925741 (+248882us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:38.925783 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.925855 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:40.190628 (+1264773us) raft_consensus.cc:1685] finished
0709 13:08:40.190648 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.190770 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4274,"raft.queue_time_us":5576,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":248489,"threads_started":1,"wal-append.queue_time_us":486989}
I20260709 13:08:40.205056 27519 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:40.169301 27360 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:40.166954 27456 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.227890 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:40.237447 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.242633 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:40.243261 23453 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:40.246727 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:40.247366 23444 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:40.202229 27437 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.114284 27359 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.169478 27287 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.188705 27415 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:08:40.290668 27250 consensus_meta.cc:287] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.606s	user 0.012s	sys 0.001s
I20260709 13:08:40.246271 27456 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.291410 27524 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.292265 27496 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.200026 27296 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.132016 27399 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.541s
I20260709 13:08:40.150029 27494 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.228572 27508 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.115628 27480 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:40.205605 27519 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:08:40.374873 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 132668us
I20260709 13:08:40.293552 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:40.374229 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:40.277463 22897 consensus_meta.cc:287] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.629s	user 0.007s	sys 0.003s
I20260709 13:08:40.384235 22897 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:08:40.393914 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.396046 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.396780 22753 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:40.293426 27510 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:40.359179 22930 heartbeater.cc:646] Failed to heartbeat to 127.21.31.126:35001 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.31.126:35001 timed out after 15.000s (SENT)
I20260709 13:08:40.409641 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:40.372810 27520 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:08:40.372192 27526 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:08:40.294057 27461 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.470s	user 0.000s	sys 0.001s
W20260709 13:08:40.413254 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 185446us
W20260709 13:08:40.372220 27381 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.676s	user 0.012s	sys 0.000s
I20260709 13:08:40.370118 27413 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:40.415374 22829 raft_consensus.cc:2410] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:08:40.271478 27505 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.433043 27505 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.419345 27510 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:40.435313 27505 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:40.293900 27509 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.436087 27509 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:40.436650 27514 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:40.437212 27514 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:40.437583 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2320) took 1924 ms. Trace:
W20260709 13:08:40.438701 27467 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.446s	user 0.001s	sys 0.001s
I20260709 13:08:40.438182 27514 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.437970 22896 rpcz_store.cc:276] 0709 13:08:38.513623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.513767 (+   144us) service_pool.cc:224] Handling call
0709 13:08:38.514148 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:38.514478 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.514506 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:38.517816 (+  3310us) log.cc:844] Serialized 26 byte log entry
0709 13:08:38.925738 (+407922us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:38.925781 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.925854 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:40.437381 (+1511527us) raft_consensus.cc:1685] finished
0709 13:08:40.437403 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.437522 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1407,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":32402,"spinlock_wait_cycles":34176,"thread_start_us":344466,"threads_started":1,"wal-append.queue_time_us":911510}
W20260709 13:08:40.426069 27454 consensus_meta.cc:287] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 1.534s	user 0.006s	sys 0.006s
I20260709 13:08:40.440148 27454 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.441040 27509 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:40.441267 27519 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.441906 27519 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:40.442230 27454 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.442324 27519 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.443921 27360 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:40.444721 27519 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.444589 27360 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:40.445611 22827 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:08:40.446692 27494 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.419751 22752 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:40.424978 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:40.353798 27476 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.625s	user 0.000s	sys 0.002s
I20260709 13:08:40.321059 27327 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.427557 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:40.392063 27351 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:40.417070 27422 consensus_meta.cc:287] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.210s	user 0.008s	sys 0.004s
I20260709 13:08:40.463410 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1304) took 1871 ms. Trace:
I20260709 13:08:40.470122 22747 rpcz_store.cc:276] 0709 13:08:38.592120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.592710 (+   590us) service_pool.cc:224] Handling call
0709 13:08:38.592985 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:38.593280 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.593305 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:38.598808 (+  5503us) log.cc:844] Serialized 26 byte log entry
0709 13:08:38.723985 (+125177us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:38.724026 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.724093 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:40.438215 (+1714122us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:40.447591 (+  9376us) raft_consensus.cc:1685] finished
0709 13:08:40.447618 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.463336 (+ 15718us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8995,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":65,"spinlock_wait_cycles":163830144,"thread_start_us":124750,"threads_started":1,"wal-append.queue_time_us":212606}
W20260709 13:08:40.475189 22751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.018s	sys 0.024s
W20260709 13:08:40.477209 22751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.018s	sys 0.024s
W20260709 13:08:40.477572 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 402304us
W20260709 13:08:40.481258 22871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.019s	sys 0.024s
W20260709 13:08:40.482103 22871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.019s	sys 0.025s
I20260709 13:08:40.435599 27510 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.448117 27039 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:40.447193 27360 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:40.398866 27524 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:40.510726 22897 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:08:40.465528 27519 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:40.474215 27314 consensus_meta.cc:287] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.265s	user 0.011s	sys 0.000s
I20260709 13:08:40.429628 27398 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.509303 22896 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:08:40.472046 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.403777 27496 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:40.512705 22721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.022s	sys 0.031s
I20260709 13:08:40.430202 27174 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:40.469519 27494 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:40.523598 22754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.024s	sys 0.029s
W20260709 13:08:40.523761 22721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.022s	sys 0.031s
I20260709 13:08:40.462021 27413 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:40.522956 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:40.475224 27506 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:40.533355 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1859) took 2016 ms. Trace:
I20260709 13:08:40.504861 27288 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
W20260709 13:08:40.531837 22754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.024s	sys 0.029s
I20260709 13:08:40.533749 22827 rpcz_store.cc:276] 0709 13:08:38.516720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.516960 (+   240us) service_pool.cc:224] Handling call
0709 13:08:38.517255 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:40.445987 (+1928732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.533115 (+ 87128us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54002c9054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:40.533288 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4434174720}
I20260709 13:08:40.545572 23453 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:40.562249 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.536294 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.562901 22747 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:40.569764 23444 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:40.475705 27422 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:40.563499 27504 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 13:08:40.581445 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.581823 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.560415 27323 raft_consensus.cc:993] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:08:40.538273 27487 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.520681 27039 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.657851 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.533550 27506 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:40.521281 27399 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.653643 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.532510 27398 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.651770 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.690337 22754 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:40.658026 22896 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:40.671358 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.522954 27519 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:40.678668 22903 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:40.684296 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.523150 27314 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.692698 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:40.652453 27403 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:40.693982 22904 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
W20260709 13:08:40.696324 22797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.016s	sys 0.078s
W20260709 13:08:40.697793 22797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.016s	sys 0.078s
I20260709 13:08:40.655505 26813 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:40.698427 22749 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:40.687598 27530 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:40.698880 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.538798 27494 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.704340 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.711110 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:40.712654 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:40.713250 22904 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:40.714182 22896 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:40.718286 22754 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
W20260709 13:08:40.720037 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 209762us
W20260709 13:08:40.700662 27325 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.035s	sys 0.053s
I20260709 13:08:40.696468 27505 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:40.686411 27506 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:40.729378 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:40.729441 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:40.731769 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:40.732700 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:40.706444 27351 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.811313 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.730408 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:40.793610 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1252) took 1819 ms. Trace:
I20260709 13:08:40.795380 22828 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:08:40.571537 27263 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.707239 26813 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.803673 22826 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:40.722759 27324 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:08:40.805100 22827 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:08:40.801795 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:40.721844 27287 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.723109 27359 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.721060 27456 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.710850 27509 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:40.725919 27325 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.035s	sys 0.054s
I20260709 13:08:40.799429 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:40.733628 27381 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.733405 27454 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:40.719497 27250 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:40.793462 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 80879.4us
I20260709 13:08:40.734105 27535 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:40.812847 22824 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:40.804713 27530 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:40.706854 27403 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.803282 22831 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:40.822827 22749 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:40.812986 23448 rpcz_store.cc:276] 0709 13:08:38.974191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.974316 (+   125us) service_pool.cc:224] Handling call
0709 13:08:38.974567 (+   251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:39.080536 (+105969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.080578 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:39.097764 (+ 17186us) log.cc:844] Serialized 26 byte log entry
0709 13:08:39.433469 (+335705us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:39.433509 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.433584 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:40.793442 (+1359858us) raft_consensus.cc:1685] finished
0709 13:08:40.793459 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.793562 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25915,"raft.queue_time_us":62354,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":18900,"spinlock_wait_cycles":25472,"thread_start_us":264463,"threads_started":1,"wal-append.queue_time_us":977011}
I20260709 13:08:40.798081 27327 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:40.719830 27386 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:40.810457 27396 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 2.064s	user 0.012s	sys 0.003s
W20260709 13:08:40.829664 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 95618.7us
I20260709 13:08:40.832442 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:40.833928 26813 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.836853 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:40.792510 27226 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:08:40.698817 27399 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.805935 27437 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:40.796406 27481 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.319s
I20260709 13:08:40.881704 23444 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:40.922006 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.922452 22897 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:40.925135 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.925397 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.925757 22903 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:40.925213 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.928344 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:40.827104 27528 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.892885 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:40.945981 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 53168.1us
I20260709 13:08:40.946609 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:40.947602 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:40.900034 22824 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:40.832474 27537 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.845876 27456 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.905720 27327 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:40.906445 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:40.886821 27532 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:08:40.878057 27381 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.964931 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:40.965725 22904 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:08:40.929921 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.931147 22831 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:08:40.966089 22904 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:40.901288 27507 raft_consensus.cc:1081] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
W20260709 13:08:40.943699 27410 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.947s	user 0.012s	sys 0.000s
I20260709 13:08:40.954394 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:40.956996 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:40.917236 27502 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:08:40.823812 27351 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:40.959188 22824 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:08:40.839397 27425 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:40.905408 27524 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.968525 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:40.986327 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:40.943795 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:40.883288 27509 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.892279 27454 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.981547 23444 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:40.942863 27413 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:40.942592 27320 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.897773 27250 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.985020 22755 raft_consensus.cc:2410] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:08:40.969713 27422 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.959854 26813 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:40.897677 27530 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.967425 27272 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:40.967067 27325 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:40.999498 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:40.951614 27396 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:40.967799 27327 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:40.990017 22904 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.003608 23444 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:40.915524 27386 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:40.987262 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43540.6us
I20260709 13:08:40.969475 27519 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.011837 22827 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:40.944242 27437 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:40.984119 27527 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:41.015098 27039 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:41.016314 27487 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:41.021379 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 106031us
I20260709 13:08:41.022388 22904 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:40.986773 27425 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:41.065946 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:41.066973 27339 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:41.067365 27339 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.026487 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:41.018405 27325 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.027819 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:41.047238 23445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.013s	sys 0.021s
I20260709 13:08:41.016031 27548 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:41.072340 23445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.013s	sys 0.022s
I20260709 13:08:41.029119 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:41.009191 27314 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:40.972862 27456 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.051729 22822 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:41.030869 23444 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:40.901633 27535 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.016860 26813 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.069048 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.026660 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.023861 27309 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:41.067776 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:41.009999 27499 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:08:41.073251 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44211.6us
I20260709 13:08:41.071362 27339 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:40.970021 27537 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.074594 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:41.075651 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.052027 27410 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:08:41.079926 23444 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:41.068503 27533 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 13:08:41.113382 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:41.050964 27494 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.016547 27320 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.029281 27524 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:08:40.969292 27528 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.070905 27425 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:41.073300 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:41.090420 27544 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:08:41.072510 27548 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.091025 27530 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.051265 27413 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:08:41.089051 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:41.077991 27543 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:08:41.072824 27325 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:41.075795 22824 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:41.050930 27396 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.103637 22825 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:08:41.105985 22822 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:08:41.106630 27488 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.614s	user 0.003s	sys 0.000s
I20260709 13:08:41.106966 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.075315 27546 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:41.107406 27535 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.079901 27519 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:41.116341 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.067984 27527 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:41.129439 27538 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:08:41.131654 22831 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:41.071842 27487 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.101781 27508 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:41.136135 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:41.114591 27499 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:41.089398 26813 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:41.166153 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.357869 22904 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:41.312753 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1259) took 1124 ms. Trace:
I20260709 13:08:41.362031 23445 rpcz_store.cc:276] 0709 13:08:40.188585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.194899 (+  6314us) service_pool.cc:224] Handling call
0709 13:08:40.195249 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:40.374193 (+178944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.374226 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:40.374699 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:08:41.073954 (+699255us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:41.110387 (+ 36433us) log.cc:844] Serialized 14 byte log entry
0709 13:08:41.113279 (+  2892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.113398 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.312516 (+199118us) raft_consensus.cc:1685] finished
0709 13:08:41.312535 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.312689 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39768,"prepare.queue_time_us":40904,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":172304,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"spinlock_wait_cycles":10459520,"thread_start_us":698909,"threads_started":1,"wal-append.queue_time_us":792012}
I20260709 13:08:41.365491 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.366122 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:41.366936 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 277921us
I20260709 13:08:41.367784 27546 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.169544 22903 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:41.226070 27529 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:41.373095 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.261451 27536 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:41.382383 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.259037 27403 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.256915 27408 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:41.260510 27509 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.305487 27381 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.358924 27508 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:41.387640 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.258798 27386 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.388736 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.219717 27410 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:41.366760 27519 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:41.369804 27351 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.288205 27399 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.373899 27454 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.304445 27552 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:41.397953 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:41.400111 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.381953 27514 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:41.288434 27516 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:41.380333 27226 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:08:41.397545 27408 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.403756 27396 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:41.404464 27550 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:08:41.400622 27508 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:41.401671 27519 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:41.406518 22830 raft_consensus.cc:2410] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:08:41.412951 22824 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:41.388432 27529 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.401098 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.395692 27536 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.419077 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:41.400110 27410 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.432236 27536 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:41.426652 27399 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:41.430559 27529 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:41.445092 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:41.450764 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.463459 27408 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:41.466946 22830 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:41.421506 27514 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.467505 22754 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:41.469539 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:41.475957 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:41.443899 27537 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.476231 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1097) took 2729 ms. Trace:
I20260709 13:08:41.481209 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1257) took 1353 ms. Trace:
I20260709 13:08:41.418378 27552 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.441828 27425 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.479593 22899 rpcz_store.cc:276] 0709 13:08:38.739707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.818147 (+ 78440us) service_pool.cc:224] Handling call
0709 13:08:38.835236 (+ 17089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.073404 (+238168us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:39.122276 (+ 48872us) log.cc:844] Serialized 14 byte log entry
0709 13:08:39.379252 (+256976us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:39.379342 (+    90us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:39.379476 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.452986 (+2073510us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54001b6f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:41.468717 (+ 15731us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.468831 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46728,"prepare.queue_time_us":48722,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":57872,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":13191,"spinlock_wait_cycles":547070336,"thread_start_us":249731,"threads_started":1,"wal-append.queue_time_us":1396938}
I20260709 13:08:41.468750 27423 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:41.425359 27516 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:41.482443 27567 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:08:41.474617 27226 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:41.495952 22830 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:41.483968 27107 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:41.499596 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.500556 22830 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:41.478777 27573 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:41.474444 27399 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:41.490420 27498 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:08:41.488945 27423 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:41.481614 23449 rpcz_store.cc:276] 0709 13:08:40.127809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.127992 (+   183us) service_pool.cc:224] Handling call
0709 13:08:40.133128 (+  5136us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:40.192766 (+ 59638us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:40.569307 (+376541us) log.cc:844] Serialized 14 byte log entry
0709 13:08:41.070911 (+501604us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:41.071354 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 13:08:41.071493 (+   139us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:41.071526 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.071588 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.481018 (+409430us) raft_consensus.cc:1685] finished
0709 13:08:41.481038 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.481154 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58321,"prepare.queue_time_us":529772,"prepare.run_cpu_time_us":8896,"prepare.run_wall_time_us":77276,"raft.queue_time_us":92363,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":9314,"spinlock_wait_cycles":13848960,"thread_start_us":754685,"threads_started":3,"wal-append.queue_time_us":323555}
I20260709 13:08:41.508375 27423 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.500195 27325 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.510814 27325 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:41.511552 27563 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:08:41.514410 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.498823 27226 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:41.522435 27571 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:08:41.524247 27552 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.525736 27514 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.526856 27410 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.424690 27499 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:41.531666 27542 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.649s	user 0.000s	sys 0.001s
I20260709 13:08:41.532986 27353 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:41.533445 27570 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:08:41.533739 27353 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.533740 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1258) took 1371 ms. Trace:
I20260709 13:08:41.503675 27527 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:41.538129 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:41.547420 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.544111 23447 rpcz_store.cc:276] 0709 13:08:40.162733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.162872 (+   139us) service_pool.cc:224] Handling call
0709 13:08:40.163233 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:40.227009 (+ 63776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.227038 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:40.241608 (+ 14570us) log.cc:844] Serialized 26 byte log entry
0709 13:08:40.681523 (+439915us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:40.681570 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.681636 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.531061 (+849425us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:41.533420 (+  2359us) raft_consensus.cc:1685] finished
0709 13:08:41.533448 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.533675 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214511,"raft.queue_time_us":63974,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":20970,"spinlock_wait_cycles":31087872,"thread_start_us":362899,"threads_started":1,"wal-append.queue_time_us":528577}
I20260709 13:08:41.503847 27573 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.517822 27423 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:41.553005 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.553172 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:41.553997 22830 raft_consensus.cc:2410] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:08:41.556581 27296 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.544904 27353 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:41.557847 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:41.546096 27226 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:41.560088 27495 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:41.561527 27528 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.521145 27546 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.568392 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:41.568913 23450 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:41.547874 27552 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.542546 27425 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.566147 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.558190 27107 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:41.567871 27506 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.765s	user 0.010s	sys 0.000s
I20260709 13:08:41.538583 27503 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:41.569773 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.586421 23446 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:41.570436 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.587527 23452 raft_consensus.cc:2410] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:08:41.571063 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.571997 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.551383 27514 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.573899 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.575083 27537 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.591100 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 2470) took 1315 ms. Trace:
I20260709 13:08:41.556834 27327 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.575771 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.576289 27575 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:41.577857 22794 connection.cc:256] RPC call timeout handler was delayed by -1.5868699349999815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260709 13:08:41.594722 22794 connection.cc:256] RPC call timeout handler was delayed by -1.427938518000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
W20260709 13:08:41.595108 22794 connection.cc:256] RPC call timeout handler was delayed by -1.3874385180000672s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:08:41.595403 22794 connection.cc:256] RPC call timeout handler was delayed by -1.3874385180000672s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:08:41.595688 22794 connection.cc:256] RPC call timeout handler was delayed by -1.1980699349999213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
I20260709 13:08:41.550772 27410 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:41.577746 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.550058 27573 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:41.597765 23445 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:08:41.597595 27410 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.599695 27410 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:41.600199 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.577728 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.576638 27548 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:41.582332 27482 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 2.111s	user 0.002s	sys 0.002s
I20260709 13:08:41.583140 27309 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:41.584390 27398 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.578894 27396 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:41.605613 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1314) took 2883 ms. Trace:
I20260709 13:08:41.605504 27396 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:41.605988 22750 rpcz_store.cc:276] 0709 13:08:38.722017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.722385 (+   368us) service_pool.cc:224] Handling call
0709 13:08:38.722638 (+   253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:38.723215 (+   577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.723241 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:38.723754 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 13:08:39.096448 (+372694us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:39.096489 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.096554 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.581755 (+2485201us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:41.605302 (+ 23547us) raft_consensus.cc:1685] finished
0709 13:08:41.605352 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.605541 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":20148,"spinlock_wait_cycles":45738752,"thread_start_us":283538,"threads_started":1,"wal-append.queue_time_us":605818}
I20260709 13:08:41.607566 27396 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.585229 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.591477 22755 rpcz_store.cc:276] 0709 13:08:40.271963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.409401 (+137438us) service_pool.cc:224] Handling call
0709 13:08:41.586851 (+1177450us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54003f11d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:41.586955 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320780032}
I20260709 13:08:41.607869 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.608466 22750 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 6.
I20260709 13:08:41.608669 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.575210 27480 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.592480 27514 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.592963 23450 raft_consensus.cc:2410] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:08:41.600061 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:41.579493 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.589485 27506 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.577225 27529 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.605023 27309 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:41.602229 27575 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.586083 27503 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:41.632994 27503 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.633989 27503 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:41.609493 22755 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:08:41.638445 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2957) took 1053 ms. Trace:
I20260709 13:08:41.638751 22755 rpcz_store.cc:276] 0709 13:08:40.584808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.637768 (+1052960us) service_pool.cc:224] Handling call
0709 13:08:41.638012 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:41.638151 (+   139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:41.638174 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:41.638220 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:41.638249 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.638293 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.638373 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:41.612943 22896 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:41.579473 27552 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.641302 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.652204 27552 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.652581 22826 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:41.652865 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2941) took 2039 ms. Trace:
I20260709 13:08:41.619485 27250 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.612023 27514 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.613817 23447 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:41.655052 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.615231 27506 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:41.656050 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.656352 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2945) took 2009 ms. Trace:
I20260709 13:08:41.615365 27529 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.644727 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.656782 22751 rpcz_store.cc:276] 0709 13:08:39.647521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.647772 (+   251us) service_pool.cc:224] Handling call
0709 13:08:39.719676 (+ 71904us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:39.827669 (+107993us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.827712 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:39.841043 (+ 13331us) log.cc:844] Serialized 26 byte log entry
0709 13:08:40.477576 (+636533us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:40.519177 (+ 41601us) log.cc:844] Serialized 14 byte log entry
0709 13:08:40.523165 (+  3988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.656021 (+1132856us) spinlock_profiling.cc:243] Waited 99.7 ms on lock 0x7b5400077bd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:41.656070 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.656115 (+    45us) raft_consensus.cc:1685] finished
0709 13:08:41.656186 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.656302 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18335,"prepare.queue_time_us":41418,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":357673,"raft.run_cpu_time_us":5710,"raft.run_wall_time_us":88893,"spinlock_wait_cycles":229266176,"thread_start_us":585394,"threads_started":1,"wal-append.queue_time_us":837397}
I20260709 13:08:41.657994 27508 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.658346 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.658951 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.657125 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.659979 23419 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:41.614262 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.643841 27487 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.660953 22825 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:41.662230 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.663419 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:41.663545 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.664354 22822 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:08:41.664698 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.616022 27396 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:41.665995 27396 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:41.594717 27537 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.625934 27519 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.628983 27454 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.630945 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.672228 22828 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:41.660043 27508 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.660672 27573 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.653188 22752 rpcz_store.cc:276] 0709 13:08:39.613727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.613858 (+   131us) service_pool.cc:224] Handling call
0709 13:08:39.628772 (+ 14914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.466759 (+837987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.652659 (+1185900us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b540016a154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:41.652816 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1818642176}
I20260709 13:08:41.625123 27575 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.666999 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.665055 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.616573 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:41.670825 27510 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.981s	user 0.008s	sys 0.009s
I20260709 13:08:41.670852 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:41.580037 27339 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.675787 22824 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:41.705693 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:41.683744 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.707160 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:41.708922 22824 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:08:41.660392 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.696444 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.710197 22825 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:08:41.623375 22830 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:41.700750 22828 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:41.698757 22822 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:41.701306 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.703050 23451 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 10 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:08:41.703792 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 87280.1us
I20260709 13:08:41.711761 22826 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:41.647595 27399 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:41.680344 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:41.714578 23444 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 15: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:41.715070 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:41.695890 27546 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:41.742983 27546 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:41.734880 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:41.701998 27510 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.008s	sys 0.010s
I20260709 13:08:41.697573 27250 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:41.733711 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:41.750003 23449 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:41.743132 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:41.748198 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.747318 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:41.697960 27454 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.752331 27386 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:41.752637 22826 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:41.752719 27386 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.753131 27386 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:41.754024 27454 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.609266 27536 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.713809 27575 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:41.767807 22829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 843) took 3152 ms (client timeout 2519 ms). Trace:
I20260709 13:08:41.768051 27339 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.769291 22823 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:08:41.776149 22823 pending_rounds.cc:85] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 12
I20260709 13:08:41.776403 22823 pending_rounds.cc:107] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:08:41.776666 22823 raft_consensus.cc:2889] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:41.666997 27396 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.771924 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2466) took 1541 ms. Trace:
I20260709 13:08:41.746156 27528 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:41.777266 22756 rpcz_store.cc:276] 0709 13:08:40.231057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.231181 (+   124us) service_pool.cc:224] Handling call
0709 13:08:40.231490 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.420210 (+188720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.420257 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.420297 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:40.420329 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.771579 (+1351250us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540011b7d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:41.771642 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.771848 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433566464}
W20260709 13:08:41.775432 22829 rpcz_store.cc:269] 0709 13:08:38.616232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.616525 (+   293us) service_pool.cc:224] Handling call
0709 13:08:41.767594 (+3151069us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54000adcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:41.767724 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3937356032}
I20260709 13:08:41.749126 27546 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:41.758853 27351 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:41.785410 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:41.816026 22904 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:41.817595 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:41.818361 22752 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:08:41.741879 27370 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:08:41.821950 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:41.824018 22756 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:41.784369 27324 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:41.784911 27528 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:41.807077 27351 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.801035 27512 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:41.765202 27250 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.801076 27287 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:08:41.786146 26813 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.831298 27579 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:41.848676 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.848912 22752 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
W20260709 13:08:41.831166 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 79199.7us
I20260709 13:08:41.838779 27528 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:41.861557 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:41.840382 27351 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:41.839696 27487 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:41.862475 27573 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:41.873402 27536 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:41.874154 27519 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.863183 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:41.880908 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:41.837693 27309 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.886994 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.887280 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:41.892452 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.893301 22755 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:08:41.894582 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2962) took 1192 ms. Trace:
I20260709 13:08:41.894917 22755 rpcz_store.cc:276] 0709 13:08:40.702580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.893856 (+1191276us) service_pool.cc:224] Handling call
0709 13:08:41.894114 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:41.894278 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:41.894306 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:41.894349 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:41.894385 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.894439 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.894530 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:41.898478 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 853) took 3262 ms. Trace:
I20260709 13:08:41.898962 22823 rpcz_store.cc:276] 0709 13:08:38.636300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.756596 (+120296us) service_pool.cc:224] Handling call
0709 13:08:41.768938 (+3012342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:41.783625 (+ 14687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.898216 (+114591us) spinlock_profiling.cc:243] Waited 3.01 s on lock 0x7b54000adcd0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:41.898379 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6927018752}
I20260709 13:08:41.900574 22823 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:08:41.901232 27537 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.904016 26813 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.907318 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2948) took 2182 ms. Trace:
I20260709 13:08:41.869601 27579 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.907770 22748 rpcz_store.cc:276] 0709 13:08:39.725398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.727072 (+  1674us) service_pool.cc:224] Handling call
0709 13:08:39.763840 (+ 36768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.996807 (+232967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.907033 (+1910226us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b5400133654. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:41.907249 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498922752}
I20260709 13:08:41.909772 27579 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.911000 27433 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.844218 27512 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.914882 27512 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:41.883126 27552 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:08:41.919595 27551 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.857s	user 0.002s	sys 0.000s
I20260709 13:08:41.923769 27386 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:41.927752 27528 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:41.877866 27327 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.935166 27528 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.843751 27226 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:41.994201 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 132730us
I20260709 13:08:41.982527 22897 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:08:41.942915 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:41.999630 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:41.987188 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:41.999668 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.988253 22904 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:08:41.986150 22905 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:08:42.046113 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.016165 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1868) took 1033 ms. Trace:
W20260709 13:08:42.046808 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.957329 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2950) took 1420 ms. Trace:
I20260709 13:08:41.984259 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.047119 22831 rpcz_store.cc:276] 0709 13:08:40.982434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.982572 (+   138us) service_pool.cc:224] Handling call
0709 13:08:40.982899 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:41.358568 (+375669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.015909 (+657341us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002c9554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:42.016097 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341236096}
I20260709 13:08:42.037792 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1752) took 1011 ms. Trace:
I20260709 13:08:42.008793 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1115) took 1293 ms. Trace:
I20260709 13:08:42.047066 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.997946 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2700) took 1163 ms. Trace:
I20260709 13:08:42.020074 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.027204 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.049033 22896 rpcz_store.cc:276] 0709 13:08:40.715812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.717634 (+  1822us) service_pool.cc:224] Handling call
0709 13:08:40.717973 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:41.650777 (+932804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.008528 (+357751us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b540031f954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:42.008730 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2057761024}
I20260709 13:08:42.050203 22751 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:42.050315 22827 rpcz_store.cc:276] 0709 13:08:40.834521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.836557 (+  2036us) service_pool.cc:224] Handling call
0709 13:08:41.997733 (+1161176us) spinlock_profiling.cc:243] Waited 98.1 ms on lock 0x7b54001fa9d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:41.997855 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225695488}
I20260709 13:08:42.050904 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.051517 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.051983 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.044843 22899 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 5.
I20260709 13:08:41.896749 22755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:08:42.054324 22753 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:42.048691 22903 rpcz_store.cc:276] 0709 13:08:41.026365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.026459 (+    94us) service_pool.cc:224] Handling call
0709 13:08:42.037603 (+1011144us) spinlock_profiling.cc:243] Waited 83.6 ms on lock 0x7b5400275454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:42.037724 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192194304}
I20260709 13:08:42.054473 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:42.055079 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 167863us
I20260709 13:08:41.986712 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.065824 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.055635 22756 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:42.067728 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.068864 22756 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:42.070044 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:41.947515 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2955) took 1380 ms. Trace:
I20260709 13:08:42.032852 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:42.056000 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.049506 22897 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:08:42.047458 22747 rpcz_store.cc:276] 0709 13:08:40.537290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.942024 (+404734us) service_pool.cc:224] Handling call
0709 13:08:40.971746 (+ 29722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:41.092459 (+120713us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:41.116426 (+ 23967us) log.cc:844] Serialized 14 byte log entry
0709 13:08:41.430300 (+313874us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:41.430345 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:41.430394 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.955639 (+525245us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540016a654. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:41.955702 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.955855 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26646,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":97,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":2769,"spinlock_wait_cycles":278824576,"thread_start_us":313435,"threads_started":1,"wal-append.queue_time_us":360070}
I20260709 13:08:42.060158 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.055426 22896 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:42.010219 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.165128 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:42.164940 22753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 1326) took 1757 ms (client timeout 1857 ms). Trace:
I20260709 13:08:42.165845 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:42.166962 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 77787.1us
I20260709 13:08:42.165161 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:42.176110 22904 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:42.175850 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.070822 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.177256 23445 raft_consensus.cc:2368] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
I20260709 13:08:41.962880 27456 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:41.940220 27309 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.208484 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:42.198539 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.209012 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1101) took 3403 ms. Trace:
W20260709 13:08:42.176890 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 144139us
I20260709 13:08:42.198709 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.988428 27437 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:41.935592 27516 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:41.947275 27578 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:08:42.061231 27561 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:42.174510 22753 rpcz_store.cc:269] 0709 13:08:40.407316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.420034 (+ 12718us) service_pool.cc:224] Handling call
0709 13:08:41.942770 (+1522736us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b5800003398. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:08:42.164878 (+222108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81529728}
I20260709 13:08:41.963789 27326 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.173182 23446 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
W20260709 13:08:42.173444 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 107679us
I20260709 13:08:42.239102 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.198756 27399 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:42.198112 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:42.239748 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.720s (SENT) [suppressed 7 similar messages]
I20260709 13:08:42.241560 22899 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:42.198216 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.198454 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:42.213192 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.178534 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:41.998027 27371 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.236s
I20260709 13:08:41.977054 27496 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.250231 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.250569 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.251217 22830 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:42.251278 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:41.936794 27575 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:42.223405 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.255489 22747 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:42.021166 27415 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:41.939788 27534 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:42.036733 27107 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:42.265764 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:42.266805 22747 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:41.997524 27576 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:42.224237 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.213189 22900 rpcz_store.cc:276] 0709 13:08:38.805629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.856821 (+ 51192us) service_pool.cc:224] Handling call
0709 13:08:38.960912 (+104091us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:39.374037 (+413125us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:39.436074 (+ 62037us) log.cc:844] Serialized 14 byte log entry
0709 13:08:40.700353 (+1264279us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b18001da820. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b23a 00007fcb74e05a2a
0709 13:08:41.017328 (+316975us) log.cc:844] Serialized 14 byte log entry
0709 13:08:41.017512 (+   184us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:41.017985 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:08:41.020671 (+  2686us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:41.020704 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.020788 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.918945 (+898157us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:42.208820 (+289875us) raft_consensus.cc:1685] finished
0709 13:08:42.208840 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:42.208959 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58451,"prepare.queue_time_us":217455,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":158997,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":32091,"spinlock_wait_cycles":18044032,"thread_start_us":673302,"threads_started":3,"wal-append.queue_time_us":1504868}
I20260709 13:08:41.884202 27487 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.230319 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.062685 27454 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:42.061491 27498 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.285773 27487 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.285522 27498 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.174443 22750 rpcz_store.cc:276] 0709 13:08:40.567325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.610150 (+1042825us) service_pool.cc:224] Handling call
0709 13:08:41.610497 (+   347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:41.610993 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:41.611281 (+   288us) log.cc:844] Serialized 14 byte log entry
0709 13:08:41.814447 (+203166us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:41.815006 (+   559us) log.cc:844] Serialized 52 byte log entry
0709 13:08:41.815164 (+   158us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:41.816935 (+  1771us) log.cc:844] Serialized 14 byte log entry
0709 13:08:41.823632 (+  6697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.823695 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:41.947318 (+123623us) raft_consensus.cc:1685] finished
0709 13:08:41.947336 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.947463 (+   127us) inbound_call.cc:177] Queueing success response
0709 13:08:42.022677 (+ 75214us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b1800419e00. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e522 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
Metrics: {"mutex_wait_us":14256,"prepare.queue_time_us":1443,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":2873,"raft.queue_time_us":10357,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":16002,"spinlock_wait_cycles":9665408,"thread_start_us":195479,"threads_started":1,"wal-append.queue_time_us":201303}
I20260709 13:08:42.286695 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.287583 23444 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:42.287683 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.288686 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.242075 27561 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.274312 27534 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:42.289659 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.290611 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.291116 22750 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:42.291232 22754 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:42.291476 22755 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:41.939848 27413 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:42.291919 22755 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:42.292160 27413 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.239801 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:42.214720 22896 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:42.293985 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.291937 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:42.179128 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 96013us
I20260709 13:08:42.214249 27370 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:08:42.180177 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 170052us
I20260709 13:08:42.242525 22897 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:08:42.243057 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.248610 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.243345 22905 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:42.165709 27503 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:08:42.256654 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.230788 27174 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:42.244498 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.305011 23450 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:42.245612 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.244201 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.178774 27579 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:08:42.318831 22755 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:42.273094 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.276604 22830 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:08:42.282999 22899 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:42.245714 27326 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:42.284010 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.334565 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.284600 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.284986 27454 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:42.297134 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 57495.2us
I20260709 13:08:42.340317 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.299053 22831 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:42.296195 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:42.299728 22871 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:42.341523 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 45422.9us
I20260709 13:08:42.341882 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.291182 27534 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:42.301833 23446 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:42.340665 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.343178 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.344449 22831 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:42.345230 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.337764 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.354709 23452 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:42.351213 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:42.356102 22827 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:42.356861 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.357741 23444 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
W20260709 13:08:42.312000 27381 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.018s	sys 0.050s
W20260709 13:08:42.360661 27381 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.018s	sys 0.050s
I20260709 13:08:42.361640 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:42.362978 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.363786 27437 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:42.364128 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:42.364392 27437 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:42.364749 27437 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:42.365089 27437 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:42.365497 27585 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:42.366369 27585 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:42.366801 27585 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.367175 27585 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:42.297516 27309 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.369004 27309 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:42.370091 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 70446.7us
I20260709 13:08:42.371878 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.353801 27454 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:42.372718 27437 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.373499 27370 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:42.374961 27437 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.375466 27573 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:42.375979 27573 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:42.376417 27573 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:42.339715 27174 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.379570 27174 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:42.382036 27579 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:42.382827 27579 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:42.383767 27496 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:42.384156 27496 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:42.384291 27525 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:42.384742 27525 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:42.384899 27496 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.385524 27525 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.386116 27496 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:42.386876 27525 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:42.272467 27546 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.389675 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.390964 27546 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:42.393949 22831 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:42.394757 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.395210 22831 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:42.395292 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:42.396359 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:42.373152 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:42.402297 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51156us
I20260709 13:08:42.404131 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.405699 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.412822 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:42.358482 27543 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:08:42.310534 27552 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:42.414618 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:42.417754 22747 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:08:42.434880 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.454526 22747 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:42.421663 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.424577 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.274286 27107 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:42.460065 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45552us
I20260709 13:08:42.461874 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.463382 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.464059 22752 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 9: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:42.460954 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:42.465523 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:42.429520 22754 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:08:42.412716 27437 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:08:42.435446 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 62385.9us
I20260709 13:08:42.329166 27576 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:42.438365 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.469688 22747 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:42.438838 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2970) took 1357 ms. Trace:
I20260709 13:08:42.470455 22750 rpcz_store.cc:276] 0709 13:08:41.080970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.438036 (+1357066us) service_pool.cc:224] Handling call
0709 13:08:42.438325 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:42.438520 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:42.438546 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:42.438588 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:42.438623 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.438678 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:42.438779 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:42.472309 22747 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:42.442107 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.358845 27415 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:42.443075 27534 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:42.453550 27552 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.443857 27573 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.444392 27575 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:42.413465 27456 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:42.446511 27386 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:42.448738 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.449301 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:42.449736 27351 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.002s
I20260709 13:08:42.450124 22755 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:42.466751 22827 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:42.468570 27437 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:42.474432 27320 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.474884 27413 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.475994 22752 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:42.487789 27552 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:42.454006 27515 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:08:42.477111 27487 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:42.530442 27487 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:42.443481 27497 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:42.459971 27545 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:42.487254 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.487471 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.547674 22905 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 8: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:42.489128 22831 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:08:42.492362 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:42.468839 27579 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:42.495898 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:42.498646 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.501523 23446 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:42.501807 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.551131 22827 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:42.503171 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.504189 27437 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:42.504690 27351 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.505676 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:38478 (request call id 2968) took 1471 ms. Trace:
I20260709 13:08:42.506278 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:42.506722 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.559576 22901 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:42.485231 27546 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:42.498288 22748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.017s	sys 0.054s
I20260709 13:08:42.511997 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:42.560101 22748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.017s	sys 0.054s
I20260709 13:08:42.560564 22902 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:42.492990 27495 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:42.571571 27416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.008s	sys 0.040s
I20260709 13:08:42.571448 27495 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:42.572022 27416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.008s	sys 0.041s
I20260709 13:08:42.528167 27454 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.487551 27415 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:42.531677 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2951) took 1994 ms. Trace:
I20260709 13:08:42.496798 27575 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:42.530938 27487 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:42.549217 22903 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:42.553426 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.476450 27107 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:42.600976 22830 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:42.558538 22755 rpcz_store.cc:276] 0709 13:08:41.028942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.896640 (+867698us) service_pool.cc:224] Handling call
0709 13:08:42.448860 (+552220us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54003cebd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74df2a3d 00007fcb79dc2cd0 00007fcb73347254 00007fcb733471e2 00007fcb73347111 00007fcb7334708c 00007fcb73346342 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:42.500389 (+ 51529us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527418752}
I20260709 13:08:42.567430 27516 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:42.569480 27399 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:42.586072 22898 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:42.587019 22827 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:08:42.585491 22749 rpcz_store.cc:276] 0709 13:08:40.537635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.419006 (+881371us) service_pool.cc:224] Handling call
0709 13:08:41.437871 (+ 18865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:41.438045 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:41.453087 (+ 15042us) log.cc:844] Serialized 14 byte log entry
0709 13:08:42.034004 (+580917us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b18002fa0a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b23a 00007fcb74e05a2a
0709 13:08:42.531353 (+497349us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:42.531412 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:42.531458 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.531527 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:42.531629 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14925,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":1440,"raft.queue_time_us":13240,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":13155584,"thread_start_us":497104,"threads_started":1}
W20260709 13:08:42.595175 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.599771 22823 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:42.560985 27503 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:42.601950 22902 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:42.554087 27545 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:42.565874 27579 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:42.619750 22827 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:08:42.626135 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.627521 22749 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:42.627501 27495 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.581971 27309 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.566040 27506 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 13:08:42.617162 22825 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.027s	sys 0.026s
I20260709 13:08:42.637995 22830 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:42.599148 27575 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:42.651894 22824 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:42.578737 27546 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:42.667539 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 427777us
I20260709 13:08:42.655213 27174 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:42.674095 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.674858 22749 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
W20260709 13:08:42.674996 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:42.701218 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.600677 27576 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.671145 27497 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:42.671811 27174 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:42.725162 27591 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:08:42.746321 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 177565us
I20260709 13:08:42.750810 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.751577 22749 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
W20260709 13:08:42.731812 27592 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:08:42.663331 27415 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:42.742841 27573 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.765865 22749 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:42.770604 27516 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:42.763391 27351 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.727175 27579 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.745884 27497 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:42.742385 27545 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.808450 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 863) took 1210 ms. Trace:
W20260709 13:08:42.794970 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 388676us
I20260709 13:08:42.785121 27454 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:42.813097 22828 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:42.788081 27516 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:42.808954 22829 rpcz_store.cc:276] 0709 13:08:41.598450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.805638 (+207188us) service_pool.cc:224] Handling call
0709 13:08:41.806051 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:41.817011 (+ 10960us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:41.817054 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:41.817593 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:08:42.253259 (+435666us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:42.793453 (+540194us) log.cc:844] Serialized 14 byte log entry
0709 13:08:42.808171 (+ 14718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.808244 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:42.808279 (+    35us) raft_consensus.cc:1685] finished
0709 13:08:42.808290 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:42.808400 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22967,"prepare.queue_time_us":424707,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":30012,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":6628,"thread_start_us":790120,"threads_started":2,"wal-append.queue_time_us":692423}
I20260709 13:08:42.833288 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:42.833484 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.833849 22749 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:42.833972 22829 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:42.812057 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 214763us
I20260709 13:08:42.819533 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2969) took 1769 ms. Trace:
I20260709 13:08:42.808315 27398 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:42.828822 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.846098 22827 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:42.811102 27456 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.831481 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:42.832048 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.787108 27437 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:42.835009 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:42.837281 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 264098us
I20260709 13:08:42.854414 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.855195 22822 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:42.834877 22748 rpcz_store.cc:276] 0709 13:08:41.050418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.908911 (+858493us) service_pool.cc:224] Handling call
0709 13:08:41.909348 (+   437us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:41.909760 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:41.909799 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:41.910480 (+   681us) log.cc:844] Serialized 52 byte log entry
0709 13:08:42.560553 (+650073us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:42.560589 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.560654 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:42.768516 (+207862us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:42.814857 (+ 46341us) raft_consensus.cc:1685] finished
0709 13:08:42.814879 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:42.819468 (+  4589us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":581,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":48517376,"thread_start_us":649662,"threads_started":1,"wal-append.queue_time_us":751062}
I20260709 13:08:42.853190 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.853696 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.857995 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.858428 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:42.859153 22827 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:42.861435 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.864694 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.870712 22829 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:42.866982 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:42.792109 27174 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.874840 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 1338) took 1221 ms. Trace:
I20260709 13:08:42.881181 22751 rpcz_store.cc:276] 0709 13:08:41.653830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.658707 (+  4877us) service_pool.cc:224] Handling call
0709 13:08:42.874687 (+1215980us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b5400267ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:42.874791 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898481536}
I20260709 13:08:42.881595 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.877521 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:42.882639 22752 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:08:42.878304 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 412804us
I20260709 13:08:42.874365 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.883606 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:42.885571 27525 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:42.859411 27456 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.891983 27456 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:42.892030 27610 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:08:42.896225 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:42.897406 27537 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:42.897822 27601 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:08:42.898130 27537 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:42.900131 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:42.888954 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:42.905181 22824 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:08:42.905282 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:42.901294 27456 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:42.911381 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:42.909807 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:42.914790 22749 raft_consensus.cc:2410] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:08:42.863648 27454 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:42.919953 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43693.8us
I20260709 13:08:42.921367 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:42.924768 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:42.877614 27396 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:42.927119 27413 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:42.928493 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:42.875291 27606 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:42.932307 27174 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:42.937049 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:42.875223 27516 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:42.941154 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:42.940778 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:42.922760 27496 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:42.932524 27272 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:42.940218 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:42.941872 22898 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:42.941890 22903 raft_consensus.cc:2368] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12.
I20260709 13:08:42.921211 27537 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:42.943591 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.911170 27616 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:42.958657 22896 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:42.926658 27351 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:42.962448 27516 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:42.927251 27515 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:08:42.930835 27381 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:42.930168 27546 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:42.969863 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:42.970544 27398 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:42.975458 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50728.5us
I20260709 13:08:42.971690 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:42.957898 27597 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:08:42.980767 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:42.981432 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53020.6us
I20260709 13:08:43.006152 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:42.983619 27607 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:08:42.984468 27604 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:08:42.986644 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.987484 23444 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:42.988279 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 861) took 1390 ms. Trace:
I20260709 13:08:42.988819 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:42.989501 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.989744 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:42.990522 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:42.991468 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.991524 27326 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:42.933328 27534 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:08:43.002192 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:42.941895 27494 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 13:08:43.087956 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 107261us
W20260709 13:08:43.098706 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 96599.9us
I20260709 13:08:43.099172 23450 raft_consensus.cc:2368] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12.
I20260709 13:08:43.102437 23447 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:42.982815 27496 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:43.106055 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:43.110055 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 100535us
I20260709 13:08:43.007460 27323 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:43.101992 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.007323 27396 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:08:43.113867 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.127848 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:43.089094 22825 rpcz_store.cc:276] 0709 13:08:41.597965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.728956 (+130991us) service_pool.cc:224] Handling call
0709 13:08:41.729234 (+   278us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:41.750890 (+ 21656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:41.750920 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:41.775279 (+ 24359us) log.cc:844] Serialized 52 byte log entry
0709 13:08:41.890173 (+114894us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b4800130158. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb79d24aa1 00007fcb79d1bb37 00007fcb79d1b9ca 00007fcb79e1326f 00007fcb79e12f37 00007fcb79e2755c 00007fcb79e27508 00007fcb79e27488 00007fcb79e27443 00007fcb79e266ec 00007fcb78d50a96
0709 13:08:42.044486 (+154313us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:42.651447 (+606961us) log.cc:844] Serialized 14 byte log entry
0709 13:08:42.653602 (+  2155us) log.cc:844] Serialized 14 byte log entry
0709 13:08:42.987770 (+334168us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b18001c5f40. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb749b1d70 00007fcb749a5293 00007fcb7499acea 00007fcb7499b23a 00007fcb74e05a2a 00007fcb74e150bf 00007fcb74e15038
0709 13:08:42.987989 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.988054 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:42.988088 (+    34us) raft_consensus.cc:1685] finished
0709 13:08:42.988099 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:42.988223 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":231131,"prepare.queue_time_us":700486,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":2317,"raft.queue_time_us":12670,"raft.run_cpu_time_us":1169,"raft.run_wall_time_us":149846,"spinlock_wait_cycles":18427008,"thread_start_us":773479,"threads_started":2,"wal-append.queue_time_us":478628}
I20260709 13:08:43.118449 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 867) took 1519 ms. Trace:
I20260709 13:08:43.099471 27528 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:43.120352 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.103581 27326 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:08:43.120532 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:43.121204 27250 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:43.125864 27599 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.545s	user 0.002s	sys 0.001s
I20260709 13:08:43.128592 22823 rpcz_store.cc:276] 0709 13:08:41.599351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.901460 (+302109us) service_pool.cc:224] Handling call
0709 13:08:41.901761 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:42.620367 (+718606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:43.118250 (+497883us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54001f90d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:43.118398 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604702464}
I20260709 13:08:43.130259 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.152321 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:43.137133 22719 connection.cc:256] RPC call timeout handler was delayed by -1.0217783680000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
I20260709 13:08:43.137956 22829 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:08:43.138473 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.134908 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:43.104624 27381 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:43.006891 27398 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:43.139494 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:43.090123 27497 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:43.114283 27399 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:43.121232 27552 consensus_meta.cc:287] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.003s
I20260709 13:08:43.154757 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.156438 22825 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:43.002992 27413 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.124s
I20260709 13:08:43.157707 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:43.126729 27323 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:43.115092 27496 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:43.180326 22751 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:43.137358 27528 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:43.140038 27351 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:43.178687 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:43.182633 27613 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
W20260709 13:08:43.184616 27487 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.555s	user 0.004s	sys 0.004s
W20260709 13:08:43.188635 27575 consensus_meta.cc:287] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.525s	user 0.003s	sys 0.003s
I20260709 13:08:43.176399 27250 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:43.190694 22719 leader_election.cc:400] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
I20260709 13:08:43.189337 27437 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:43.148416 27339 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:43.182034 27496 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:43.200466 22898 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:43.201699 22825 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:43.172880 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:43.234416 23450 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:43.199918 22896 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:08:43.211704 23447 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
W20260709 13:08:43.170051 27602 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
I20260709 13:08:43.217653 27627 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
W20260709 13:08:43.218148 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 86663.9us
I20260709 13:08:43.223977 22719 leader_election.cc:403] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:08:43.190701 22794 leader_election.cc:400] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12.
I20260709 13:08:43.180179 27323 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:43.226264 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:43.176868 27611 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:43.228883 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:43.200501 27497 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:43.200109 27528 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:43.257826 27621 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:08:43.263778 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2707) took 1588 ms. Trace:
I20260709 13:08:43.264254 22826 rpcz_store.cc:276] 0709 13:08:41.675236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.256953 (+581717us) service_pool.cc:224] Handling call
0709 13:08:42.257276 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:42.258652 (+  1376us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:42.258696 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:42.261571 (+  2875us) log.cc:844] Serialized 26 byte log entry
0709 13:08:42.452071 (+190500us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b18002b8700. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:08:42.822816 (+370745us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:42.822854 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.823502 (+   648us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:43.262237 (+438735us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:43.263522 (+  1285us) raft_consensus.cc:1685] finished
0709 13:08:43.263549 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:43.263703 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80,"raft.queue_time_us":3150,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":174,"spinlock_wait_cycles":22430208,"thread_start_us":370572,"threads_started":1,"wal-append.queue_time_us":895301}
I20260709 13:08:43.271065 27398 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:43.272164 27635 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:43.272835 27635 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.275513 27635 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:43.232867 23452 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:08:43.277705 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 842) took 1068 ms. Trace:
I20260709 13:08:43.278071 23448 rpcz_store.cc:276] 0709 13:08:42.209423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.209563 (+   140us) service_pool.cc:224] Handling call
0709 13:08:42.209919 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:42.228964 (+ 19045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:42.229008 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:42.233953 (+  4945us) log.cc:844] Serialized 26 byte log entry
0709 13:08:42.676722 (+442769us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:42.676803 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.676868 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:43.277523 (+600655us) raft_consensus.cc:1685] finished
0709 13:08:43.277542 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:43.277654 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14460,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6228,"thread_start_us":397581,"threads_started":1,"wal-append.queue_time_us":544454}
I20260709 13:08:43.236012 22902 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:43.236205 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:43.240386 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 120378us
I20260709 13:08:43.241717 22794 leader_election.cc:403] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:08:43.235590 27585 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.865s	user 0.009s	sys 0.003s
W20260709 13:08:43.281236 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 103466us
I20260709 13:08:43.282188 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:43.249584 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 73915.3us
I20260709 13:08:43.243695 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:43.228385 27575 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.190317 27381 consensus_queue.cc:1059] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:08:43.180987 27552 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.229935 27339 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:43.295264 22897 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:43.230521 27546 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:43.230770 27174 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:43.254253 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:43.224557 27487 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:43.282261 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.248183 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:43.298748 22903 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:43.245409 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.284164 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:43.299878 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54617.6us
I20260709 13:08:43.253445 27534 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
I20260709 13:08:43.251508 27624 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:43.294500 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.300693 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:43.300695 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.299095 22903 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:43.304772 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:43.304975 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.305984 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.285445 27611 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12.
I20260709 13:08:43.310592 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.308048 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:43.291105 27398 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:43.311895 27497 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:43.314208 27622 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:43.309798 27454 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:43.315258 27454 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:43.316143 27454 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:43.312026 27347 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:43.283552 27585 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.310078 27624 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.320829 27585 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:43.320940 27624 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:43.311136 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:43.325598 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:43.327180 22750 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:43.298111 27339 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:43.284595 27326 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:08:43.293421 27272 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:08:43.310747 27537 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:43.335199 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.309417 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.307173 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:43.337018 27624 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:43.340361 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2718) took 1031 ms. Trace:
I20260709 13:08:43.341487 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:43.343086 27575 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:43.307344 27627 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:43.303069 22903 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:43.312096 27250 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:43.310101 27413 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:43.348937 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.349943 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.348603 27622 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:43.353101 22830 rpcz_store.cc:276] 0709 13:08:42.308766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.328833 (+ 20067us) service_pool.cc:224] Handling call
0709 13:08:43.340204 (+1011371us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54003818d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:43.340309 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590390016}
I20260709 13:08:43.335850 27347 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:43.302071 27174 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.302174 27546 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.334496 27636 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:43.360136 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:43.351805 27634 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:08:43.399818 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.312443 27635 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:43.367487 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:43.369480 27632 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:08:43.362598 27624 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:43.366109 27497 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:43.365561 27250 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:08:43.686115 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 323650us
I20260709 13:08:43.362172 27545 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.270s
W20260709 13:08:43.686755 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 54093.7us
I20260709 13:08:43.358103 27631 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.297s
I20260709 13:08:43.698323 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.604765 27575 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:43.700121 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.702910 22830 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:43.700731 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.704010 22826 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:43.366328 27534 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.322s
I20260709 13:08:43.631079 27636 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:43.707718 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:43.711197 22899 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:43.707816 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.684230 27606 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:43.709776 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.713125 22825 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:43.713968 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.714982 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.646324 27413 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:08:43.687636 27634 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:43.730350 22899 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:43.704638 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:43.715492 22825 raft_consensus.cc:2376] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13. Re-sending same reply.
I20260709 13:08:43.709847 22823 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:43.630654 27622 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.104s
W20260709 13:08:43.705296 27516 consensus_meta.cc:287] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.000s
I20260709 13:08:43.720885 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.713363 27497 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:43.775849 22826 raft_consensus.cc:2410] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:08:43.705180 27624 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:08:43.791232 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:43.777462 27639 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.409s	user 0.001s	sys 0.000s
I20260709 13:08:43.735440 27561 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:43.708453 27575 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.718943 27606 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:43.744688 27347 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:43.876142 23447 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:43.796084 27552 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:43.797164 27508 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:43.799232 23448 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:08:43.799500 27496 consensus_meta.cc:287] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.557s	user 0.006s	sys 0.000s
I20260709 13:08:43.800361 27498 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:43.800750 27636 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.778820 27635 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 13:08:43.805646 27624 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:43.805941 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.895557 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:43.896179 22829 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:43.806289 27528 consensus_meta.cc:287] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.000s
I20260709 13:08:43.809748 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1120) took 1319 ms. Trace:
I20260709 13:08:43.903085 22897 rpcz_store.cc:276] 0709 13:08:42.490618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.490731 (+   113us) service_pool.cc:224] Handling call
0709 13:08:42.491034 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:43.298270 (+807236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:43.809518 (+511248us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54001b7454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:43.809690 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849050880}
I20260709 13:08:43.828200 27327 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:43.830405 27226 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:43.833943 27174 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:43.834098 27603 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.895s	user 0.006s	sys 0.002s
W20260709 13:08:43.846622 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55495.3us
I20260709 13:08:43.748754 27614 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:43.779561 27534 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:43.875064 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:43.778093 27487 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:43.881908 27347 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:43.882531 27595 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.209s	user 0.024s	sys 0.022s
I20260709 13:08:43.793813 27415 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:43.744992 27628 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
I20260709 13:08:43.892670 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.925768 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:43.803934 27323 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.001s
I20260709 13:08:43.899698 23448 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:43.905949 27327 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:43.905061 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:43.911708 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:43.941668 23450 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:08:43.889583 27508 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:43.913234 22829 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:43.914682 23452 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:43.914824 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.780455 27250 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:43.894627 27624 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:43.944954 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:43.914417 27534 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:43.904106 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 196343us
I20260709 13:08:43.892436 27496 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:43.944057 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.950933 22829 raft_consensus.cc:2376] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15. Re-sending same reply.
I20260709 13:08:43.951828 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:43.952066 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.953416 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:43.940745 27351 consensus_meta.cc:287] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.707s	user 0.009s	sys 0.000s
I20260709 13:08:43.910699 27528 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:43.926471 22826 raft_consensus.cc:2376] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12. Re-sending same reply.
I20260709 13:08:43.927947 27546 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:43.945346 27339 consensus_meta.cc:287] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.000s
I20260709 13:08:43.949759 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.920692 27635 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:08:43.966673 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:43.954640 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:43.970369 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:43.974100 22829 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:43.974732 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.974982 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:43.947479 27614 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:43.978015 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:43.978776 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:43.980604 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.950563 27226 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:43.979758 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:43.983121 23453 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:43.982908 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:43.984274 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.984654 22794 leader_election.cc:400] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12.
I20260709 13:08:43.985440 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:43.960618 27351 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:43.944293 27323 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:43.989384 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:43.986660 27487 raft_consensus.cc:2764] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:08:43.986256 27646 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:43.997438 27622 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:43.997849 27622 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:43.983708 27537 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:43.998188 27622 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:44.002954 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:44.006122 22754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38478 (request call id 2980) took 2009 ms (client timeout 1532 ms). Trace:
W20260709 13:08:44.006644 22754 rpcz_store.cc:269] 0709 13:08:41.996968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.999416 (+  2448us) service_pool.cc:224] Handling call
0709 13:08:44.005882 (+2006466us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54000780d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:44.006024 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":566855936}
I20260709 13:08:44.008164 22747 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:44.009413 27327 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:44.010326 27579 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:44.010847 27579 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:43.985745 27508 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.011583 27579 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.012688 27579 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:44.015718 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1283) took 1786 ms. Trace:
I20260709 13:08:44.011837 27508 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:44.016112 23451 rpcz_store.cc:276] 0709 13:08:42.232820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.233099 (+   279us) service_pool.cc:224] Handling call
0709 13:08:42.233340 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:42.383150 (+149810us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:08:42.450755 (+ 67605us) log.cc:844] Serialized 14 byte log entry
0709 13:08:42.851562 (+400807us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:42.851625 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:42.851686 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:44.015487 (+1163801us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54006dea54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:44.015537 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.015665 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110680,"prepare.queue_time_us":74163,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":38766,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":18377,"spinlock_wait_cycles":344010624,"thread_start_us":363894,"threads_started":1,"wal-append.queue_time_us":459873,"wal-append.run_cpu_time_us":1088,"wal-append.run_wall_time_us":1042667}
I20260709 13:08:43.982028 27415 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:44.022753 22905 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: []
W20260709 13:08:43.981385 27616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.024s	sys 0.056s
I20260709 13:08:44.028069 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:44.002214 27552 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:44.030297 23452 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:44.034986 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 2982) took 1414 ms. Trace:
I20260709 13:08:44.035380 22750 rpcz_store.cc:276] 0709 13:08:42.620230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.620648 (+   418us) service_pool.cc:224] Handling call
0709 13:08:42.620889 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:43.327655 (+706766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:44.034766 (+707111us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54000771d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:44.034930 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1623778176}
I20260709 13:08:43.973549 27339 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:44.029048 27616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.024s	sys 0.056s
I20260709 13:08:44.039489 27616 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:44.040300 27616 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:44.025519 27327 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:44.043988 27339 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:44.052824 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:44.022085 27415 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:44.018520 27561 consensus_queue.cc:1175] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3b626c41600d4737a178ebab8a7ebfc2" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:44.050407 27646 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:44.068041 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:44.071134 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:44.058590 27628 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:44.002671 27624 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.003692 27534 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:44.074402 27575 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:44.075248 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.075702 27323 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:44.076414 27503 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:44.060281 27552 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:44.076602 27528 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:44.069635 27606 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:44.063335 27579 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:44.077947 27496 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:44.082705 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 116086us
I20260709 13:08:44.104626 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:44.107281 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:44.063158 27415 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:44.083096 27646 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:44.095289 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:44.113215 22901 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:44.115227 22898 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:44.113749 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.114269 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.120688 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:44.097400 27226 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:44.094872 27644 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:44.126935 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.096372 27525 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:08:44.119616 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 134229us
I20260709 13:08:44.139741 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:44.131629 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.134279 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:44.142221 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.140738 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:44.137341 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:44.145210 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:44.109694 27624 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:44.150472 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:44.129572 27606 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:44.100165 27546 consensus_queue.cc:1175] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3b626c41600d4737a178ebab8a7ebfc2" responder_term: 13 status { last_received { term: 10 index: 10 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:44.155867 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:44.148123 27614 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:08:44.158850 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.136685 27534 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:44.156333 27529 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:44.161970 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:44.143101 27579 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:44.186458 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:44.161541 27370 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.187338 22824 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:44.187811 27370 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:44.164484 27634 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:44.149633 27612 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:44.195235 27612 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.196362 22824 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:44.149884 27575 consensus_queue.cc:1059] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:08:44.152937 27633 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:44.197225 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:44.198371 22824 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:44.157421 27606 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.151921 27573 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:44.154753 27496 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:44.202124 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:44.233422 27496 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:44.233644 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:44.234421 22754 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:44.234270 27496 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:44.235653 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:44.236320 22753 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:44.237222 27573 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.239262 27573 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:44.240903 27606 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:44.205478 22824 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:44.210099 27646 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.211158 27622 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.212081 27552 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.264592 22824 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:08:44.148172 27431 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:08:44.218453 27226 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.228897 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:44.232918 27487 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.156471 27628 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:44.243587 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:44.269261 22755 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
W20260709 13:08:44.269665 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 162455us
I20260709 13:08:44.270083 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:44.244493 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:44.233685 23452 raft_consensus.cc:2410] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 11 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:44.245709 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:44.247399 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:44.271440 22749 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:44.257856 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:44.246313 27579 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.267954 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:44.287563 22750 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:44.288491 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:44.280352 23419 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:44.281188 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:44.285734 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.290004 22748 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:44.257215 22753 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
I20260709 13:08:44.290303 22750 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:44.286497 22749 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:44.291791 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:44.292847 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:44.293674 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:44.293913 23419 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:44.298703 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:44.299371 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:44.303023 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:44.305969 22748 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:44.309329 22748 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:44.292882 22749 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:44.294147 27644 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:44.295603 27508 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:44.318637 22749 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:44.290788 27506 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:44.300187 27635 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:44.299755 27528 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:44.306906 27431 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:44.323009 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:44.330577 22753 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:44.318886 27371 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:44.305217 27634 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:44.304736 27649 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:08:44.279093 27487 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.297302 27579 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.326325 22750 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:08:44.298272 27636 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:08:44.336764 27371 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:44.340905 27644 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.327224 27506 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:44.327399 27635 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:44.355611 22756 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:44.358575 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:44.359313 22898 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:44.362190 22822 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:08:44.363062 27415 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:44.363512 27635 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:44.364622 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2343) took 1927 ms. Trace:
I20260709 13:08:44.336453 27431 raft_consensus.cc:697] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:44.371520 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:44.372905 22750 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:08:44.364977 22900 rpcz_store.cc:276] 0709 13:08:42.437625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.440606 (+  2981us) service_pool.cc:224] Handling call
0709 13:08:42.440912 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:42.476947 (+ 36035us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:42.477230 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 13:08:42.676719 (+199489us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:42.676778 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:42.676838 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:44.363799 (+1686961us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b18003063a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419
0709 13:08:44.364436 (+   637us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b5400107554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:44.364477 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.364574 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7056,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":4973,"raft.queue_time_us":585,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":48384,"spinlock_wait_cycles":90050176,"thread_start_us":198538,"threads_started":1,"wal-append.queue_time_us":334113,"wal-append.run_cpu_time_us":1647,"wal-append.run_wall_time_us":1080646}
W20260709 13:08:44.374449 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.374881 27508 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:44.375218 27622 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:44.357638 27506 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:44.381963 22751 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:44.365126 27415 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:44.356030 27371 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:44.390395 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:44.393904 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:44.371194 27525 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:44.380910 27628 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.394145 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:44.384047 27659 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:44.398834 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:44.399390 22753 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:44.387764 27545 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:08:44.439744 22897 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:44.427194 22753 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:44.444684 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:44.402539 27525 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:44.404728 27628 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.488293 27636 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:44.498595 27525 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:44.455399 27431 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:44.503358 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:44.508635 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 112433us
I20260709 13:08:44.512969 27579 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:44.513036 27272 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:44.521950 27272 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:44.522976 27272 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.524233 27272 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:44.510984 27487 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:44.526315 27596 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.798s	user 0.001s	sys 0.019s
I20260709 13:08:44.527384 27628 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:44.527717 27498 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:44.528095 27498 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:44.530417 27576 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:44.530938 27576 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:44.531339 27612 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:44.528914 27498 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.532545 27498 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:44.531916 27576 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.533619 27576 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:44.534324 27506 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.542236 27628 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:44.542506 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1872) took 3519 ms. Trace:
I20260709 13:08:44.542754 27628 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:44.542872 22828 rpcz_store.cc:276] 0709 13:08:41.023190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.711162 (+687972us) service_pool.cc:224] Handling call
0709 13:08:41.711403 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:42.818474 (+1107071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:44.542293 (+1723819us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000ac3d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:44.542453 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2533199104}
I20260709 13:08:44.544445 22828 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:08:44.545053 27635 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:44.546034 27609 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.458s	user 0.000s	sys 0.011s
I20260709 13:08:44.546132 23446 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:44.546628 27628 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:44.562120 27635 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:44.568475 27506 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:44.571398 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.573639 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.574930 27625 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:44.575114 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 857) took 1442 ms. Trace:
I20260709 13:08:44.575606 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:44.575543 23445 rpcz_store.cc:276] 0709 13:08:43.132981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.133117 (+   136us) service_pool.cc:224] Handling call
0709 13:08:43.133430 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:44.547738 (+1414308us) spinlock_profiling.cc:243] Waited 9.94 ms on lock 0x7b1800428aa0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb749a34bc 00007fcb71cf148b 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0b455 00007fcb74df3060 00007fcb74df3bfb 00007fcb74dfe64b
0709 13:08:44.547913 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:44.547951 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:44.548515 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:08:44.574726 (+ 26211us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:44.574750 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:44.574835 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:44.574879 (+    44us) raft_consensus.cc:1685] finished
0709 13:08:44.574892 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.575065 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":65967,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":4462,"spinlock_wait_cycles":22851968,"thread_start_us":115313,"threads_started":2,"wal-append.queue_time_us":24853}
I20260709 13:08:44.575830 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:44.573182 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:44.584887 27437 raft_consensus.cc:3055] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:44.587678 27670 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:08:44.590236 27638 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.226s	user 0.001s	sys 0.001s
I20260709 13:08:44.585150 27371 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.592276 27370 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:44.592597 27371 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:44.595410 27670 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:44.595813 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2355) took 1288 ms. Trace:
I20260709 13:08:44.595419 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:44.596604 23450 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:44.596201 22905 rpcz_store.cc:276] 0709 13:08:43.307407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.308689 (+  1282us) service_pool.cc:224] Handling call
0709 13:08:43.321692 (+ 13003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:44.026693 (+705001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:44.595601 (+568908us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54003e9254. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:44.595760 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1611379584}
W20260709 13:08:44.598242 27620 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.510s	user 0.002s	sys 0.007s
I20260709 13:08:44.585300 27437 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:44.601405 23452 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:08:44.603940 23450 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:08:44.604108 27670 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:44.615538 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:44.723910 27666 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 13:08:44.876649 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.883203 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:44.872196 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:44.890514 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:44.915673 27688 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.926515 27688 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.928764 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2717) took 2620 ms. Trace:
W20260709 13:08:44.935778 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.924588 27686 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:44.936925 27686 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:44.933037 22831 rpcz_store.cc:276] 0709 13:08:42.308478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.561460 (+252982us) service_pool.cc:224] Handling call
0709 13:08:42.561782 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:42.562172 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:42.562211 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:42.562772 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:08:42.845863 (+283091us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:42.845904 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.845975 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:44.597584 (+1751609us) spinlock_profiling.cc:243] Waited 9.18 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:44.927962 (+330378us) raft_consensus.cc:1685] finished
0709 13:08:44.928005 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.928422 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2246,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":10737,"spinlock_wait_cycles":21105536,"thread_start_us":217071,"threads_started":1,"wal-append.queue_time_us":357132}
I20260709 13:08:44.940536 27686 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:44.943140 27688 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:44.947911 27688 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:44.932281 27689 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:44.954259 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.954893 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.961855 27689 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:44.965612 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.968111 27416 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.969132 27416 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.948628 27690 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:08:44.964712 27689 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:44.984025 27370 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:44.984177 27411 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.984974 27370 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.985131 27411 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:44.988339 27370 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:44.989333 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:44.990043 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:44.990243 22902 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:44.990989 27634 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:44.991715 27498 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.992041 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:44.993909 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:44.999482 27625 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:45.001490 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.003106 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:45.005816 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.004412 27534 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.015703 27689 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:45.024252 23450 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:45.024747 23453 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:45.024381 23451 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:45.025625 27646 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.025332 27498 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.026425 27646 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.028324 27498 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.025740 23447 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
W20260709 13:08:45.028990 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.024327 27689 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.032860 27416 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.029028 27646 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.038900 27411 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:45.039172 23451 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:45.040900 27690 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:45.041714 27690 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.044075 27690 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.009763 27634 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:45.023707 27625 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:45.048801 27534 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.049211 27287 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.049914 27370 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:45.050246 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:45.051517 23447 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:08:45.052939 27689 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.055222 22902 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:45.064904 27287 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.065915 27634 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.067480 27634 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:45.067926 27287 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.069615 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.074208 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:45.075443 27636 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:45.076015 27636 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:45.069541 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.077831 27494 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:45.079548 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 856) took 1947 ms. Trace:
I20260709 13:08:45.083740 23452 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:45.084971 27636 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.087371 27528 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:45.087752 27528 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:45.090214 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2494) took 2229 ms. Trace:
I20260709 13:08:45.079988 23444 rpcz_store.cc:276] 0709 13:08:43.132091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.133218 (+  1127us) service_pool.cc:224] Handling call
0709 13:08:43.133984 (+   766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:44.584677 (+1450693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:44.584711 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:44.585151 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:08:44.968365 (+383214us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:44.968472 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.078877 (+110405us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54005e1ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:45.078941 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:45.078999 (+    58us) raft_consensus.cc:1685] finished
0709 13:08:45.079016 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:45.079318 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":956,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":1935,"spinlock_wait_cycles":3335423360,"thread_start_us":382408,"threads_started":1,"wal-append.queue_time_us":103861}
I20260709 13:08:45.090642 22752 rpcz_store.cc:276] 0709 13:08:42.860612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.902478 (+ 41866us) service_pool.cc:224] Handling call
0709 13:08:42.911031 (+  8553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:44.081740 (+1170709us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:44.081770 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:44.081808 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:44.081837 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.089642 (+1007805us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400336dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:45.089722 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:45.089994 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2691859200}
I20260709 13:08:45.093271 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1343) took 3438 ms. Trace:
I20260709 13:08:45.093725 22747 rpcz_store.cc:276] 0709 13:08:41.655140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:42.485304 (+830164us) service_pool.cc:224] Handling call
0709 13:08:44.008011 (+1522707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:44.008525 (+   514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.092800 (+1084275us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54000780d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:45.093055 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3501084928}
I20260709 13:08:45.097167 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:45.098963 27494 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:45.107581 23445 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:45.115931 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.116648 22899 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:45.120561 27516 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:45.120998 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:45.121127 27516 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:45.124785 27676 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:45.088943 27528 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.130136 27676 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.130643 27676 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:45.128398 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:45.111387 27494 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:45.128059 27516 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.128629 27508 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:45.111176 27689 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:45.133956 27494 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.135711 27494 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:45.134622 22903 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:45.135987 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:45.140378 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 978935us
I20260709 13:08:45.140770 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2735) took 1293 ms. Trace:
I20260709 13:08:45.134569 27689 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.141752 27689 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:45.142233 27573 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:45.142711 27573 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:45.144313 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.144572 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:45.144203 27573 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.146149 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.147094 22755 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:45.135942 27516 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:45.150171 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.153348 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:45.158072 27573 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:45.141196 22826 rpcz_store.cc:276] 0709 13:08:43.847744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.971700 (+123956us) service_pool.cc:224] Handling call
0709 13:08:43.971954 (+   254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:43.974904 (+  2950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:43.974931 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:43.978106 (+  3175us) log.cc:844] Serialized 26 byte log entry
0709 13:08:44.154160 (+176054us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:44.154530 (+   370us) log.cc:844] Serialized 14 byte log entry
0709 13:08:44.376901 (+222371us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b180041dee0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:45.139743 (+762842us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b180041dee0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb74df3060 00007fcb74e05716 00007fcb74e16287 00007fcb74e16208
0709 13:08:45.140147 (+   404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.140286 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:45.140375 (+    89us) raft_consensus.cc:1685] finished
0709 13:08:45.140387 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:45.140595 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5173,"prepare.queue_time_us":619,"prepare.run_cpu_time_us":7917,"prepare.run_wall_time_us":18212,"raft.queue_time_us":2669,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":18889600,"thread_start_us":175722,"threads_started":1,"wal-append.queue_time_us":178790}
I20260709 13:08:45.152655 27508 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:45.154089 27686 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:45.156682 27688 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:08:45.175630 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.162220 27528 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:45.181454 27628 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:08:45.181519 27690 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:45.184933 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.171540 27646 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:08:45.186977 22902 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:08:45.175596 27688 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.187668 27688 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:45.175839 27686 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.202076 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:45.212538 23445 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: [17.16-17.16]
I20260709 13:08:45.213546 27323 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.214370 27323 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:45.215329 27537 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.215449 27552 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.215966 27537 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:45.216346 27534 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.216138 27552 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.216764 27323 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:45.217468 27686 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:45.218511 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.218716 27506 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:45.219487 27351 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.212482 27250 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:45.214649 27454 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.218578 27552 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.227383 27351 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.233405 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.209692 27508 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:44.911270 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:45.244042 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.245168 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.245559 27454 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.245954 27561 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:08:45.246687 27561 consensus_queue.cc:1243] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:08:45.247242 27561 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:45.247167 27676 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.247761 27561 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:45.247900 27454 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.229182 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:45.251722 22903 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:45.238201 27508 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.257709 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:45.259548 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:45.260788 27431 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:45.256757 27687 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:08:45.257402 27309 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:08:45.261667 27506 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:45.262146 27308 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.262229 27506 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:45.262566 27351 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.262636 27308 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.254740 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:45.271610 22903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1773) took 2023 ms (client timeout 2136 ms). Trace:
W20260709 13:08:45.272145 22903 rpcz_store.cc:269] 0709 13:08:43.247762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.247913 (+   151us) service_pool.cc:224] Handling call
0709 13:08:43.819119 (+571206us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5400439520. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d70a1d 00007fcb74d571a4 00007fcb74dd9447 00007fcb74df1fe7 00007fcb74df37de 00007fcb74dff27c 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591
0709 13:08:45.133798 (+1314679us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b580065e1d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d570aa 00007fcb74dd9447 00007fcb74df1fe7 00007fcb74df37de 00007fcb74dff27c 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c
0709 13:08:45.271514 (+137716us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771773568}
I20260709 13:08:45.248853 27561 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.261344 27431 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:45.261492 27687 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.285547 27687 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:45.287065 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.290779 22751 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:08:45.291209 22751 pending_rounds.cc:85] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 5
I20260709 13:08:45.291430 22751 pending_rounds.cc:107] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:08:45.291708 22751 raft_consensus.cc:2889] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:45.293346 22752 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:45.294540 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:45.295156 27415 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:45.295581 27415 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:45.262656 27506 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:45.300367 27308 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:45.296622 27415 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.302062 27537 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:45.302275 22748 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:08:45.302398 27415 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:45.305132 27690 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.305735 27690 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.307693 27690 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:45.309748 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 85490.7us
I20260709 13:08:45.310042 27622 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:45.310686 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:45.311422 27534 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:45.311825 27534 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:45.313513 27411 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:45.314190 22901 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:08:45.312834 27534 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.316430 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.316712 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:45.317384 22755 raft_consensus.cc:2376] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17. Re-sending same reply.
I20260709 13:08:45.278971 27399 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:45.281754 23448 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:45.312899 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:45.312215 27625 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.343839 27625 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:45.343947 27415 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:45.344493 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.345655 27415 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:45.345794 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.346086 27689 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.347033 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.348160 27689 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.349392 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.353689 27496 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:45.363900 27506 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.366525 27676 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.318348 27431 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:45.372028 22871 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.373644 23444 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:45.375092 22749 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:45.375412 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:45.376529 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:45.379802 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.348191 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.382459 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.385980 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.392568 23444 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:45.394488 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.401780 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.400303 22749 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:45.400543 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.405441 22753 raft_consensus.cc:2410] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:08:45.382164 27496 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:08:45.409644 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.411667 27694 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.411856 27693 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.413127 27107 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:45.413661 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 885) took 1702 ms. Trace:
I20260709 13:08:45.399844 27250 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:45.424283 27250 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:45.426071 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.408093 27561 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:45.412443 27694 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.430830 23444 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:08:45.421540 27573 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:45.412560 27693 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.439891 27692 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.440645 27534 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:45.440892 27692 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.446671 27454 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.447271 27694 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:45.449535 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 71035.1us
I20260709 13:08:45.501500 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1899) took 1429 ms. Trace:
I20260709 13:08:45.464923 27687 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.501885 22822 rpcz_store.cc:276] 0709 13:08:44.061933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:44.362869 (+300936us) service_pool.cc:224] Handling call
0709 13:08:44.363155 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:45.237553 (+874398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:45.237609 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:45.238350 (+   741us) log.cc:844] Serialized 26 byte log entry
0709 13:08:45.266678 (+ 28328us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:45.266707 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.277247 (+ 10540us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:45.491354 (+214107us) raft_consensus.cc:1685] finished
0709 13:08:45.491384 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:45.491559 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":868370,"raft.queue_time_us":973466,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":5692,"thread_start_us":80210,"threads_started":2,"wal-append.queue_time_us":191693}
I20260709 13:08:45.503499 22750 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:45.503674 27687 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.503814 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:45.450038 27552 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.505268 27552 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.507011 27272 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:45.507853 27674 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:45.450649 27696 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:45.501910 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:45.451283 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.508831 27696 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.504523 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.510589 27694 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:45.511016 27696 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.451191 27606 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.513078 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.513025 27606 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.513441 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.514575 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.453814 27351 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.515383 27606 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.516018 27606 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:45.515398 23447 raft_consensus.cc:2410] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:08:45.516595 27606 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.453573 27654 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.454435 22829 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:45.457283 22823 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
W20260709 13:08:45.458541 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 67833.9us
I20260709 13:08:45.455644 27437 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.460165 22654 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 9 to 10, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 10 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:45.461973 22830 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:45.463241 27693 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.465590 27506 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:45.467608 27686 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.456501 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:45.440186 27669 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:08:45.479027 27697 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:45.480245 27692 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.483021 27537 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.484488 27705 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.489470 27698 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:45.500746 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.468150 27431 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.507342 27552 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.515693 27351 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.455637 22826 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:08:45.513103 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.456874 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:45.502956 22653 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 340) took 20237 ms (client timeout 15000 ms). Trace:
I20260709 13:08:45.417465 22825 rpcz_store.cc:276] 0709 13:08:43.710788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.743602 (+ 32814us) service_pool.cc:224] Handling call
0709 13:08:43.743862 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:43.744041 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:43.909639 (+165598us) log.cc:844] Serialized 14 byte log entry
0709 13:08:45.403678 (+1494039us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:45.413230 (+  9552us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:45.413329 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.413400 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:45.413512 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":896292,"prepare.queue_time_us":253449,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1101810,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":691521,"threads_started":3,"wal-append.queue_time_us":394861,"wal-append.run_cpu_time_us":1268,"wal-append.run_wall_time_us":1055068}
I20260709 13:08:45.529747 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:45.529687 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:45.529268 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:45.566836 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.526567 22748 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:45.566023 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.551486 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:45.517488 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:45.549445 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45012.7us
W20260709 13:08:45.577427 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.521147 23453 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:45.525913 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.534744 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:45.591390 22752 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:45.593511 22749 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15.
I20260709 13:08:45.525913 22753 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:45.537269 27503 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.549165 27308 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:45.614791 22831 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:45.615315 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.616776 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.616935 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:45.567924 27694 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:45.620615 22756 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:45.620906 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.621374 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:45.618556 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:45.623919 23449 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:45.620798 23447 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:45.598201 27516 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:45.625038 22827 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:45.625169 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:45.627172 27437 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:45.627315 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 170927us
I20260709 13:08:45.628185 27705 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.629545 27503 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.630478 27705 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.631300 27351 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.631683 27503 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.632182 27694 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.633488 22755 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:45.530848 27454 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.597636 27654 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.615998 27692 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:45.614359 27687 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:45.643931 22748 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:45.645054 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.646176 22903 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:45.591150 27696 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.620954 27697 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.653389 27370 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:45.658175 23453 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:08:45.659446 27654 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.659588 27454 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.660077 27308 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:45.658164 22751 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:08:45.550981 27688 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.653254 27694 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.653749 22654 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:45.670774 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.646463 27692 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:45.667779 22751 pending_rounds.cc:85] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:08:45.671977 22751 pending_rounds.cc:107] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:08:45.672294 22751 raft_consensus.cc:2889] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:45.658123 27503 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:45.673494 27503 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:45.678392 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.678755 23449 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:45.680801 23448 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:08:45.680850 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.681285 23448 pending_rounds.cc:85] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 10
I20260709 13:08:45.681593 23448 pending_rounds.cc:107] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:08:45.681828 23448 raft_consensus.cc:2889] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:45.683195 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68018.3us
I20260709 13:08:45.661178 27431 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:45.626466 27698 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.646684 27537 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.673979 22654 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:45.689069 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:45.661334 27634 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:08:45.656157 27687 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.692701 27399 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.708657 22903 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:45.676105 27696 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.695950 27494 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:45.686554 27694 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:08:45.719000 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.719352 27308 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.721603 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2737) took 1774 ms. Trace:
I20260709 13:08:45.721954 22824 rpcz_store.cc:276] 0709 13:08:43.862800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:44.269436 (+406636us) service_pool.cc:224] Handling call
0709 13:08:44.295686 (+ 26250us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:45.250615 (+954929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:45.253833 (+  3218us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:45.263325 (+  9492us) log.cc:844] Serialized 52 byte log entry
0709 13:08:45.383189 (+119864us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:45.578893 (+195704us) log.cc:844] Serialized 14 byte log entry
0709 13:08:45.618692 (+ 39799us) log.cc:844] Serialized 14 byte log entry
0709 13:08:45.621765 (+  3073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.625717 (+  3952us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:45.625761 (+    44us) raft_consensus.cc:1685] finished
0709 13:08:45.625771 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:45.637454 (+ 11683us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":904232,"raft.queue_time_us":1119678,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":21526,"spinlock_wait_cycles":6494592,"thread_start_us":360783,"threads_started":3,"wal-append.queue_time_us":133593}
I20260709 13:08:45.707698 27454 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:08:45.726965 27399 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.729456 27494 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.729871 27494 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:45.732234 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.733589 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.733791 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.735066 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.735402 23452 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:08:45.736297 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.736888 22896 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:08:45.737578 27692 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:45.711990 27528 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:08:45.739629 27528 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:45.739995 27528 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:45.740960 27516 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:45.741411 27516 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:45.743871 27688 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.745994 27561 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.746553 27561 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.748477 27561 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.748584 23419 leader_election.cc:400] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15.
I20260709 13:08:45.748929 23419 leader_election.cc:403] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:08:45.749127 27431 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:45.750064 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:45.675232 27697 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.756044 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.735658 22897 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:45.737195 22898 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:08:45.735188 22899 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:08:45.754451 23447 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
W20260709 13:08:45.760572 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54075us
W20260709 13:08:45.760830 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.756423 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:45.764745 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:45.674539 27503 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:45.771188 27692 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:45.756045 27688 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.773957 27688 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.776041 27688 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.776679 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.776898 27431 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:45.778069 27674 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:45.778853 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:45.779747 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.780081 27272 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:45.782410 27687 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.760479 27676 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15.
I20260709 13:08:45.783535 27552 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:45.784466 27552 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:45.784916 27552 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:45.785379 27552 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:45.785491 27431 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:45.785993 27431 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.786319 27431 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:45.787678 27654 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:45.788307 27654 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:45.791339 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.792398 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:45.781143 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.794291 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:45.795899 22756 raft_consensus.cc:2368] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12.
I20260709 13:08:45.796758 27351 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:45.793257 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:45.740754 27528 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.797785 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:45.748947 27561 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.798240 22823 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:08:45.800895 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.801656 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:45.809644 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.815423 22827 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:45.803341 27323 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:45.816623 27323 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.816248 27272 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:45.817034 27323 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:45.818048 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:45.818392 22825 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:08:45.806777 27573 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:08:45.819499 22654 catalog_manager.cc:5697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 9 to 10, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 10 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:45.824469 22654 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 7 to 9, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 9 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:45.833287 22831 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:45.834519 27399 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:45.799500 27370 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:45.843918 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.811560 27693 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:08:45.576428 22653 rpcz_store.cc:269] 0709 13:08:25.265032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.265310 (+   278us) service_pool.cc:224] Handling call
0709 13:08:25.274884 (+  9574us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b54000ab4d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d65090 00007fcb74e03f3a 00007fcb7abba1cc 00007fcb7acb837e 00007fcb791792a4 00007fcb79179232 00007fcb79179161 00007fcb791790dc 00007fcb79178392 00007fcb7313648c 00007fcb731358c6
0709 13:08:45.502699 (+20227815us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:38.298348 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:38.435273 (+136925us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:38.435325 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:08:38.435347 (+    22us) tablet.cc:662] Decoding operations
0709 13:08:38.436423 (+  1076us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:38.436526 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:38.436556 (+    30us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:08:38.437418 (+   862us) tablet.cc:701] Row locks acquired
0709 13:08:38.437479 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:08:38.437637 (+   158us) write_op.cc:270] Start()
0709 13:08:38.437776 (+   139us) write_op.cc:276] Timestamp: P: 1783602518437573 usec, L: 0
0709 13:08:38.437794 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:08:38.438471 (+   677us) log.cc:844] Serialized 3647 byte log entry
0709 13:08:38.586071 (+147600us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b54000aba20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d70dbe 00007fcb74d7b930 00007fcb74d7b8d8 00007fcb74d7b858 00007fcb74d7b813 00007fcb74d7aa9c 00007fcb7ac66626 00007fcb7ac60dd8 00007fcb74db77cb 00007fcb74dba2c0
0709 13:08:39.357112 (+771041us) op_driver.cc:464] REPLICATION: finished
0709 13:08:40.079016 (+721904us) write_op.cc:301] APPLY: starting
0709 13:08:40.091957 (+ 12941us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:40.135687 (+ 43730us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:40.135712 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:40.880701 (+744989us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:40.950841 (+ 70140us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 13:08:40.966895 (+ 16054us) write_op.cc:312] APPLY: finished
0709 13:08:41.017393 (+ 50498us) log.cc:844] Serialized 89 byte log entry
0709 13:08:41.382833 (+365440us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:41.401173 (+ 18340us) write_op.cc:454] Released schema lock
0709 13:08:41.402244 (+  1071us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:41.424832 (+ 22588us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:41.426871 (+  2039us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1349786,"reported_tablets":37,"spinlock_wait_cycles":10096512,"child_traces":[["op",{"apply.queue_time_us":613192,"apply.run_cpu_time_us":43091,"apply.run_wall_time_us":1415884,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41747,"num_ops":9,"prepare.queue_time_us":272281,"prepare.run_cpu_time_us":30633,"prepare.run_wall_time_us":314273,"raft.queue_time_us":663104,"raft.run_cpu_time_us":28905,"raft.run_wall_time_us":552512,"replication_time_us":855172,"spinlock_wait_cycles":59335424,"thread_start_us":968705,"threads_started":5,"wal-append.queue_time_us":529160,"wal-append.run_cpu_time_us":22984,"wal-append.run_wall_time_us":2294164}]]}
I20260709 13:08:45.849537 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:45.853456 27687 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:45.854275 27687 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.854722 27687 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.855266 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:45.855657 27698 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.856237 27496 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:45.856575 27496 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:45.857348 27689 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:45.860931 27496 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.818486 27575 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:45.810777 27686 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.856565 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:45.865151 22654 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 10 to 11, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 11 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:45.876786 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.853066 27516 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.811093 27561 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.856534 27687 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:45.882897 22825 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:45.846899 27688 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:45.879427 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.864184 27415 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:45.894282 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:45.894824 23448 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:45.895364 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.897512 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:45.898169 23449 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:45.862242 27634 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:08:45.895915 23444 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:45.894428 23445 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:45.896054 22654 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 7 to 10, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 10 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:45.891182 27686 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:45.946307 22654 catalog_manager.cc:5697] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 4 to 6, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 6 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:45.952857 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:45.953663 23453 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:08:45.901392 27415 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:45.924578 27496 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:45.881415 27690 consensus_queue.cc:1175] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c45e1805f049467697a7358fb566de2f" responder_term: 14 status { last_received { term: 14 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 14 index: 11 } }
I20260709 13:08:45.959333 27686 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:45.948413 27688 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.982560 23445 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:45.988727 22823 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:08:45.997611 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 236091us
I20260709 13:08:45.999675 22824 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:08:46.009862 27686 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:45.989035 27537 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.025236 22823 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:08:46.024080 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 219688us
I20260709 13:08:46.033095 23449 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:46.033366 23448 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:46.030757 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:46.049629 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.049767 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.072592 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.074805 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.087033 22896 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:46.075457 22904 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:08:46.045765 27537 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:46.079823 22901 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:46.052706 27688 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.105037 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:46.128331 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:46.051025 27250 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:08:46.045517 27575 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:08:46.128278 22654 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 10 to 11, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 11 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:46.140025 22654 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:46.141906 27712 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:08:46.143476 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:46.144951 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:46.146970 27579 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.147724 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:46.148351 22747 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:08:46.156816 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.157543 22905 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:46.102454 27537 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.091653 27454 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:46.147549 27579 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.107084 27687 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.044392 27308 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 13:08:46.162184 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.105947 27506 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:46.073101 27415 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.178936 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:46.171911 27537 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:46.134164 27561 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.206542 27534 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:46.206738 27309 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:46.207118 27534 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:46.195533 27494 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:46.225873 27534 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.226887 27534 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:46.227272 27494 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:46.229264 27506 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.231236 27506 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.189946 27687 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.234301 27537 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.235102 27687 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.235680 22823 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 15: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:46.238265 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:46.239068 23453 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:46.240201 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:46.243130 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:46.244509 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:46.246718 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:46.247563 22825 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:08:46.195873 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:46.197677 27437 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.570s	user 0.005s	sys 0.005s
I20260709 13:08:46.198011 27579 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.198750 22751 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:08:46.199442 27617 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:46.199829 27552 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:46.203136 27454 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.195253 22822 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:46.130352 27688 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:08:46.207285 27309 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:46.129387 27705 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:08:46.213824 27431 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.219586 27528 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:46.259407 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.286218 22902 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:46.260465 22829 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:08:46.286659 22829 pending_rounds.cc:85] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 9
I20260709 13:08:46.286891 22829 pending_rounds.cc:107] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:08:46.287104 22829 raft_consensus.cc:2889] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:46.261969 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:46.274398 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:46.285388 23419 leader_election.cc:400] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12.
I20260709 13:08:46.290561 23419 leader_election.cc:403] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:08:46.260138 27508 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:46.298185 22826 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:46.270596 27506 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:08:46.277727 27716 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:08:46.267668 27617 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:46.284823 27694 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:46.308812 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.310258 23452 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:46.312147 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.262872 27323 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.297539 27250 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:46.317898 22902 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:46.319478 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:46.325006 22756 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:46.301882 27506 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:46.327591 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.327883 22750 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:08:46.336534 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 212563us
I20260709 13:08:46.337617 22905 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:46.326154 27733 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.342716 22751 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:08:46.340124 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:46.347518 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:46.332971 27506 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:46.331544 27508 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12.
I20260709 13:08:46.356382 22828 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:46.357583 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:46.358194 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 56655.1us
I20260709 13:08:46.358480 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:46.360291 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:46.361272 27534 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:46.361459 27718 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.361756 27454 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.342401 27687 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:46.356938 27622 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:46.365741 27719 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 13:08:46.359714 27688 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
W20260709 13:08:46.374295 27713 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:46.374696 27534 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:46.375191 27724 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:46.384454 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 89076.9us
I20260709 13:08:46.361976 27718 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.387504 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:46.389053 23449 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:46.369618 27496 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:46.392354 23448 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:46.395241 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:46.396409 27431 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:46.378466 27537 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.382031 27622 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.403153 27431 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:46.407542 23446 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:46.387909 27724 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.408831 23450 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:46.409670 23444 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:08:46.410064 27579 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:46.410375 27693 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.412720 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:46.431468 22751 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:46.415910 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:46.416616 27552 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:46.417573 27323 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.350432 27733 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.418269 27496 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:46.419710 22748 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:08:46.375255 27731 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:46.446095 27731 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:46.434460 27693 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:46.446650 27731 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:46.449472 27730 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:46.450057 27730 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.450497 27730 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:46.457263 27399 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:46.429494 23449 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:08:46.379717 27687 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.426530 27622 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:46.472415 22756 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:46.466666 23449 pending_rounds.cc:85] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 9
I20260709 13:08:46.429963 27724 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:46.488610 27399 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.489495 27399 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.433702 27308 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.495015 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:46.503237 22756 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:46.492600 27399 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:46.473834 23449 pending_rounds.cc:107] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.10
I20260709 13:08:46.441655 27431 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.442061 27579 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.508759 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:46.513454 23449 raft_consensus.cc:2917] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Aborting config change with OpId 10.10: Aborted: Op aborted by new leader
I20260709 13:08:46.498399 27537 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:08:46.511595 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 95827us
I20260709 13:08:46.516388 23449 raft_consensus.cc:2889] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 11 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:46.519345 27399 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:46.526825 27399 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.527168 27399 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:46.563411 22751 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:46.565465 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:46.504295 27308 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.569270 27508 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:46.571146 27697 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.572683 27431 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:46.578846 27725 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:08:46.579560 27718 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.580260 27733 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.581432 27687 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.518105 27537 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:46.534874 27719 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.586716 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:46.597358 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:46.600690 27579 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.601177 27697 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:46.601692 27606 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.600610 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:46.602326 27693 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.603605 22797 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:46.604079 27606 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:46.604347 27693 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.604646 27606 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:46.604965 27693 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:46.605942 27736 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:08:46.605746 27606 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:46.606853 27732 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:08:46.608063 27719 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.610502 27719 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.611156 22897 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:46.611955 22897 raft_consensus.cc:606] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:46.509390 27534 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.612294 22897 raft_consensus.cc:614] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:46.615428 23449 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 13:08:46.617929 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:46.618388 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:46.618543 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:46.592947 27508 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:46.596688 27537 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:46.560528 27496 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.623054 27516 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.626866 27399 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.628801 27689 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:46.628873 27687 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:46.622290 27508 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.617027 27731 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.633790 23446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:08:46.643450 23446 raft_consensus.cc:606] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:46.643883 23446 raft_consensus.cc:614] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:46.634096 27697 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:46.634260 27745 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:08:46.637701 22901 raft_consensus.cc:2410] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:08:46.638232 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:46.636646 27309 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.655016 27399 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:46.664904 27693 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:46.665506 27731 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.624818 27724 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.670858 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:46.686453 22747 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:46.672629 27308 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:46.673141 27506 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.687319 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:46.695983 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 57962.6us
I20260709 13:08:46.697044 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:46.699630 22797 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:46.703275 22896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:46.672729 27537 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.711711 27724 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.713457 27693 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:08:46.730196 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:46.730803 22747 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:46.731096 23446 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:46.732522 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:46.724836 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:46.732465 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:46.737727 23452 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:46.758677 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.759584 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.739493 27606 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:46.739374 27272 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:08:46.760869 27606 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.762071 27728 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:46.762483 27728 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:46.762935 27606 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.763303 27728 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.764575 27728 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:46.765141 27730 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.773418 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:46.734472 27693 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:46.778298 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45875.5us
I20260709 13:08:46.721804 27712 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:08:46.755702 27731 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:46.754804 27506 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.768236 27399 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:46.756964 27552 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:46.786764 22829 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:46.767402 27730 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.788426 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.714433 27617 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.780066 27693 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:46.792726 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:46.757494 27719 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:08:46.776417 27622 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.795764 27687 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:46.802363 27705 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:46.803484 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 897) took 1348 ms. Trace:
I20260709 13:08:46.803725 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:46.804600 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:46.813089 23448 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:46.805896 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:46.807204 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:46.808148 27693 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.794766 27399 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:46.809327 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.811088 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:46.813604 22905 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:46.812482 22828 rpcz_store.cc:276] 0709 13:08:45.454909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.455094 (+   185us) service_pool.cc:224] Handling call
0709 13:08:45.455482 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:46.356767 (+901285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.803291 (+446524us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b540008b454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:46.803435 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1869296000}
I20260709 13:08:46.816020 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:46.820256 23452 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:08:46.822037 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:46.822938 27690 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:46.823400 27690 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:46.823724 27308 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:46.792790 27696 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:46.824393 27690 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.827476 27705 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.830801 27687 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.840342 22748 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:46.833272 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:46.838044 23449 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:46.795729 27731 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:46.841641 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:46.847155 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:46.847991 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:46.848982 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:46.838847 27721 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.471s	user 0.002s	sys 0.000s
I20260709 13:08:46.850915 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:46.859894 22750 raft_consensus.cc:2410] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:08:46.860772 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:46.854436 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:46.845134 27415 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.865202 22828 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:46.869921 22824 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:46.864270 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:46.846832 27752 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:08:46.854092 27751 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:46.870972 22754 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:46.850852 27496 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:46.903759 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:46.878496 27723 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.465s	user 0.004s	sys 0.000s
I20260709 13:08:46.926282 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:46.926705 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:46.928700 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 57918.1us
I20260709 13:08:46.927135 22827 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:46.932658 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:46.933748 22903 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:46.867790 27506 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:46.937090 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:46.937724 22754 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:46.941579 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1354) took 1132 ms. Trace:
I20260709 13:08:46.900111 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:46.944008 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:46.919904 27751 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:46.945932 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:46.944643 22830 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:46.935060 27698 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:46.898758 27415 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:46.941941 23451 rpcz_store.cc:276] 0709 13:08:45.808837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.808964 (+   127us) service_pool.cc:224] Handling call
0709 13:08:45.809240 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:45.811081 (+  1841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:45.811105 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:45.842000 (+ 30895us) log.cc:844] Serialized 26 byte log entry
0709 13:08:46.093931 (+251931us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:46.093970 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.094027 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:46.808920 (+714893us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:46.935831 (+126911us) raft_consensus.cc:1685] finished
0709 13:08:46.935852 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:46.941524 (+  5672us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2029,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":15496,"spinlock_wait_cycles":2848256,"thread_start_us":244144,"threads_started":1,"wal-append.queue_time_us":518056}
I20260709 13:08:46.961213 22905 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:08:46.951957 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:46.952581 27758 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:08:46.952857 27744 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:46.953600 27762 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 13:08:46.954247 27761 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:46.955164 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1353) took 1146 ms. Trace:
I20260709 13:08:46.962658 27694 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.944712 27494 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.945356 27537 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.985646 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:46.955675 27698 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.948643 27431 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.977306 23447 rpcz_store.cc:276] 0709 13:08:45.808472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.809273 (+   801us) service_pool.cc:224] Handling call
0709 13:08:45.831023 (+ 21750us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:45.897887 (+ 66864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:45.897929 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:45.909937 (+ 12008us) log.cc:844] Serialized 26 byte log entry
0709 13:08:46.137405 (+227468us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:46.137440 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.578222 (+440782us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:46.954865 (+376643us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b5400672d54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:46.954924 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:46.954972 (+    48us) raft_consensus.cc:1685] finished
0709 13:08:46.954988 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:46.955103 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":43712,"spinlock_wait_cycles":158259840,"thread_start_us":172164,"threads_started":1,"wal-append.queue_time_us":486721}
I20260709 13:08:46.983445 27768 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:46.979596 27751 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:46.994019 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:46.978250 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:46.974259 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:46.922125 27399 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:46.976456 27759 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:08:46.976650 27561 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:46.992527 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:46.995700 22903 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:46.977841 27508 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.943897 27506 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.985949 27694 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:46.984241 27622 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:47.005568 27561 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:46.992285 27698 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:46.954898 27731 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.013564 22828 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:47.027710 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:46.999755 27768 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.001578 27537 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:46.980476 27744 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.010789 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:47.035668 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:47.042060 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.043817 22828 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:47.046348 27742 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:46.990547 27494 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.680019 27742 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.680454 27742 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:47.682036 27494 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:47.048795 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:08:47.860865 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 812110us
I20260709 13:08:47.029851 27693 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:46.997165 27431 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.059751 27687 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:47.014219 27508 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.078285 27561 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:47.078410 27506 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:47.056100 27537 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:47.864153 27537 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:47.076891 27697 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:47.095857 27765 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:08:47.106360 27768 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:47.107234 27739 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.740s	user 0.002s	sys 0.000s
I20260709 13:08:47.108135 27698 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.871426 27698 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:47.109014 27415 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:47.105016 27552 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:47.110164 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:47.873342 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2388) took 1071 ms. Trace:
I20260709 13:08:47.873761 22899 rpcz_store.cc:276] 0709 13:08:46.801699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.104953 (+303254us) service_pool.cc:224] Handling call
0709 13:08:47.873249 (+768296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:47.875010 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:47.875824 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2392) took 1073 ms. Trace:
I20260709 13:08:47.113837 27351 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:47.131337 27744 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:47.876808 27351 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:47.133803 27694 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:47.573932 27625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.007s	sys 0.003s
I20260709 13:08:47.585155 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2759) took 1408 ms. Trace:
W20260709 13:08:47.652029 27779 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.584s	user 0.004s	sys 0.000s
I20260709 13:08:47.688359 27742 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.879539 22831 rpcz_store.cc:276] 0709 13:08:46.176385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.185056 (+  8671us) service_pool.cc:224] Handling call
0709 13:08:46.187251 (+  2195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:46.324113 (+136862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:46.324158 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:46.324210 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:46.324253 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.584902 (+1260649us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400231ed4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:47.584970 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.585101 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263156608}
I20260709 13:08:47.881189 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:47.881407 27742 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:47.879501 27625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.466s	user 0.007s	sys 0.003s
I20260709 13:08:47.882544 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.883766 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.884264 22831 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:47.740605 27579 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:47.884908 27415 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:47.765470 27754 consensus_queue.cc:1175] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c45e1805f049467697a7358fb566de2f" responder_term: 15 status { last_received { term: 14 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:47.771889 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:47.884996 27351 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:47.886185 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1160) took 1117 ms. Trace:
I20260709 13:08:47.782356 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:47.783744 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:47.887254 27694 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:47.791667 27309 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:47.800884 22751 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 13:08:47.807108 22750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.006s	sys 0.007s
W20260709 13:08:47.808051 22868 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.937s (SENT) [suppressed 21 similar messages]
I20260709 13:08:47.959856 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.810901 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:47.961293 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:47.961587 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:47.962414 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:47.963198 22903 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:47.963694 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1801) took 1177 ms. Trace:
I20260709 13:08:47.963972 22903 rpcz_store.cc:276] 0709 13:08:46.786262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.962181 (+1175919us) service_pool.cc:224] Handling call
0709 13:08:47.963634 (+  1453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:47.964887 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:47.815290 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2391) took 1012 ms. Trace:
I20260709 13:08:47.820942 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:47.822026 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.822737 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.966220 22822 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:47.823038 27794 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.823451 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.968004 22830 raft_consensus.cc:2410] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:08:47.968222 22826 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:47.968132 27794 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.969081 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.969345 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.969864 22826 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:47.970654 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.971374 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:47.971717 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:47.972743 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.825650 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.973205 27794 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:47.973721 22825 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:47.973781 27794 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:47.974392 27794 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.974637 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:47.975201 22825 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:47.975705 22822 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:47.976234 27794 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:47.827003 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.976547 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.976696 27794 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:47.977069 22756 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:47.977106 22822 raft_consensus.cc:2376] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15. Re-sending same reply.
I20260709 13:08:47.977766 22903 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:47.978127 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.828447 22754 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:47.980159 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.829372 22748 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:47.982120 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.983301 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:47.984011 22748 raft_consensus.cc:2368] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11.
I20260709 13:08:47.984759 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.835552 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:47.986109 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.986661 22823 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:47.962450 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.987454 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.987471 22903 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:47.844990 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.989200 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:47.869341 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2387) took 1475 ms. Trace:
I20260709 13:08:47.990085 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.852308 27788 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.798807 27696 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.078179 27696 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:47.808449 27717 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:47.863127 27508 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:47.886560 27351 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:47.863111 27431 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:47.808403 27494 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.174842 27494 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.175931 27804 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.176537 27804 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.177121 27494 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:47.860954 27573 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:48.179296 27573 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.179736 27573 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:47.852806 27712 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:48.185333 27712 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.187291 27712 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:47.086750 27705 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:47.858055 27731 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.876216 22899 rpcz_store.cc:276] 0709 13:08:46.802718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.874711 (+1071993us) service_pool.cc:224] Handling call
0709 13:08:47.875763 (+  1052us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:48.189102 27807 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:48.189910 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:48.189760 27807 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.190761 22899 raft_consensus.cc:2410] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:08:47.893038 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:48.193637 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 300653us
I20260709 13:08:48.194976 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:48.195986 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:48.196961 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:47.885074 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:47.894886 27768 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.189198 27731 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.203311 27431 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:48.203790 27731 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:47.896458 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:48.204118 27431 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:48.204459 23451 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:47.886559 22898 rpcz_store.cc:276] 0709 13:08:46.768484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.771610 (+1003126us) service_pool.cc:224] Handling call
0709 13:08:47.886124 (+114514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:48.205818 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:47.925987 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:47.928354 27797 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:47.931315 27795 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:08:47.935815 27798 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:47.941850 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51766.8us
I20260709 13:08:47.942080 27799 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:47.958334 22750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.007s	sys 0.007s
I20260709 13:08:48.210242 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1393) took 1280 ms. Trace:
I20260709 13:08:48.209206 27797 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.210574 22750 rpcz_store.cc:276] 0709 13:08:46.930158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.930373 (+   215us) service_pool.cc:224] Handling call
0709 13:08:46.943314 (+ 12941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:46.943444 (+   130us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:47.009957 (+ 66513us) log.cc:844] Serialized 14 byte log entry
0709 13:08:48.209902 (+1199945us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:48.209950 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:48.209988 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.210054 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.210166 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23162,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"thread_start_us":1258638,"threads_started":2,"wal-append.queue_time_us":100020,"wal-append.run_cpu_time_us":1211,"wal-append.run_wall_time_us":1025109}
I20260709 13:08:48.212072 27797 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:47.965513 22905 rpcz_store.cc:276] 0709 13:08:46.802478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.010609 (+208131us) service_pool.cc:224] Handling call
0709 13:08:47.815181 (+804572us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:48.212661 27799 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:47.977388 22896 raft_consensus.cc:606] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:48.213773 22896 raft_consensus.cc:614] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:47.979097 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:48.214402 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 235359us
I20260709 13:08:47.990020 22904 rpcz_store.cc:276] 0709 13:08:46.393841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.411130 (+ 17289us) service_pool.cc:224] Handling call
0709 13:08:46.411393 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:46.560750 (+149357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:46.560780 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:46.807233 (+246453us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b2c00013588. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:08:46.807586 (+   353us) log.cc:844] Serialized 26 byte log entry
0709 13:08:46.988069 (+180483us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:46.988093 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.988135 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.869034 (+880899us) raft_consensus.cc:1685] finished
0709 13:08:47.869054 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.869252 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35911,"raft.queue_time_us":99837,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":5073,"spinlock_wait_cycles":8219008,"thread_start_us":274877,"threads_started":2,"wal-append.queue_time_us":193186}
I20260709 13:08:47.809010 27606 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:48.221341 27606 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.221791 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3057) took 1188 ms. Trace:
I20260709 13:08:48.223017 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1162) took 1372 ms. Trace:
I20260709 13:08:48.222373 22747 rpcz_store.cc:276] 0709 13:08:47.033481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.812014 (+778533us) service_pool.cc:224] Handling call
0709 13:08:47.812362 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:47.812691 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:47.812719 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:47.813172 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:08:47.864802 (+ 51630us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:47.864857 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.864936 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:48.221534 (+356598us) raft_consensus.cc:1685] finished
0709 13:08:48.221557 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.221727 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1372,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":51242,"threads_started":1,"wal-append.queue_time_us":51169}
I20260709 13:08:47.992033 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:48.223601 22904 rpcz_store.cc:276] 0709 13:08:46.850981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.222319 (+1371338us) service_pool.cc:224] Handling call
0709 13:08:48.222559 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.222729 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:48.222753 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:48.222795 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:48.222828 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.222888 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.222968 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:48.224315 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.224915 22903 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:08:48.225467 22903 pending_rounds.cc:85] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 8
I20260709 13:08:48.208626 27508 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:48.225749 22903 pending_rounds.cc:107] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:08:48.226126 22903 raft_consensus.cc:2889] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:48.226814 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1806) took 1375 ms. Trace:
I20260709 13:08:48.221726 27798 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.227240 22903 rpcz_store.cc:276] 0709 13:08:46.851297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.224332 (+1373035us) service_pool.cc:224] Handling call
0709 13:08:48.224620 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.226562 (+  1942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.226735 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:47.977149 27794 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:48.228678 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.229039 27798 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:47.992046 27788 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:47.978152 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:47.997887 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.000725 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:48.001655 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:47.981241 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:47.985270 22748 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
W20260709 13:08:48.008370 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 200322us
I20260709 13:08:48.014945 27698 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.020037 27693 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:48.020984 23450 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:48.028707 27694 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:48.284439 27694 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:48.285531 27694 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.287281 22828 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:48.038789 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2386) took 1645 ms. Trace:
I20260709 13:08:48.042084 27697 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:47.885398 27579 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:48.064812 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.066934 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:48.296512 23449 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:48.068039 23448 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:08:48.068949 23452 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:08:48.069954 23453 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:48.071589 22897 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:08:48.073596 22901 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1793) took 1939 ms (client timeout 2188 ms). Trace:
I20260709 13:08:48.087423 27754 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:48.102280 27696 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:48.103490 22823 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:48.118873 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:47.885478 27415 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.136140 22825 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:48.137482 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:48.140655 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.263839 27494 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:48.154371 27809 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:48.164605 22824 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:48.165094 27537 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:48.168907 27717 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:48.178750 27804 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:48.182425 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:48.201778 27768 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:47.896201 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:48.206463 23446 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:48.206730 22756 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:48.207082 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:48.212347 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:48.217516 27799 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:48.219642 27528 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:48.216562 27370 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.992797 22829 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:48.225221 22904 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:48.232987 27794 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:48.236490 27807 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:48.238937 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58800 (request call id 41) took 1547 ms. Trace:
I20260709 13:08:48.223644 27606 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:48.240511 27793 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:48.240958 27676 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:48.241490 27797 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:48.244326 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:48.208268 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:48.262486 22822 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:48.268126 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.207873 27731 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:48.242666 27674 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:48.283200 27705 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:48.283705 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:48.306342 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:48.492633 23450 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:48.492933 23448 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:48.320430 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:48.494315 23452 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:08:48.399197 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:48.495436 22748 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:08:48.299494 22901 rpcz_store.cc:269] 0709 13:08:46.133932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.144658 (+ 10726us) service_pool.cc:224] Handling call
0709 13:08:48.073406 (+1928748us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54001b7454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:48.073540 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131054464}
I20260709 13:08:48.304941 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1800) took 1519 ms. Trace:
I20260709 13:08:48.150777 27617 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:48.321014 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:48.496110 22897 rpcz_store.cc:276] 0709 13:08:46.779483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.071050 (+1291567us) service_pool.cc:224] Handling call
0709 13:08:48.071363 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:48.299069 (+227706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.299296 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:48.496847 23449 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:48.325726 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.498670 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2393) took 1566 ms. Trace:
I20260709 13:08:48.499153 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2390) took 1697 ms. Trace:
W20260709 13:08:48.487053 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 191393us
I20260709 13:08:48.491542 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.290956 22900 rpcz_store.cc:276] 0709 13:08:46.393596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.408561 (+ 14965us) service_pool.cc:224] Handling call
0709 13:08:46.408849 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:46.591917 (+183068us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:46.598580 (+  6663us) log.cc:844] Serialized 14 byte log entry
0709 13:08:46.722430 (+123850us) log.cc:844] Serialized 14 byte log entry
0709 13:08:46.727124 (+  4694us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:46.738893 (+ 11769us) log.cc:844] Serialized 26 byte log entry
0709 13:08:46.739063 (+   170us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:46.739108 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.739187 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:46.968692 (+229505us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b4800061a58. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb79d24aa1 00007fcb79d1bb37 00007fcb79d1b9ca 00007fcb79e1326f 00007fcb79e12f37 00007fcb79e2755c 00007fcb79e27508 00007fcb79e27488 00007fcb79e27443 00007fcb79e266ec 00007fcb78d50a96
0709 13:08:48.038553 (+1069861us) raft_consensus.cc:1685] finished
0709 13:08:48.038579 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.038732 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36099,"prepare.queue_time_us":7099,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":111685,"raft.run_cpu_time_us":2994,"raft.run_wall_time_us":240968,"spinlock_wait_cycles":5875968,"thread_start_us":203979,"threads_started":2,"wal-append.queue_time_us":167112,"wal-append.run_cpu_time_us":5710,"wal-append.run_wall_time_us":1326886}
W20260709 13:08:48.502390 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 501692us
I20260709 13:08:48.302400 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1805) took 1467 ms. Trace:
I20260709 13:08:48.499259 22904 rpcz_store.cc:276] 0709 13:08:46.932127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.224743 (+1292616us) service_pool.cc:224] Handling call
0709 13:08:48.224976 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.498457 (+273481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.498620 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:48.495148 22827 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:48.495502 23447 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:48.491077 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.493501 23451 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:48.319614 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.505450 22755 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:48.503372 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:48.511512 23448 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:48.504117 22896 rpcz_store.cc:276] 0709 13:08:46.691852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.692000 (+   148us) service_pool.cc:224] Handling call
0709 13:08:48.238749 (+1546749us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540035cad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74df5853 00007fcb79dc360a 00007fcb733497d4 00007fcb73349762 00007fcb73349691 00007fcb7334960c 00007fcb733488c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:48.238855 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880042240}
W20260709 13:08:48.385159 22654 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 338) took 20029 ms (client timeout 15000 ms). Trace:
I20260709 13:08:48.510597 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:48.516714 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.499797 22899 rpcz_store.cc:276] 0709 13:08:46.802225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.191413 (+1389188us) service_pool.cc:224] Handling call
0709 13:08:48.191750 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:48.192126 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:48.192155 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:48.192720 (+   565us) log.cc:844] Serialized 52 byte log entry
0709 13:08:48.232484 (+ 39764us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:48.232548 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.239896 (+  7348us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:48.490836 (+250940us) raft_consensus.cc:1685] finished
0709 13:08:48.490858 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.499089 (+  8231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6507,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":93,"thread_start_us":39311,"threads_started":1,"wal-append.queue_time_us":130676}
I20260709 13:08:48.541007 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1911) took 1527 ms. Trace:
W20260709 13:08:48.541440 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 333213us
I20260709 13:08:48.543140 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.543983 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.544677 22753 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:48.503346 22898 rpcz_store.cc:276] 0709 13:08:46.835437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.217516 (+1382079us) service_pool.cc:224] Handling call
0709 13:08:48.217791 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.217969 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:48.271705 (+ 53736us) log.cc:844] Serialized 14 byte log entry
0709 13:08:48.301993 (+ 30288us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:48.302060 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:48.302127 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.302210 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.302338 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"thread_start_us":83118,"threads_started":2}
I20260709 13:08:48.550324 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:48.541416 22829 rpcz_store.cc:276] 0709 13:08:46.983682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.985797 (+  2115us) service_pool.cc:224] Handling call
0709 13:08:48.510472 (+1524675us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54002c9054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:48.510549 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2295161600}
W20260709 13:08:48.499984 22719 connection.cc:256] RPC call timeout handler was delayed by -1.0221635740000465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
I20260709 13:08:48.553519 22747 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:48.555755 22822 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
W20260709 13:08:48.557578 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.558910 22749 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:48.559108 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:48.560817 22755 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:48.561394 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 58117.9us
I20260709 13:08:48.566771 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:48.384814 22652 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:08:48.567359 22825 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:48.565732 22797 leader_election.cc:400] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11.
I20260709 13:08:48.384914 27687 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:48.568498 22797 leader_election.cc:403] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:08:48.569869 23446 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:48.410513 27803 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:48.493791 27793 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:48.571033 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:48.571179 22756 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:08:48.289376 27698 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.576833 23447 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:48.512588 27431 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.510448 27730 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:48.497303 27696 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:48.455813 27788 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:48.303613 27697 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.510138 27731 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.538967 27537 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.378903 27754 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.510263 27674 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.511003 27809 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:48.759454 22719 connection.cc:256] RPC call timeout handler was delayed by -1.014362320000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
I20260709 13:08:49.159358 27793 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:49.131569 27768 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:48.563503 27804 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:48.319820 27634 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.160346 27705 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11.
W20260709 13:08:49.013558 27813 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 13:08:48.923612 23475 debug-util.cc:398] Leaking SignalData structure 0x7b0800106a60 after lost signal to thread 21629
I20260709 13:08:49.161531 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1362) took 1265 ms. Trace:
I20260709 13:08:49.149876 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:48.570757 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 64125.3us
W20260709 13:08:48.990526 27694 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.006s	sys 0.007s
I20260709 13:08:48.558827 27794 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.042488 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1914) took 2024 ms. Trace:
I20260709 13:08:49.198550 22824 rpcz_store.cc:276] 0709 13:08:47.018259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.019822 (+  1563us) service_pool.cc:224] Handling call
0709 13:08:47.020169 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.538731 (+1518562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.814877 (+276146us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b540045b620. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d72dc0 00007fcb74dfea02 00007fcb74dfadd6 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:08:49.042200 (+227323us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540043b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:49.042418 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2662805760}
W20260709 13:08:48.811968 27573 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.000s
I20260709 13:08:48.903554 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2516) took 1040 ms. Trace:
W20260709 13:08:49.080035 27812 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:08:49.149763 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:49.079322 27744 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.516919 27494 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.631973 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:48.652701 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:48.559195 27606 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:48.513880 27370 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:48.805085 27309 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:48.653611 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:49.014045 27508 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:48.542236 27561 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:48.512823 22654 rpcz_store.cc:269] 0709 13:08:28.355221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.356307 (+  1086us) service_pool.cc:224] Handling call
0709 13:08:48.385016 (+20028709us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:46.367770 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:46.374088 (+  6318us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:46.374154 (+    66us) write_op.cc:435] Acquired schema lock
0709 13:08:46.374176 (+    22us) tablet.cc:662] Decoding operations
0709 13:08:46.397728 (+ 23552us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:46.426235 (+ 28507us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:46.428456 (+  2221us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:08:46.689232 (+260776us) tablet.cc:701] Row locks acquired
0709 13:08:46.719081 (+ 29849us) write_op.cc:260] PREPARE: finished
0709 13:08:46.756526 (+ 37445us) write_op.cc:270] Start()
0709 13:08:46.766050 (+  9524us) write_op.cc:276] Timestamp: P: 1783602526737092 usec, L: 0
0709 13:08:46.766099 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 13:08:46.767017 (+   918us) log.cc:844] Serialized 3964 byte log entry
0709 13:08:47.153031 (+386014us) op_driver.cc:464] REPLICATION: finished
0709 13:08:47.491457 (+338426us) write_op.cc:301] APPLY: starting
0709 13:08:47.491642 (+   185us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:47.498190 (+  6548us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:47.498293 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:47.503962 (+  5669us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:47.504288 (+   326us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:08:47.504332 (+    44us) write_op.cc:312] APPLY: finished
0709 13:08:47.504668 (+   336us) log.cc:844] Serialized 97 byte log entry
0709 13:08:47.504857 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:47.505021 (+   164us) write_op.cc:454] Released schema lock
0709 13:08:47.505074 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:47.505528 (+   454us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:47.505557 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1609064,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":337973,"apply.run_cpu_time_us":13618,"apply.run_wall_time_us":14089,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45927,"num_ops":10,"prepare.queue_time_us":191809,"prepare.run_cpu_time_us":22932,"prepare.run_wall_time_us":659265,"raft.queue_time_us":43890,"raft.run_cpu_time_us":8553,"raft.run_wall_time_us":328614,"replication_time_us":386702,"thread_start_us":726340,"threads_started":4,"wal-append.queue_time_us":300844,"wal-append.run_cpu_time_us":11812,"wal-append.run_wall_time_us":1429846}]]}
I20260709 13:08:48.926623 22899 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:08:48.578015 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:49.166770 23452 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:08:48.804469 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:48.996271 22748 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:49.160622 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:49.153023 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:49.167212 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1363) took 1271 ms. Trace:
W20260709 13:08:49.160522 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 594865us
W20260709 13:08:48.900610 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:49.168387 22868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.003s	sys 0.003s
I20260709 13:08:48.654335 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:48.843762 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:49.155691 22903 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.007s	sys 0.004s
I20260709 13:08:48.675246 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:48.897516 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:08:49.189095 23475 debug-util.cc:398] Leaking SignalData structure 0x7b080035dc80 after lost signal to thread 21632
I20260709 13:08:49.189352 23447 rpcz_store.cc:276] 0709 13:08:47.895828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.895964 (+   136us) service_pool.cc:224] Handling call
0709 13:08:49.161452 (+1265488us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.188944 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:49.196978 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.579007 22652 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 9 to 10, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 10 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:49.197600 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:49.035120 27797 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:49.200572 22751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.001s	sys 0.021s
I20260709 13:08:48.616909 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:48.780390 27693 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.384s
I20260709 13:08:49.201531 22754 rpcz_store.cc:276] 0709 13:08:47.863284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.541454 (+678170us) service_pool.cc:224] Handling call
0709 13:08:48.541699 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.903223 (+361524us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:48.903257 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:48.903297 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:48.903326 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.903388 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.903503 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":361183,"threads_started":1}
I20260709 13:08:49.207019 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:48.759835 27579 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.162820 27697 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.155131 27415 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:49.198285 27694 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.008s	sys 0.007s
I20260709 13:08:49.189888 27431 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.217942 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.131377 27676 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:08:48.613425 27687 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:48.613657 27625 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.187866 27634 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.320946 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1809) took 1075 ms. Trace:
I20260709 13:08:49.189672 27696 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.222468 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:49.377815 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1168) took 1073 ms. Trace:
I20260709 13:08:49.342027 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38478 (request call id 3064) took 1115 ms. Trace:
W20260709 13:08:49.340606 22751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.007s	sys 0.027s
I20260709 13:08:49.380234 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1167) took 1079 ms. Trace:
I20260709 13:08:49.341881 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.377352 22896 rpcz_store.cc:276] 0709 13:08:48.245961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.573186 (+327225us) service_pool.cc:224] Handling call
0709 13:08:49.320883 (+747697us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:49.381937 23475 debug-util.cc:398] Leaking SignalData structure 0x7b080035dd40 after lost signal to thread 22688
I20260709 13:08:49.341005 22828 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:49.378564 22904 rpcz_store.cc:276] 0709 13:08:48.304519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.804219 (+499700us) service_pool.cc:224] Handling call
0709 13:08:49.377751 (+573532us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.355324 22747 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:49.381711 22900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:49.383312 22900 raft_consensus.cc:606] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:49.383424 23475 debug-util.cc:398] Leaking SignalData structure 0x7b0800365120 after lost signal to thread 22780
I20260709 13:08:49.383663 22900 raft_consensus.cc:614] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:49.381742 22827 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:49.381204 22754 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:49.379304 22748 rpcz_store.cc:276] 0709 13:08:48.226849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.324178 (+ 97329us) service_pool.cc:224] Handling call
0709 13:08:49.341937 (+1017759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.380470 22797 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:49.383651 23449 raft_consensus.cc:2410] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 11 LEARNER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 13:08:49.334280 27617 consensus_meta.cc:287] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.824s	user 0.010s	sys 0.000s
I20260709 13:08:49.380802 22899 pending_rounds.cc:85] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 9
W20260709 13:08:49.368727 22903 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.009s	sys 0.008s
I20260709 13:08:49.386691 22905 rpcz_store.cc:276] 0709 13:08:48.300858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.577743 (+276885us) service_pool.cc:224] Handling call
0709 13:08:49.380185 (+802442us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:49.204648 27528 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.005s	sys 0.010s
I20260709 13:08:49.387159 22823 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:49.390017 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:49.384822 23475 debug-util.cc:398] Leaking SignalData structure 0x7b080003d0e0 after lost signal to thread 22855
I20260709 13:08:49.393002 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:49.398895 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:49.399137 27537 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:49.399495 23475 debug-util.cc:398] Leaking SignalData structure 0x7b08004028a0 after lost signal to thread 22929
I20260709 13:08:49.399760 27506 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:49.395823 27705 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:49.400774 23475 debug-util.cc:398] Leaking SignalData structure 0x7b0800170340 after lost signal to thread 23478
I20260709 13:08:49.400322 27768 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:49.387403 22868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.006s	sys 0.006s
I20260709 13:08:49.401227 27768 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.400170 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:49.401139 23475 debug-util.cc:398] Leaking SignalData structure 0x7b0800147940 after lost signal to thread 23479
W20260709 13:08:49.401611 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 839905us
I20260709 13:08:49.389858 22899 pending_rounds.cc:107] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.10
I20260709 13:08:49.401676 27768 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:49.404071 22899 raft_consensus.cc:2917] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Aborting config change with OpId 10.10: Aborted: Op aborted by new leader
I20260709 13:08:49.404763 22899 raft_consensus.cc:2889] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:49.405905 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1810) took 1159 ms. Trace:
I20260709 13:08:49.406273 22899 rpcz_store.cc:276] 0709 13:08:48.246253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.925860 (+679607us) service_pool.cc:224] Handling call
0709 13:08:48.926223 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.405659 (+479436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.405856 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.391116 22830 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:49.400178 27506 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:49.407904 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 369609us
I20260709 13:08:49.408702 27506 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:49.342094 23446 rpcz_store.cc:276] 0709 13:08:47.896123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.922214 (+ 26091us) service_pool.cc:224] Handling call
0709 13:08:49.167158 (+1244944us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.409349 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:49.393772 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 496316us
I20260709 13:08:49.412253 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:49.394591 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1169) took 1062 ms. Trace:
I20260709 13:08:49.414423 22897 rpcz_store.cc:276] 0709 13:08:48.332587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.570765 (+238178us) service_pool.cc:224] Handling call
0709 13:08:49.394522 (+823757us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.335143 27809 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.420022 22896 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:49.424695 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1811) took 1161 ms. Trace:
I20260709 13:08:49.420238 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:49.421277 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:49.426998 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:49.301905 27822 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.222280 27309 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.205614 27573 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:49.378223 27717 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.458860 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1397) took 1634 ms. Trace:
I20260709 13:08:49.206050 27794 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.217694 27606 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.218014 27674 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.475685 22752 rpcz_store.cc:276] 0709 13:08:47.824424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.827443 (+  3019us) service_pool.cc:224] Handling call
0709 13:08:47.827724 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.320086 (+492362us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:48.756939 (+436853us) log.cc:844] Serialized 14 byte log entry
0709 13:08:49.409882 (+652943us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:49.409936 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:49.409982 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.458667 (+ 48685us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b5400045454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:49.458707 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.458811 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4121,"prepare.queue_time_us":1104218,"spinlock_wait_cycles":1131822848,"thread_start_us":1009362,"threads_started":3,"wal-append.queue_time_us":570678}
I20260709 13:08:49.482100 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:49.419269 27528 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.005s	sys 0.011s
I20260709 13:08:49.461830 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1340) took 1195 ms. Trace:
I20260709 13:08:49.425136 22896 rpcz_store.cc:276] 0709 13:08:48.263099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.382064 (+1118965us) service_pool.cc:224] Handling call
0709 13:08:49.412018 (+ 29954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.424414 (+ 12396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.424635 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:49.468508 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 56773.3us
W20260709 13:08:49.456086 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53352.3us
W20260709 13:08:49.469007 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 67778.7us
I20260709 13:08:49.397166 27820 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.218536 27744 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:49.478341 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66181us
I20260709 13:08:49.410228 27693 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:49.430202 27537 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:49.410710 27742 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.369503 27625 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.488999 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:49.479806 22652 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 10 to 11, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 11 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.467810 27309 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:49.496675 22902 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:49.389572 27508 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.511916 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:49.491557 22797 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:49.487790 23453 rpcz_store.cc:276] 0709 13:08:48.266526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.322735 (+ 56209us) service_pool.cc:224] Handling call
0709 13:08:48.323019 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.323203 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:08:48.323459 (+   256us) log.cc:844] Serialized 14 byte log entry
0709 13:08:49.461296 (+1137837us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x7b18001d0ec0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb749a34bc 00007fcb71cf148b 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0b455 00007fcb74df3060 00007fcb74e05716 00007fcb74e16287
0709 13:08:49.461426 (+   130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:49.461478 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:49.461548 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.461634 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.461765 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":379,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":13975,"raft.queue_time_us":61503,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":60800,"spinlock_wait_cycles":195634432,"thread_start_us":447319,"threads_started":2,"wal-append.queue_time_us":979446}
I20260709 13:08:49.467257 27494 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.368319 27561 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.518826 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:49.483259 27694 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:49.520845 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:49.503129 23452 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:49.467454 27370 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.494590 27693 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.514959 27698 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.515347 27731 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.515852 22896 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:08:49.492434 27606 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:49.514982 22899 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:49.535490 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:49.535784 22899 raft_consensus.cc:606] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:49.397224 27797 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.490778 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:49.482159 27817 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:49.525210 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:49.483372 27818 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.481225 27799 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:08:49.494642 27537 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.555953 23452 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:49.509766 27676 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:49.520102 22652 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.511574 27804 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:49.510032 27742 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:49.544054 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 77363.1us
I20260709 13:08:49.519829 27788 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:49.556712 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66033.4us
I20260709 13:08:49.562320 22652 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 7 to 10, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 10 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.525440 27693 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:49.560549 23449 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:49.540181 27819 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:49.519623 27705 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.564765 27705 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:49.567909 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67613.2us
I20260709 13:08:49.563784 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:49.569224 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:49.554862 27698 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:49.542300 27579 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:49.556138 27731 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:49.595543 22722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.003s	sys 0.008s
I20260709 13:08:49.584151 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.536187 27617 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:08:49.598984 22898 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:49.561175 27676 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.599467 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:49.597780 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 175598us
I20260709 13:08:49.562106 27494 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:08:49.600066 22722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.003s	sys 0.010s
I20260709 13:08:49.577983 22652 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 9 to 10, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 10 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.557307 27754 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.572299 27804 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:49.579411 27717 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:49.559135 27537 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:49.605176 22752 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:49.600651 27819 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:49.599382 27825 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:08:49.615958 27676 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:49.617959 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:49.628007 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:49.640322 22756 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:49.630584 27852 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:49.613987 27848 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:08:49.584947 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 68033.9us
I20260709 13:08:49.615526 27494 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:49.655956 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:49.660187 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:49.661556 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:49.602773 27579 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.628553 27431 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.601658 27788 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:49.604669 27731 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:49.670060 27829 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 13:08:49.658994 27494 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:49.656121 27848 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:49.712944 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 221467us
I20260709 13:08:49.656019 27634 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.624106 27846 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:08:49.681674 27853 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:49.165772 27803 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:49.723908 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:49.467674 27822 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:49.690409 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:49.555310 27818 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.640640 27819 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.662750 27561 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.670493 27849 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:49.735824 22797 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:49.489698 27820 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.664845 27754 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:08:49.730605 27848 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:49.670903 27696 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.739521 22652 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 7 to 9, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 9 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:49.741833 27847 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:08:49.743111 27731 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.743343 27803 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.743813 22750 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:49.812045 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.744096 27768 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.746256 27804 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.748333 27508 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.679623 27625 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.777506 27698 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:08:49.752110 27822 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.753454 27705 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.755381 22756 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:49.840394 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.756451 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1166) took 1462 ms. Trace:
I20260709 13:08:49.841483 22898 rpcz_store.cc:276] 0709 13:08:48.294247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.578256 (+284009us) service_pool.cc:224] Handling call
0709 13:08:49.756299 (+1178043us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540035cfd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:49.756399 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":925893888}
I20260709 13:08:49.757014 27841 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:49.758203 27843 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:49.759249 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:49.681308 27573 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:49.749927 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 59624.8us
I20260709 13:08:49.763937 22797 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:49.853430 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 89563.6us
I20260709 13:08:49.854103 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:49.555652 27817 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.673520 27537 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:49.737353 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72900.1us
W20260709 13:08:49.759382 27793 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.599s	user 0.005s	sys 0.001s
I20260709 13:08:49.779908 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2526) took 1851 ms. Trace:
I20260709 13:08:49.781817 27579 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.782611 27820 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.783763 27788 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.786216 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2397) took 1849 ms. Trace:
I20260709 13:08:49.786463 27309 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:49.786887 27819 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.762754 27853 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:49.762564 27852 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:49.887194 27852 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.887290 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:49.887686 27852 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:49.888301 27309 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.888922 27309 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:49.860236 27822 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:49.889497 27573 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:49.889418 27309 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.890169 27573 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:49.891211 27537 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.891729 27309 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.892237 27573 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.892743 22750 raft_consensus.cc:2410] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:08:49.892863 27573 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:49.737840 27794 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.758222 27744 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.794381 27818 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.684165 27694 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:08:49.795953 27742 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.796314 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3067) took 1529 ms. Trace:
I20260709 13:08:49.894338 22753 rpcz_store.cc:276] 0709 13:08:48.266929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.161030 (+894101us) service_pool.cc:224] Handling call
0709 13:08:49.161267 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.161457 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:08:49.296247 (+134790us) log.cc:844] Serialized 14 byte log entry
0709 13:08:49.790694 (+494447us) spinlock_profiling.cc:243] Waited 9.12 ms on lock 0x7b1800479d40. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb787f33c7 00007fcb74e059a7 00007fcb74e150bf 00007fcb74e15038
0709 13:08:49.790883 (+   189us) log.cc:844] Serialized 14 byte log entry
0709 13:08:49.795984 (+  5101us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:49.796035 (+    51us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:08:49.796081 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.796157 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.796264 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":267352,"prepare.queue_time_us":639426,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":2600,"raft.queue_time_us":6007,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":20966784,"thread_start_us":322711,"threads_started":2,"wal-append.queue_time_us":389100}
I20260709 13:08:49.896385 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.796653 27823 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.892318 27309 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:49.897585 27823 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.898399 27309 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:49.797029 27828 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:49.899113 27817 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.797432 22747 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:08:49.899380 27309 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:49.900306 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.901489 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.902150 22747 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:49.902845 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.797482 27697 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:49.905443 27697 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:49.906217 27768 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.906316 27697 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.907509 27697 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:49.797993 27506 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.800678 27494 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.910744 27506 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.910959 27494 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.801458 27561 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.911381 27693 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.861542 27698 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:49.735937 27431 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:49.802855 27797 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.764938 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:49.912479 27431 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.912997 27431 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:49.913187 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 151336us
I20260709 13:08:49.914880 27676 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.914947 27606 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:49.915951 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:49.916494 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:49.808985 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 145081us
I20260709 13:08:49.916916 27676 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.917668 22825 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:49.918671 27494 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:08:49.920004 22825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 917) took 1351 ms (client timeout 1574 ms). Trace:
I20260709 13:08:49.920104 27754 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:49.762950 27849 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:49.920527 22825 rpcz_store.cc:269] 0709 13:08:48.568264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.568369 (+   105us) service_pool.cc:224] Handling call
0709 13:08:49.919798 (+1351429us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54001f9ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:49.919941 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1642129152}
I20260709 13:08:49.826622 27308 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:49.663189 27834 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:49.897145 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.777220 27687 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.922103 27308 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:49.861536 27793 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.923306 27308 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.924098 27793 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:49.924499 27308 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:49.924710 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:49.925520 22905 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:49.925935 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:49.851039 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.928592 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:49.855129 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:49.855796 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 97902.8us
I20260709 13:08:49.899324 27828 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:49.930094 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:49.930781 27828 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:49.931238 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:49.932090 27828 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:49.932324 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:49.856976 22751 rpcz_store.cc:276] 0709 13:08:47.928053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.550681 (+622628us) service_pool.cc:224] Handling call
0709 13:08:48.551003 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:49.380068 (+829065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:49.380096 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:49.380518 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 13:08:49.658621 (+278103us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:49.659389 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.659578 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:49.779663 (+120085us) raft_consensus.cc:1685] finished
0709 13:08:49.779682 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.779835 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59072,"raft.queue_time_us":835140,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":30544,"thread_start_us":1006962,"threads_started":2,"wal-append.queue_time_us":303717}
I20260709 13:08:49.933506 27687 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:49.857671 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:49.933876 27687 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.934196 27843 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:49.934285 27687 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:49.934645 27843 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.935036 27843 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:49.935679 27431 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.936132 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:49.849946 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:49.937131 27803 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:49.920835 27494 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:49.937577 27803 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:49.937620 22898 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:08:49.938045 27803 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:49.852028 27841 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:49.938815 27841 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:49.939621 27841 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:49.940582 27841 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:49.941955 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:49.943624 27856 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:49.943933 27853 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:49.925810 23444 raft_consensus.cc:2410] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:08:49.838955 27717 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:49.945240 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 90169us
I20260709 13:08:49.856655 27508 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.945830 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:49.946280 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1365) took 1764 ms. Trace:
I20260709 13:08:49.946233 27508 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.946905 23444 rpcz_store.cc:276] 0709 13:08:48.182011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.182159 (+   148us) service_pool.cc:224] Handling call
0709 13:08:49.945281 (+1763122us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54005e1ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:49.946216 (+   935us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3282225920}
I20260709 13:08:49.911706 27561 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:49.948040 27809 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:49.948484 27508 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:49.859071 27705 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:49.921543 27849 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:49.823557 22652 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 7 to 9, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 9 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.825620 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:49.825925 22902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1799) took 3056 ms (client timeout 2323 ms). Trace:
I20260709 13:08:49.820724 27696 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.926047 22905 raft_consensus.cc:606] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:49.912395 27698 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:49.861922 22903 rpcz_store.cc:276] 0709 13:08:47.936682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.228018 (+291336us) service_pool.cc:224] Handling call
0709 13:08:48.230903 (+  2885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.562735 (+331832us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:49.391055 (+828320us) log.cc:844] Serialized 14 byte log entry
0709 13:08:49.693021 (+301966us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:49.693081 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:49.693144 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.785991 (+ 92847us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54000d0a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:49.786036 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.786159 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10030,"prepare.queue_time_us":248317,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":21826,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":6933,"spinlock_wait_cycles":762623744,"thread_start_us":1115847,"threads_started":3,"wal-append.queue_time_us":199419}
I20260709 13:08:49.949565 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.953233 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:49.953678 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.953954 22751 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:49.981065 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:49.954882 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:49.962363 27573 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:49.972981 22902 rpcz_store.cc:269] 0709 13:08:46.769049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.782098 (+1013049us) service_pool.cc:224] Handling call
0709 13:08:49.825681 (+2043583us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540034b7d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:49.825842 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3699085696}
I20260709 13:08:49.976130 22905 raft_consensus.cc:614] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:49.978366 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.987567 27817 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:49.988353 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.986357 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:49.917044 27606 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.988759 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:49.988484 27693 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:49.990818 27817 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:49.989997 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:49.991748 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:49.992671 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:49.993193 22752 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:08:49.993543 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:49.994259 27818 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:49.995045 27818 raft_consensus.cc:2764] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:08:49.995468 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:49.996443 27818 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:49.996683 27822 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:49.996940 27818 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:49.997462 27818 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:49.997646 22755 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:49.998601 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.999099 22755 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:08:49.999848 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:49.997193 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:50.001672 27822 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:50.002025 27822 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:50.002385 27822 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:50.004547 27838 raft_consensus.cc:1064] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
W20260709 13:08:50.005082 27838 raft_consensus.cc:1068] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Unable to promote non-voter c45e1805f049467697a7358fb566de2f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:50.005282 27817 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:50.007335 27817 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:50.009476 27817 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:50.010717 27573 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:50.011546 22752 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:50.012145 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:50.014462 27817 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:50.014842 27817 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:49.945546 27694 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:50.015641 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.037142 22748 raft_consensus.cc:2393] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 6: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:50.037927 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:50.038640 22747 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:50.037525 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.043143 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:50.018261 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:50.045401 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.018635 22652 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 8 to 10, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 10 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.937656 27431 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:50.047029 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 189455us
I20260709 13:08:50.044100 27843 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:50.062493 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:50.022056 22751 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:50.015419 27698 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:50.058014 27804 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:50.082070 27804 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:50.082465 27804 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:50.082902 27561 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.059140 27793 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:50.063704 27844 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:08:50.088364 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.090067 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.066324 23444 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:08:50.077657 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.077872 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.091570 22749 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:50.015219 27817 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:50.065361 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:50.105042 22750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 2528) took 2156 ms (client timeout 2373 ms). Trace:
I20260709 13:08:50.102993 23449 raft_consensus.cc:2368] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16.
I20260709 13:08:50.105592 23448 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:08:50.106266 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43851.5us
W20260709 13:08:50.105448 22750 rpcz_store.cc:269] 0709 13:08:47.934689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.212154 (+277465us) service_pool.cc:224] Handling call
0709 13:08:50.090776 (+1878622us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002684d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:50.090949 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2872331648}
W20260709 13:08:50.062384 22719 connection.cc:256] RPC call timeout handler was delayed by -1.5802355430000716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2323s for a response
I20260709 13:08:50.103822 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.112645 22828 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:50.047540 27838 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:50.114305 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.102437 22827 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:50.116858 22823 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:50.118552 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.119637 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.120368 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:50.121394 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 164491us
I20260709 13:08:50.121781 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.122473 22830 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:50.121699 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.123792 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.124027 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.124396 22900 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:50.121217 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.124846 22903 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:50.125613 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:49.945758 27717 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.015722 27696 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:49.975216 27856 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.130s
I20260709 13:08:50.128408 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:50.134052 23444 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:50.134486 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.127764 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.128973 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.135473 22903 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:50.129561 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.070792 27843 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.064486 27848 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.101660 27793 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:50.131206 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.085340 27831 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
W20260709 13:08:50.132511 23416 connection.cc:256] RPC call timeout handler was delayed by -1.421592235999924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 13:08:50.145363 22823 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:50.147001 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:50.121102 27674 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:50.148092 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.127995 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.151340 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:50.144804 23448 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:50.101608 27820 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:50.144726 22652 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:50.148720 22827 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:50.155798 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.157059 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.157773 23450 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:50.154471 22747 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:50.158355 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.158977 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:50.143941 27849 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.158357 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.165678 23451 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:50.160877 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.073050 27821 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 13:08:50.161952 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.141054 27687 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:50.162827 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.171119 23445 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:50.163676 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.162520 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:50.168175 22652 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 11 to 12, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 12 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:50.169149 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.173381 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.174052 22830 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:50.146235 27793 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:50.175537 22903 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:50.172173 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.176462 22896 raft_consensus.cc:2410] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:08:50.177296 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.200966 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.180163 27687 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.180711 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.181483 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.182117 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.182513 22652 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 6 to 7, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 7 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:50.077463 27833 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:08:50.148062 27561 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:50.172326 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:50.201386 22827 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:50.204460 22719 leader_election.cc:400] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16.
I20260709 13:08:50.207087 23453 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:50.209419 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.233197 22719 leader_election.cc:403] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:50.209424 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3073) took 1015 ms. Trace:
I20260709 13:08:50.208205 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.209492 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.237993 27843 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:50.234100 22896 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:50.172890 27674 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:50.233909 22749 rpcz_store.cc:276] 0709 13:08:49.193564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.197432 (+  3868us) service_pool.cc:224] Handling call
0709 13:08:50.091378 (+893946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.103061 (+ 11683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.209233 (+106172us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54002684d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:50.209372 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055010688}
I20260709 13:08:50.245564 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.253760 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.164613 27820 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.254931 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.256404 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.258908 23447 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:50.261000 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.207872 27696 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.262876 22823 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 9: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:50.263554 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.264418 22828 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:50.264540 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.265164 22824 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:50.246579 27687 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:08:50.199949 27822 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:50.268786 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.271361 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:08:50.269580 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 97309.3us
I20260709 13:08:50.271626 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.272713 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:50.272846 22748 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:08:50.274668 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.275365 27820 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.275524 27848 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.275799 27844 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:50.276463 27844 consensus_queue.cc:1243] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:08:50.277429 27849 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.279452 27822 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.281699 27561 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:50.281994 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.282670 22831 raft_consensus.cc:2376] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11. Re-sending same reply.
I20260709 13:08:50.284087 23450 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:50.270431 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.286798 22747 raft_consensus.cc:2410] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:50.282195 27561 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.221961 27415 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:08:50.291965 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.292551 22824 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:50.293466 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.271561 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:50.269820 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.296757 22827 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:50.298223 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:50.302552 23444 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:50.300027 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.292043 27561 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:50.284873 27803 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:50.271970 27871 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.318462 22756 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:50.311177 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.312958 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:50.340054 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.340759 22748 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
W20260709 13:08:50.317442 27832 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.614s	user 0.013s	sys 0.000s
I20260709 13:08:50.342996 22899 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:08:50.343753 22831 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:08:50.344628 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.350207 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.350922 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.319288 27803 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.310959 27730 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:50.360383 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.331326 27820 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:50.309867 27863 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:50.297475 27818 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:50.378152 22831 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:50.347368 27634 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.388002 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.388638 22827 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:50.378624 22831 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:50.363708 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:50.346783 27849 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:50.319742 27698 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.346073 27561 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.333849 27817 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:50.333472 27871 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:50.366055 27674 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.398041 27634 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.423306 27634 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.425761 22753 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:50.424988 22824 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:50.428543 22830 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:50.398545 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.413518 22825 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:50.417248 22750 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:50.417891 22826 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:08:50.434935 22823 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:50.428490 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:50.419147 27852 consensus_meta.cc:287] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.002s
I20260709 13:08:50.479056 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.487186 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.479367 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:50.405488 27869 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:08:50.540048 23447 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:50.487517 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.486900 23449 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:50.488032 27821 raft_consensus.cc:1064] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:08:50.483690 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:50.478992 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.440162 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.544420 22827 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:50.544590 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 60970.4us
I20260709 13:08:50.549623 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:50.552711 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:50.535470 22831 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.553743 22831 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:50.557466 22903 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:08:50.489876 27579 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:50.491915 27823 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:50.535408 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 133843us
I20260709 13:08:50.593547 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.594296 22896 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:50.414968 27561 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.536020 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.538324 27852 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.418133 27871 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.538923 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:50.379998 27820 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:50.613570 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:50.416505 27817 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:50.448688 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.619617 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:50.437388 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 73746.7us
I20260709 13:08:50.591322 22753 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:50.535945 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.622395 22824 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:50.572897 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:50.623471 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50613.5us
I20260709 13:08:50.624356 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:50.426635 27730 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:50.541582 27506 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:50.411909 27687 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16.
I20260709 13:08:50.603603 23444 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:50.604121 22748 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:50.413375 27698 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:50.626646 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.547494 27494 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:08:50.562188 27705 raft_consensus.cc:3055] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:50.626358 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.570757 27878 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:50.644047 22826 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:08:50.598603 27822 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.589267 27831 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:50.481838 27804 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.627036 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:50.626386 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:50.544001 27849 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:50.494763 27793 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.556022 27803 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:50.635470 27871 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:50.638240 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.640264 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:50.587101 27579 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:50.634716 22829 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:50.644860 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.698720 22826 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:50.562724 27821 raft_consensus.cc:1068] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Unable to promote non-voter c45e1805f049467697a7358fb566de2f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:50.645620 22901 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:50.651436 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.655094 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.594965 27841 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:08:50.590678 27838 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:08:50.628525 27506 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.635865 27844 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:08:50.687228 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 60261us
I20260709 13:08:50.712064 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:50.658129 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:50.699702 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.641837 27693 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:50.701222 27820 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.643765 27705 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:50.701977 27730 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.726570 22828 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:50.656194 27694 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
W20260709 13:08:50.706185 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 79854.6us
I20260709 13:08:50.729938 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:50.678923 27891 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:08:50.707867 27881 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:08:50.733299 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:50.734059 22753 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:50.658674 27831 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:50.715255 22903 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:50.722302 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:50.723284 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.713335 27506 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:50.709689 27862 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:50.734783 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:50.735703 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:50.750983 22829 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:50.723582 22824 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:50.722249 27309 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:50.657125 27822 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.755054 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:50.711632 27844 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:50.760788 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.740067 27370 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:08:50.752727 27862 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:50.769196 27855 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:50.770511 27849 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.771760 27826 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:50.773468 27705 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:50.711227 27809 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:08:50.777267 27852 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:50.778553 27818 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.753931 27309 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.738618 27819 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:50.784914 27804 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:50.785429 27573 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:50.785851 27625 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:50.786283 27793 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:50.807369 27862 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:50.758769 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:50.803634 22827 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:50.806234 22824 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:50.883338 22830 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
W20260709 13:08:50.883602 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.806699 22896 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:50.736685 27693 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:50.742069 27831 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:50.809610 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 178811us
I20260709 13:08:50.884693 22904 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:50.887123 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 79262.6us
I20260709 13:08:50.804044 27844 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:50.888115 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.891359 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:50.833724 27309 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:50.896771 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.902359 22824 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:50.897215 27693 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.899439 22827 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:50.885047 27819 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:50.836086 27625 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:50.891323 27887 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:08:50.866606 27370 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:08:50.906397 27528 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.029s	sys 0.064s
W20260709 13:08:50.910501 27528 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.029s	sys 0.064s
I20260709 13:08:50.913450 27370 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:50.901804 27705 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:50.914547 27370 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.916034 27370 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:50.907932 27674 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:50.917936 27674 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:50.918419 27674 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:50.909468 27625 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:50.924141 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 165469us
I20260709 13:08:50.924944 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:50.926882 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:50.927096 22831 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:08:50.927656 22831 pending_rounds.cc:85] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 9
I20260709 13:08:50.927903 22831 pending_rounds.cc:107] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:08:50.928133 22831 raft_consensus.cc:2889] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:50.932432 27494 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:50.932878 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:50.933699 27494 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:50.935357 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:50.937899 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.938661 22831 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:50.939680 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:50.941109 22868 leader_election.cc:336] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.629s (SENT)
I20260709 13:08:50.941533 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:50.946797 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.948376 22824 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:50.948777 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.959744 27804 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.960412 27804 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:50.962001 27822 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:50.962936 27804 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:50.963307 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.963918 22831 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:50.964701 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.966293 22831 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:50.967057 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.968400 22831 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:08:50.966305 22827 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:50.965372 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:50.969589 22824 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:08:50.970675 22824 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:50.972718 27862 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:50.975524 22831 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:08:50.973664 22827 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:08:50.977058 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:50.986654 22826 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
W20260709 13:08:50.984953 22823 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.002s
I20260709 13:08:50.988025 22823 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
W20260709 13:08:50.936447 27606 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:08:50.990067 22829 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:08:50.937427 27846 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:50.996271 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:50.948974 27819 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:50.949136 27820 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:51.001152 22823 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:08:51.004472 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.006871 27819 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:51.007885 27819 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.017072 27625 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.027735 22753 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
W20260709 13:08:51.028354 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 85734.9us
I20260709 13:08:51.028828 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:51.034823 23446 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:51.023270 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:51.053655 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:51.054376 22750 raft_consensus.cc:2368] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12.
I20260709 13:08:51.055140 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:51.055734 22749 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:51.056614 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.031330 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:51.062965 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:51.063215 22749 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:51.063060 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:51.064258 22756 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:51.069645 22826 raft_consensus.cc:1217] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Deduplicated request from leader. Original: 13.9->[14.10-14.10]   Dedup: 14.10->[]
I20260709 13:08:51.070369 22826 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:08:51.073933 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.075338 27528 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:51.091668 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:08:51.092420 22797 connection.cc:256] RPC call timeout handler was delayed by -1.0624797750000425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23730000000000004s for a response
I20260709 13:08:51.092602 22747 raft_consensus.cc:1217] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 10.9->[11.10-11.10]   Dedup: 11.10->[]
I20260709 13:08:51.093065 22747 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:08:51.092994 22750 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:51.093506 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.094751 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.096050 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.097944 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.099529 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.099570 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.101369 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.104061 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.104431 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.105126 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.089638 27823 raft_consensus.cc:2764] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:08:51.108170 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.111297 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:51.111882 22823 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:51.114483 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.065191 27370 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:08:51.098191 27697 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:51.116945 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:51.096966 27696 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:51.103649 27698 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:51.098838 27717 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:51.106667 27822 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:51.116887 27833 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:51.129875 27901 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:08:51.130211 27892 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:51.153106 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:51.130524 27309 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.131968 27625 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:51.131922 27871 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:51.106849 27841 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:51.136108 22823 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:51.137219 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:51.113385 27826 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:08:51.128223 27697 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:51.105182 27866 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:51.113655 27852 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:51.128926 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.144588 22900 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:08:51.113694 27893 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:51.114962 27799 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:51.166661 22900 pending_rounds.cc:85] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 9
I20260709 13:08:51.169689 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:51.170421 22900 pending_rounds.cc:107] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:08:51.170742 22900 raft_consensus.cc:2889] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:51.171852 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.127944 27573 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:08:51.164999 27894 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:08:51.177597 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.172660 27799 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:51.179688 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73510.6us
I20260709 13:08:51.167454 27893 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:51.180578 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.167963 27697 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:51.193639 27370 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:51.174405 27833 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:51.193987 27694 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:51.141075 27674 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.150632 27831 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.181233 27799 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:51.186740 27895 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:08:51.187027 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.189527 27841 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:51.202078 22900 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:51.201179 23444 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:08:51.183161 27893 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:51.179863 27822 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:51.153636 27506 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.209975 27852 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:51.184782 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.222960 27831 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:51.224390 27674 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:51.229283 27844 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.232120 27893 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.274279 27893 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:51.274892 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:51.236354 27308 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:51.276974 27804 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:51.277281 27793 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:51.277899 27793 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:51.278218 27804 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:51.278599 27804 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:51.278771 27793 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.279335 27804 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.280447 27804 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:51.283239 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.284356 27844 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:51.287583 22751 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:51.290380 22748 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:08:51.293744 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 192470us
I20260709 13:08:51.264770 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:51.295889 23448 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:51.297195 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.298632 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:51.299322 22750 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:51.264114 23449 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:51.271449 22830 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:51.263279 23445 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
W20260709 13:08:51.239567 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54870.1us
I20260709 13:08:51.302619 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.270885 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.307410 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:51.308109 22901 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:51.310544 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:51.291680 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:51.322861 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.323276 22749 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:51.324208 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:08:51.324703 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53945.4us
I20260709 13:08:51.325618 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.322894 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.327718 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.309422 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:51.330615 22904 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:51.330797 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.331954 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.333204 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.332522 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:51.335808 22756 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:51.277103 27308 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:51.337661 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.340255 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:51.338872 23450 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:51.267395 27900 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 13:08:51.347246 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:51.348826 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.349192 23450 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:51.340343 23416 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.345221 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:51.338696 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.263003 27528 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:51.237731 27822 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:51.248996 27892 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:51.238945 27506 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:51.241588 27841 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:51.353412 22797 leader_election.cc:400] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12.
I20260709 13:08:51.265270 27309 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:51.270551 27852 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:51.349798 27696 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:51.334712 27893 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:51.329698 27817 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:51.335197 27820 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:51.336318 27537 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:51.335196 27844 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:51.335822 27809 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:51.359349 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:51.335922 27705 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:51.339941 27856 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:51.253994 27698 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:08:51.237381 27846 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 13:08:51.350911 27431 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:51.363173 22797 leader_election.cc:403] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:08:51.350147 27838 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:51.266754 27799 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.352905 27694 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:51.354449 27842 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:51.245105 27697 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:51.358788 27902 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.215s	user 0.002s	sys 0.000s
I20260709 13:08:51.336195 27833 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:08:51.335539 27831 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:08:51.374984 27856 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:51.369935 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.344362 27818 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:08:51.370550 27823 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:51.243486 27579 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.362869 27841 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:51.374536 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 937) took 1083 ms. Trace:
I20260709 13:08:51.386781 27826 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:51.380013 22797 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.371055 27820 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:51.371928 27817 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:51.388434 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 78006.4us
I20260709 13:08:51.406401 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.372483 27844 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:51.391451 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.387979 27537 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:51.344720 27863 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:08:51.399946 22901 raft_consensus.cc:2410] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:08:51.401715 22828 rpcz_store.cc:276] 0709 13:08:50.291552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.292283 (+   731us) service_pool.cc:224] Handling call
0709 13:08:50.292503 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.727106 (+434603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.374260 (+647154us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54000adcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:51.374450 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997690624}
I20260709 13:08:51.416268 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.367203 27893 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:51.378981 27431 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:51.390246 27842 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:51.400871 27856 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:51.413262 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.431524 22830 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:08:51.431803 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:51.432618 22828 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:08:51.432612 22826 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:51.434597 22827 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:51.434742 22825 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:51.411422 27820 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:51.439972 22826 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:51.426087 27924 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:51.420584 27893 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:51.426532 27799 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:51.398084 27822 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.404028 27826 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:51.422392 27579 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:51.418902 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 44873us
I20260709 13:08:51.437362 22830 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:51.437649 22828 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:51.403106 27841 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:51.438453 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:51.386196 27308 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.413619 27694 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:51.396291 27494 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:08:51.415495 27606 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:51.441349 22831 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:51.413153 27844 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:51.457728 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:51.423327 27431 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:51.483089 23450 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:51.484006 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:51.429304 27842 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:51.488596 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72412.8us
I20260709 13:08:51.439263 27705 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:51.495646 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:51.434777 27846 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:08:51.493907 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:51.503489 23447 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:51.442042 27924 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:51.491400 27625 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:51.506821 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:51.457998 27852 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:08:51.493160 27694 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:51.505241 22750 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:51.543339 22829 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:08:51.553174 22749 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:51.510361 22823 raft_consensus.cc:2376] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11. Re-sending same reply.
I20260709 13:08:51.548179 22830 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
W20260709 13:08:51.502130 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:51.557602 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50871.4us
I20260709 13:08:51.561781 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:51.503647 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 64054.8us
I20260709 13:08:51.554082 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:51.553038 27916 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:08:51.569657 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.555506 27694 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:51.550763 27924 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:51.572415 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:51.580404 22827 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:51.568011 27819 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:51.510896 27799 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
W20260709 13:08:51.577833 27919 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:08:51.583527 27927 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:51.589661 27852 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:08:51.601557 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 252786us
I20260709 13:08:51.587924 27793 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:51.553247 27817 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.586149 27819 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:51.595360 27915 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
W20260709 13:08:51.602618 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.604809 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:51.584053 27913 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:08:51.554100 27826 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:51.636224 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 66621.5us
I20260709 13:08:51.636847 27826 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:51.609694 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:51.602064 27852 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:51.637774 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43879us
I20260709 13:08:51.594077 27697 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:51.622922 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:51.639461 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.616106 27927 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:51.646673 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.634112 27819 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:51.649358 22896 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:51.660287 22896 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:51.638763 27573 consensus_queue.cc:1175] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aad11e70ca94fce8f672db05f1c28c3" responder_term: 13 status { last_received { term: 11 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: [12.12-12.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:51.635775 27494 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 13:08:51.663127 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.647475 27833 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:51.658538 27928 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:51.631974 27537 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12.
I20260709 13:08:51.635305 27831 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.635099 27793 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:51.666198 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:51.653966 27927 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.660352 27933 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:08:51.678107 27934 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:08:51.681761 27793 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.677954 27831 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.684597 27831 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:51.672561 22896 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.662111 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.665401 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.694986 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:51.681097 27911 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:08:51.669921 27928 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:51.703637 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 192664us
I20260709 13:08:51.662446 27431 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.638609 27852 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.677134 27833 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:51.680411 27862 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:51.709782 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:51.710203 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48209us
I20260709 13:08:51.685093 27831 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:08:51.696269 27932 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:51.671869 27844 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:51.716292 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:51.718186 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:51.716651 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:51.727572 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 47503.4us
I20260709 13:08:51.704406 27537 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:51.704591 27846 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:51.741716 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:51.742826 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:51.744122 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:51.735230 22904 raft_consensus.cc:2368] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13.
I20260709 13:08:51.713836 27828 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:51.751967 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:51.752753 22830 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:51.753788 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:51.754364 22900 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:51.720464 27932 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:51.757165 22756 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:51.703902 27696 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 13:08:51.764439 27938 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:08:51.748839 27933 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:51.766352 27494 raft_consensus.cc:2764] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:08:51.767391 27939 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:51.770454 27844 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:51.771252 27873 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:51.749050 27828 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:51.773515 27929 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:51.774292 27828 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.775714 27828 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:51.777565 27928 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.773536 27833 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.779165 27928 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:51.760999 27804 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.781780 27804 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.783730 27804 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:51.766222 27933 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:51.765976 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 59170us
I20260709 13:08:51.732206 27817 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:08:51.852093 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:51.786497 22899 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:51.841655 22905 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:08:51.852187 22903 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:51.763038 27932 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:51.756073 27852 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:51.758697 27820 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.854722 27846 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
W20260709 13:08:51.936259 27730 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.009s	sys 0.016s
W20260709 13:08:51.936714 27730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.015s	user 0.010s	sys 0.016s
I20260709 13:08:51.933921 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:51.849032 27856 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:51.951035 23453 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:51.856784 27929 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:08:51.937404 27804 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:51.965000 27831 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:51.965773 27831 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:51.968266 27831 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:51.963016 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 133619us
I20260709 13:08:51.969780 22825 raft_consensus.cc:2368] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11.
W20260709 13:08:51.963735 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 268721us
W20260709 13:08:51.991715 27896 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:08:51.993187 27944 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 13:08:52.007098 27917 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:08:51.991242 27955 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:51.993942 27935 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:08:51.972033 27911 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:52.005437 27936 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.972234 27797 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:52.012473 27948 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 13:08:52.050474 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 413530us
I20260709 13:08:52.053234 27955 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:51.975256 27893 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:52.060194 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 308821us
I20260709 13:08:51.972594 27804 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:52.053618 27842 consensus_meta.cc:287] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.002s
I20260709 13:08:52.047333 27855 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:52.071668 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:52.046559 27431 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:52.060961 27841 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
I20260709 13:08:52.071964 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:51.948076 27933 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.104471 27893 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.104658 27933 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:52.088806 27579 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.599s	user 0.002s	sys 0.005s
W20260709 13:08:52.089342 27947 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.108s	user 0.002s	sys 0.000s
I20260709 13:08:52.089440 27841 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:52.089949 27945 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.002s
I20260709 13:08:52.091579 27833 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:52.091512 27924 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:52.098685 27694 consensus_meta.cc:287] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.000s
I20260709 13:08:52.102562 27820 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.079174 27842 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.113161 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:52.139310 22754 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:52.113049 27955 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.113471 27940 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.142s
I20260709 13:08:52.076283 27911 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:52.082235 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:52.077527 27855 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:52.129724 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67649us
I20260709 13:08:52.124367 27841 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.138603 27957 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:52.140596 22753 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:08:52.107458 27856 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.146168 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:52.147199 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:52.170660 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88600.2us
I20260709 13:08:52.149448 23453 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:08:52.171337 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:52.150210 27962 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:52.182381 27967 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.182874 27967 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.186458 27924 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:52.129271 27694 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.168941 22753 pending_rounds.cc:85] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 9
I20260709 13:08:52.194166 22753 pending_rounds.cc:107] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:08:52.194520 22753 raft_consensus.cc:2889] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:52.174476 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.174614 27828 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:52.165289 27932 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.187872 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:52.138938 27833 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:52.302738 27833 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.070036 27936 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.306769 27936 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:52.164660 27537 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:52.279573 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 108299us
I20260709 13:08:52.307641 27537 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.308449 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:52.185205 27967 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:52.187915 27819 consensus_meta.cc:287] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.000s
I20260709 13:08:52.309458 27968 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:52.310030 27537 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.309724 27819 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.310793 27968 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:52.311584 27694 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.311836 27819 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:52.187088 27955 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:52.317950 27913 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:52.318318 27913 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:52.318399 27842 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:52.319722 22747 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:52.319478 27913 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.320840 27932 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:52.322131 27826 consensus_meta.cc:287] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.682s	user 0.004s	sys 0.012s
I20260709 13:08:52.322746 27949 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:52.323297 27949 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.326730 27911 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.329504 27855 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.331727 27804 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.320655 27913 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:52.140655 27693 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:52.193145 27822 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:52.341876 27822 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.342703 27822 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:52.280759 27856 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:52.347388 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 159577us
I20260709 13:08:52.298218 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:08:52.329993 23451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1380) took 1901 ms (client timeout 1902 ms). Trace:
W20260709 13:08:52.348906 23451 rpcz_store.cc:269] 0709 13:08:50.428115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.428236 (+   121us) service_pool.cc:224] Handling call
0709 13:08:52.329770 (+1901534us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54007d0fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:52.329920 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2967530880}
W20260709 13:08:52.293111 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 127717us
I20260709 13:08:52.371258 22754 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:52.376063 27957 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:52.376747 27820 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.381338 27820 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.381551 27862 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:52.381479 27957 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.382665 27862 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:52.383028 27862 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:52.383594 27820 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.384227 27862 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:52.377416 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 69030us
I20260709 13:08:52.387069 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.387633 22898 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:52.388760 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.391397 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.392575 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.348222 27924 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:52.393671 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.401791 22901 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:52.397686 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.405260 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.407935 27985 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.408190 22897 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:08:52.408445 27985 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.409116 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.410506 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.410945 27985 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:52.411386 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:52.416967 22898 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:08:52.407306 27862 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:52.378440 27933 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:08:52.346479 27693 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.418138 22898 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:08:52.418469 22898 pending_rounds.cc:85] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 14
I20260709 13:08:52.418622 27933 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:52.418859 22897 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:52.419018 27933 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:52.418655 22898 pending_rounds.cc:107] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:08:52.420334 22898 raft_consensus.cc:2889] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:52.374534 27970 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:52.421250 27970 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:52.421682 27799 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:52.422094 27799 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:52.422283 27970 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.422829 27799 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.424147 27799 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:52.419845 27933 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.421701 27831 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.426855 27831 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.427493 22897 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:52.427829 22897 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:52.379364 27856 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.428777 27831 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:52.430094 27856 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.428508 22897 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.433089 22899 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:52.435410 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:52.437184 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:52.439215 22897 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:52.440969 22905 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
W20260709 13:08:52.443235 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 64191.7us
I20260709 13:08:52.445027 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:52.433940 27924 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:52.448968 27924 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.419904 27693 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:52.457648 23451 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:52.431330 27949 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:52.458009 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:52.437233 27933 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:52.454639 27928 consensus_meta.cc:287] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.675s	user 0.003s	sys 0.010s
I20260709 13:08:52.468137 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:52.468907 22826 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:52.469642 23451 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:52.469977 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:52.470062 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:52.473124 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:52.474467 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:52.494560 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:52.495035 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:52.505791 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:52.506749 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:52.508409 23453 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:52.511027 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:52.511519 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:52.511766 22752 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:08:52.512677 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:52.514632 22825 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:52.517602 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:52.517881 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:52.519330 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:52.519338 22748 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:52.521584 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:52.471418 27730 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:52.522751 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48374.5us
I20260709 13:08:52.524294 22719 leader_election.cc:400] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11.
I20260709 13:08:52.524597 22719 leader_election.cc:403] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:08:52.524904 22900 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:08:52.527240 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:52.454692 27856 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:08:52.531477 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:52.535161 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:52.518044 27698 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:52.566874 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:52.518069 27804 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.565851 22829 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:52.517596 27873 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.568361 22747 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:52.572120 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:52.573194 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:52.533354 27980 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:52.529906 27974 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11.
I20260709 13:08:52.595679 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:52.596277 22829 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:52.597644 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:52.598694 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:52.599635 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:52.600590 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:52.601940 27976 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:08:52.602643 22896 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:52.578814 27698 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.605438 22831 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:52.579007 27694 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:52.607812 27694 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:52.608677 27694 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.601086 22825 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
W20260709 13:08:52.582778 27852 consensus_meta.cc:287] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.689s	user 0.015s	sys 0.007s
I20260709 13:08:52.565869 27952 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:52.579550 27537 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:52.617774 22831 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:52.618927 22898 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:52.617129 22897 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:52.593274 27980 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.632349 22825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:08:52.633107 22825 raft_consensus.cc:606] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:52.634678 27986 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:08:52.635144 22825 raft_consensus.cc:614] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:52.636644 27866 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:08:52.592109 27873 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.593006 27910 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:52.566939 27822 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.566020 27985 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:08:52.577956 27831 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:08:52.620339 27537 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.582424 27804 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.641665 27698 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:52.642019 27980 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:52.650480 27974 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:52.651876 27537 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:52.646456 27910 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.676000 27910 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:52.660207 27974 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:52.685742 23453 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:52.687548 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:52.722895 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2785) took 1291 ms. Trace:
I20260709 13:08:52.723379 22822 rpcz_store.cc:276] 0709 13:08:51.431149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.431279 (+   130us) service_pool.cc:224] Handling call
0709 13:08:51.458811 (+ 27532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:51.459186 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:51.459225 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:51.478617 (+ 19392us) log.cc:844] Serialized 26 byte log entry
0709 13:08:51.564549 (+ 85932us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:51.711786 (+147237us) log.cc:844] Serialized 14 byte log entry
0709 13:08:51.712357 (+   571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.712419 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:52.329037 (+616618us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b1800475a20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e522 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:52.644461 (+315424us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b1800475a20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:52.689440 (+ 44979us) raft_consensus.cc:1685] finished
0709 13:08:52.689465 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:52.722810 (+ 33345us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19634,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":2743,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":24266,"spinlock_wait_cycles":7534336,"thread_start_us":188227,"threads_started":2,"wal-append.queue_time_us":90818}
W20260709 13:08:52.730993 27977 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.204s	user 0.003s	sys 0.000s
I20260709 13:08:52.733435 27974 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.734948 22831 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:52.735105 27974 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:52.736666 22831 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:52.736248 27705 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:52.741441 27797 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:52.740965 27705 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:52.743851 27705 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:52.723155 27988 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.002s
I20260709 13:08:52.747282 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3109) took 1022 ms. Trace:
I20260709 13:08:52.747421 22831 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:52.752321 27822 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.751956 22822 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:08:52.748875 22750 rpcz_store.cc:276] 0709 13:08:51.725177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.725785 (+   608us) service_pool.cc:224] Handling call
0709 13:08:51.726146 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:52.134857 (+408711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:52.134907 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:52.135385 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:08:52.149533 (+ 14148us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:52.149575 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.149648 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:52.730325 (+580677us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:52.738236 (+  7911us) raft_consensus.cc:1685] finished
0709 13:08:52.738281 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:52.747192 (+  8911us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5527,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":6456576,"thread_start_us":422035,"threads_started":2,"wal-append.queue_time_us":319489}
I20260709 13:08:52.741974 27797 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:52.756619 27625 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:52.757076 27625 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:52.758598 27537 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:52.759028 22868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.019s	sys 0.032s
W20260709 13:08:52.777406 22868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.021s	sys 0.032s
W20260709 13:08:52.777702 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 818179us
W20260709 13:08:52.780387 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.780270 27730 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:52.759333 27873 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:52.781132 23444 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:52.784794 23453 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:52.759970 27987 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:08:52.761538 22897 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:52.785802 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:52.774610 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2795) took 1336 ms. Trace:
I20260709 13:08:52.789028 22823 rpcz_store.cc:276] 0709 13:08:51.437920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.438144 (+   224us) service_pool.cc:224] Handling call
0709 13:08:52.774408 (+1336264us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b5400230ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:52.774541 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51004800}
I20260709 13:08:52.775509 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3105) took 1438 ms. Trace:
I20260709 13:08:52.785260 23451 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:52.790932 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:52.790081 22749 rpcz_store.cc:276] 0709 13:08:51.337330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.337536 (+   206us) service_pool.cc:224] Handling call
0709 13:08:51.553001 (+215465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.555813 (+  2812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.775263 (+1219450us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540016ab50. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:52.775428 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494802304}
W20260709 13:08:52.780798 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.788569 22897 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:52.793110 22897 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.790400 23453 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:52.794348 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:52.739847 27985 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:52.795516 22897 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:52.798074 23445 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:08:52.810467 27866 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:52.811053 27866 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:08:52.814260 27994 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.091s	user 0.004s	sys 0.003s
I20260709 13:08:52.814538 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:52.814900 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.815186 27946 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:52.815605 27946 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:52.815500 27852 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:52.816423 27946 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.780649 27980 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.818974 27980 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.821648 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:52.822299 22823 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:52.839246 22823 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:52.857903 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:52.859766 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.844456 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:52.813301 27866 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:52.823441 27848 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:52.880106 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.884507 22896 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:08:52.891945 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.874341 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:52.946542 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 224486us
I20260709 13:08:52.798640 27506 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.113s
I20260709 13:08:52.948268 22826 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:08:52.949028 22826 raft_consensus.cc:606] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:52.950075 23446 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:52.949996 22823 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:08:52.861310 22897 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:08:52.802009 27730 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 13:08:52.951465 22897 pending_rounds.cc:85] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 9
I20260709 13:08:52.914153 27696 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.977032 22897 pending_rounds.cc:107] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:08:52.868453 27946 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:52.998988 27979 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:52.953048 27844 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:53.005456 27844 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.013796 22748 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:53.014256 22750 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:53.015265 22754 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:53.016520 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:52.886512 27537 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.018607 27873 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:52.981611 27696 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.019310 27873 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.865154 27957 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:53.021591 27696 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.982183 27308 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:53.022101 27957 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.024919 27957 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:52.982887 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:52.883661 27797 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.025902 22902 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:53.027130 27797 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:52.883267 27913 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:53.027837 27913 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.029696 27913 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.031093 22751 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:08:53.035216 27910 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:53.035856 27910 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.038350 27606 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:53.038995 27606 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.868750 27983 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:53.042178 27983 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:53.042867 27693 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.043231 27983 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:52.868873 27985 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:52.978884 22897 raft_consensus.cc:2889] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:53.016701 27973 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:53.043556 27693 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:52.936340 27528 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:53.050721 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.057976 22754 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:53.061144 22748 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:53.063297 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.064234 22749 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:53.096726 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:52.865434 27933 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.209s
I20260709 13:08:53.103840 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.107125 22756 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:53.104333 22747 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:53.115124 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:53.115818 22826 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:52.869247 27975 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.227s
I20260709 13:08:53.067718 27537 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:53.119817 22823 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:53.123893 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.123972 22823 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:08:53.124564 22831 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:53.105726 27705 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.129827 27705 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.132282 27705 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.138711 27935 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:53.138986 27911 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:53.139160 27935 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:53.092909 27957 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:53.142802 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2796) took 1704 ms. Trace:
I20260709 13:08:53.094681 27871 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:53.142802 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 46172.1us
I20260709 13:08:53.099665 27528 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:53.070513 27973 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.167141 22823 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:53.168286 22902 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:53.139578 27911 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.143280 22827 rpcz_store.cc:276] 0709 13:08:51.438676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.438787 (+   111us) service_pool.cc:224] Handling call
0709 13:08:51.480825 (+ 42038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.580819 (+ 99994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.142605 (+1561786us) spinlock_profiling.cc:243] Waited 99.4 ms on lock 0x7b54001f9ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:53.142744 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228533504}
I20260709 13:08:53.092322 27674 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:08:53.196118 27674 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:53.196480 27674 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:53.197201 27974 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.197717 27674 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.198184 27974 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.199020 27674 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:53.199361 27995 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.459s	user 0.002s	sys 0.000s
I20260709 13:08:53.200114 27974 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:53.200834 27930 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
W20260709 13:08:53.201462 27990 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.599s	user 0.000s	sys 0.006s
W20260709 13:08:53.202301 27959 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.446s	user 0.010s	sys 0.000s
I20260709 13:08:53.154865 27717 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:53.202960 27717 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.203292 27717 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:53.204155 27974 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.204610 27974 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.206111 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1968) took 1455 ms. Trace:
I20260709 13:08:53.207213 27625 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.208274 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.209765 27731 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:53.206487 22824 rpcz_store.cc:276] 0709 13:08:51.750993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.751093 (+   100us) service_pool.cc:224] Handling call
0709 13:08:51.751390 (+   297us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:51.755987 (+  4597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:51.756018 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:51.762615 (+  6597us) log.cc:844] Serialized 52 byte log entry
0709 13:08:52.049031 (+286416us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:52.072521 (+ 23490us) log.cc:844] Serialized 14 byte log entry
0709 13:08:52.100531 (+ 28010us) log.cc:844] Serialized 14 byte log entry
0709 13:08:52.122219 (+ 21688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.122370 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:52.756388 (+634018us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b1800146800. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:53.201740 (+445352us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:53.205889 (+  4149us) raft_consensus.cc:1685] finished
0709 13:08:53.205923 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:53.206056 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54157,"prepare.queue_time_us":75573,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":35127,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":38452,"spinlock_wait_cycles":41055872,"thread_start_us":177812,"threads_started":1,"wal-append.queue_time_us":300972}
I20260709 13:08:53.210242 27731 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.148075 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:53.211369 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 63390.7us
I20260709 13:08:53.163164 22826 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:53.117039 27606 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.092087 27913 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:08:53.212704 27606 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:53.212807 27370 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:53.213305 27370 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:53.213694 27370 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:53.213970 27913 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:53.215478 27799 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:53.215911 27799 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:53.216822 27799 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.206826 27974 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.217962 27974 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:53.212129 27731 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.217401 27852 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.095916 27946 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.096s
I20260709 13:08:53.124048 27696 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.067147 27866 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:53.140064 27537 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.231844 27822 raft_consensus.cc:1064] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:08:53.228847 27852 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.231506 27866 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.233405 27537 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:53.233613 28001 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:08:53.154573 27957 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.234675 27866 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.235791 27957 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.236953 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.209154 27625 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:53.165944 27873 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.218297 27799 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:53.157665 27871 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.223661 27985 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.241824 27573 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:08:53.247692 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 294587us
I20260709 13:08:53.166220 27940 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
W20260709 13:08:53.258965 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 282972us
I20260709 13:08:53.262583 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.263898 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.264591 22904 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:08:53.265791 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.266454 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.269783 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.218397 27974 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.277915 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.278688 22896 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:53.259835 27573 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:53.281539 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.282279 22899 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:53.282709 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.281611 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:53.238152 27696 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.283987 22903 raft_consensus.cc:2368] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13.
I20260709 13:08:53.286592 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.218900 27935 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:53.286576 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:53.276504 27625 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:53.295660 27696 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.297827 27573 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:53.301913 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:53.307190 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.246234 27528 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.309165 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.289039 27848 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:53.323130 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:53.306106 27935 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:53.325812 22897 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:53.328162 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.328331 22897 raft_consensus.cc:606] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:53.328883 22897 raft_consensus.cc:614] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:53.315258 27573 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:53.317950 27730 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:53.317436 27975 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:53.305346 27913 consensus_queue.cc:1175] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6a5191aaff76469bbf6c4735dc9e6509" responder_term: 12 status { last_received { term: 11 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:08:53.335810 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 123648us
I20260709 13:08:53.347504 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.322391 27933 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.337836 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.324888 27866 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.321727 27826 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:53.337199 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.337671 23451 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:53.308228 28003 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 13:08:53.363175 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.364238 23451 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:53.347892 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:53.334870 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.348122 22897 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 6.
I20260709 13:08:53.338879 23444 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:53.373425 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.323176 27625 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:08:53.335929 27793 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:53.384545 27793 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:53.374606 27696 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:53.374302 27826 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.370460 27705 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:53.386514 27705 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.386874 27855 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:53.387511 27826 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:53.388043 27826 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:53.379452 27928 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:53.388830 27928 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:53.380632 27997 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.633s	user 0.000s	sys 0.010s
I20260709 13:08:53.390488 27928 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.391466 27928 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:08:53.392020 28007 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:08:53.376245 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:53.396471 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.385768 27696 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.397527 27696 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:53.399185 28004 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:53.399824 28004 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:53.400445 27838 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:53.400266 28004 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.400856 27838 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:53.397533 27826 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:53.401744 27826 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:53.402393 27826 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:53.347716 27929 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:53.402566 27871 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.401288 27838 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:53.401777 27974 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.402906 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2542) took 2358 ms. Trace:
I20260709 13:08:53.409523 22753 rpcz_store.cc:276] 0709 13:08:51.044861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.044983 (+   122us) service_pool.cc:224] Handling call
0709 13:08:51.045311 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:52.299355 (+1254044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.402694 (+1103339us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003cebd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:53.402849 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2490022656}
I20260709 13:08:53.383513 27370 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.388423 27705 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.405881 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.347908 27973 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.402486 28013 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.370016 27818 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.413048 22826 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:53.369463 27309 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.422030 27309 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.422981 23445 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:53.424268 27309 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.425969 27370 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.426563 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.428985 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:53.369629 27866 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.402913 27929 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:53.375577 28006 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:08:53.418004 22900 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:53.418115 28013 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.419281 27818 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.419492 27848 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.420783 27844 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.414959 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.349290 27975 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.362847 27933 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.436023 27974 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.409750 27940 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:08:53.446118 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.447196 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.449990 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.467417 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.450618 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:53.468384 22905 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:53.451448 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.452934 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.433205 27866 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:53.428717 27822 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 9, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.381207 27717 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.470942 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:53.476938 22823 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:53.471222 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:53.471832 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 185289us
I20260709 13:08:53.473712 23444 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:53.477769 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:53.474051 27823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.819s	user 0.000s	sys 0.027s
W20260709 13:08:53.478947 27823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.824s	user 0.000s	sys 0.027s
I20260709 13:08:53.485615 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.486505 23445 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:53.467022 22904 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:53.488427 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.489604 22904 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:08:53.494304 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.495023 22904 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:08:53.476038 22827 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:08:53.474608 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.497905 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.506951 22829 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:08:53.503551 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.506904 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.510463 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.510829 22827 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:53.510990 22750 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:53.511343 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:53.513737 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.514045 22905 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:53.514438 22748 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:53.512045 22823 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:53.515746 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.516572 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.517408 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.517915 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.518242 22752 raft_consensus.cc:2393] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:53.518929 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.519668 22753 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
W20260709 13:08:53.519871 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 186684us
I20260709 13:08:53.520649 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.506673 27817 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:53.536429 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.528924 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.537305 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.530833 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.532924 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.548556 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.511972 27844 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:53.472680 27974 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.552517 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.514904 28014 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:53.567636 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.561131 22755 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:53.528966 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.568876 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.568954 22823 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:53.562933 22752 raft_consensus.cc:2410] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:08:53.529711 27730 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:53.510226 27823 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:08:53.570775 22755 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:53.533087 27871 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:53.572525 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.562291 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.563325 22905 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
W20260709 13:08:53.588992 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 73295.7us
I20260709 13:08:53.589290 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.564707 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.566152 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.564504 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.591342 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.577140 22750 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:53.582561 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.542958 27940 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:53.586886 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.592665 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.594254 22747 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:08:53.595278 22752 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:08:53.597894 22753 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:08:53.517040 27696 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.571110 27817 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.601894 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.542164 27625 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:08:53.573608 27799 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:53.552043 27973 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:08:53.572111 28014 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.552295 28010 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:08:53.547971 27797 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:08:53.609970 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.621744 22751 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:53.622396 22749 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:53.581934 28019 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:53.599584 27856 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:53.590090 28004 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.600708 27871 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:53.609259 27866 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:53.612459 27817 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:53.636999 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.565032 27844 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.557937 28018 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:08:53.575176 27985 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:08:53.610724 27940 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.633944 27818 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.626047 28011 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:08:53.634634 27975 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.622700 28014 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:53.635103 27871 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:53.612699 27892 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:53.639408 28013 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.639627 27696 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.639680 27974 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.637349 28004 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:08:53.625120 27856 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.682010 27856 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:53.683275 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2802) took 2188 ms. Trace:
I20260709 13:08:53.683634 22828 rpcz_store.cc:276] 0709 13:08:51.494868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.495004 (+   136us) service_pool.cc:224] Handling call
0709 13:08:52.775099 (+1280095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:52.775604 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:52.775642 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:52.776264 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:08:52.930922 (+154658us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:52.952110 (+ 21188us) log.cc:844] Serialized 14 byte log entry
0709 13:08:52.952349 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.952423 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:53.236040 (+283617us) raft_consensus.cc:1685] finished
0709 13:08:53.236060 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:53.683125 (+447065us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400230ad0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:53.683227 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20940,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":3164,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":24561,"spinlock_wait_cycles":2943021824,"thread_start_us":150958,"threads_started":1,"wal-append.queue_time_us":379324}
I20260709 13:08:53.643851 27852 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:53.639834 28021 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:53.653986 27866 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.685945 28004 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:53.685846 28021 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.686919 28004 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.689360 27892 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.648447 27910 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.649113 22902 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:53.646597 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.692219 22900 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:53.693074 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.638366 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.694126 22900 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:53.668371 27985 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:08:53.700021 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.672200 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.680380 22754 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 7.
I20260709 13:08:53.687352 27866 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:53.625550 27705 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 13:08:53.667866 27991 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.065s	user 0.004s	sys 0.003s
I20260709 13:08:53.608676 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.695670 28031 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:53.697386 27933 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.697101 27856 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.697613 27731 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:53.676136 27730 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.693112 27833 raft_consensus.cc:1027] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: attempt to promote peer c45e1805f049467697a7358fb566de2f: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 13:08:53.640479 28017 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:53.700891 22755 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:53.703732 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.716354 28029 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.719758 28029 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.720808 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.721688 22899 raft_consensus.cc:2410] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:08:53.722471 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:53.723191 27929 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.724460 27929 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:53.707343 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:53.711864 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 164279us
I20260709 13:08:53.712600 27717 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.708101 27973 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:08:53.759999 27871 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.713222 27694 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:53.725556 22905 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:53.714180 27970 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:08:53.715513 27731 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:53.700203 27985 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:53.731256 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.732766 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:53.733713 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.718688 27940 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.739131 22825 raft_consensus.cc:2368] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16.
I20260709 13:08:53.764648 27985 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.765100 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.765941 22901 raft_consensus.cc:2410] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:08:53.764662 22903 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:53.766712 22901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:08:53.767252 22901 raft_consensus.cc:606] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:53.768008 22901 raft_consensus.cc:614] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:53.740396 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1845) took 1315 ms. Trace:
I20260709 13:08:53.722844 28029 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.741807 27937 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:53.741752 27826 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:53.742851 27924 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:53.781131 27924 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:53.781570 27924 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:53.782575 27924 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.783123 27893 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.783744 27924 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:53.783576 27893 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.743585 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:53.785475 27893 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.771118 27910 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.787333 27910 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.787957 27980 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:53.788599 27980 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:53.702692 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.746655 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 946) took 1279 ms. Trace:
I20260709 13:08:53.747009 22900 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:53.732677 27838 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.790216 27910 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.789521 22830 rpcz_store.cc:276] 0709 13:08:52.467375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.467477 (+   102us) service_pool.cc:224] Handling call
0709 13:08:52.467762 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:52.471484 (+  3722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:52.471510 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:52.471859 (+   349us) log.cc:844] Serialized 26 byte log entry
0709 13:08:52.527356 (+ 55497us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:52.564849 (+ 37493us) log.cc:844] Serialized 14 byte log entry
0709 13:08:52.565036 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.565104 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:52.985845 (+420741us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b1800378100. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:53.667250 (+681405us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:53.711005 (+ 43755us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b1800378100. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e522 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:08:53.746477 (+ 35472us) raft_consensus.cc:1685] finished
0709 13:08:53.746497 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:53.746607 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67274,"prepare.queue_time_us":34599,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":6521,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":4390,"spinlock_wait_cycles":56730112,"thread_start_us":55057,"threads_started":1,"wal-append.queue_time_us":131030}
I20260709 13:08:53.792178 27838 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.792285 27844 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.792372 22749 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
W20260709 13:08:53.795357 27999 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.777s	user 0.003s	sys 0.000s
I20260709 13:08:53.747478 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.722950 22899 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:08:53.753595 27992 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.147s	user 0.008s	sys 0.005s
I20260709 13:08:53.761633 22827 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:53.762548 28031 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:53.763118 27694 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:53.799629 27844 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:53.799819 27694 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:53.800160 27844 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.800160 28031 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.765884 27985 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:53.763017 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 154404us
I20260709 13:08:53.769548 22898 rpcz_store.cc:276] 0709 13:08:52.425488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.425617 (+   129us) service_pool.cc:224] Handling call
0709 13:08:52.425873 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:52.631597 (+205724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.740161 (+1108564us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400184554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:53.740340 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443022592}
I20260709 13:08:53.770594 27705 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:53.772122 27528 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:53.774817 28021 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.744971 28014 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.706413 27817 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.807324 22904 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:53.789863 27973 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:53.809476 27980 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:53.810882 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1398) took 1006 ms. Trace:
I20260709 13:08:53.811268 23453 rpcz_store.cc:276] 0709 13:08:52.805055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.820382 (+ 15327us) service_pool.cc:224] Handling call
0709 13:08:53.810708 (+990326us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54004f3154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:53.810826 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893969152}
W20260709 13:08:53.797112 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49703.8us
I20260709 13:08:53.799346 28012 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:53.800788 27844 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:53.805160 22797 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.741362 27856 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:53.789853 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 87265.2us
I20260709 13:08:53.821373 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.823168 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:53.823987 27431 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:08:53.825906 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 216091us
I20260709 13:08:53.816939 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.826144 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.817600 27705 raft_consensus.cc:2764] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:08:53.828527 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.830155 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.830931 22901 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:53.831171 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.816027 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1396) took 1025 ms. Trace:
I20260709 13:08:53.833490 27833 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:53.835556 27826 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:53.837574 22896 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:53.840206 27694 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.841688 27705 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:53.842963 27705 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:53.844059 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.844247 27962 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:53.844882 27929 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:53.845237 27929 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.845631 27929 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:53.848778 23419 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.850417 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.854573 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:53.854629 22755 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:53.855172 22898 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:53.855866 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.855602 27308 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:53.856426 27308 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:53.856426 22749 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:53.857524 27911 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:53.857682 22897 raft_consensus.cc:2410] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:08:53.860260 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:53.862843 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.863541 22755 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:53.864777 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.865625 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.866376 22897 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:53.872501 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:53.873137 23445 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:08:53.874502 23416 leader_election.cc:400] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16.
I20260709 13:08:53.874799 23416 leader_election.cc:403] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:53.878026 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:53.878465 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 47353.2us
I20260709 13:08:53.878742 23444 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:08:53.831912 23449 rpcz_store.cc:276] 0709 13:08:52.790416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.791989 (+  1573us) service_pool.cc:224] Handling call
0709 13:08:52.805840 (+ 13851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:52.806164 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:52.806193 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:53.737947 (+931754us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b2c0013bd48. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:08:53.738285 (+   338us) log.cc:844] Serialized 26 byte log entry
0709 13:08:53.797851 (+ 59566us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:53.797898 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.798095 (+   197us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:53.813109 (+ 15014us) raft_consensus.cc:1685] finished
0709 13:08:53.813128 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:53.815969 (+  2841us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1682,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"spinlock_wait_cycles":51719936,"thread_start_us":59175,"threads_started":1,"wal-append.queue_time_us":68298}
I20260709 13:08:53.879628 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.880764 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.881326 22747 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:53.882382 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.882965 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.883505 22831 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:53.857230 27308 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:53.849681 28021 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:53.829722 27506 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:53.885682 22896 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:08:53.885821 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.898597 22748 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:53.887344 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.889385 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.890743 22825 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:53.895475 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.900274 22827 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:53.905258 23451 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:53.906853 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.908187 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.910609 22898 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
W20260709 13:08:53.911106 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.895568 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.912603 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.913659 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.916280 22752 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:08:53.918479 22831 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:53.920233 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.891145 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.893402 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:53.893497 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.895908 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.921888 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:53.897058 22823 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:08:53.897847 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:53.923925 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.923396 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.849371 27838 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:08:53.849545 27848 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:08:53.885074 27957 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:53.967316 22827 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:08:53.958057 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 66956.2us
I20260709 13:08:53.969234 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.961169 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:53.970254 22901 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:08:53.970364 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:53.954035 22822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:08:53.971112 22822 raft_consensus.cc:606] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:53.971997 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.973457 22719 leader_election.cc:400] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13.
I20260709 13:08:53.889242 28004 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:53.886224 27985 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:53.981531 27949 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:08:53.966243 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72834.7us
I20260709 13:08:53.865545 27730 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:08:53.983054 22749 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:53.983137 27717 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:53.983700 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.983902 27717 consensus_queue.cc:1243] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:08:53.984431 27573 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:53.984885 27573 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:53.985280 27949 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:53.985654 22750 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 12. Ops: []
I20260709 13:08:53.986603 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.984481 27717 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:53.985409 27913 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:53.981912 22748 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:53.966255 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:53.987213 27717 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.987241 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.986845 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:53.976712 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:53.987938 22755 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:53.977967 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 54845.9us
I20260709 13:08:53.988483 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:54.008504 22898 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:54.008569 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:54.009370 22896 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:08:53.909639 27826 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:54.011221 22871 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:53.965137 27693 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.909672 27871 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:53.926507 27910 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.953536 28029 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:53.921264 27927 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:53.965170 27838 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:53.973026 27415 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:53.973815 27957 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:53.974634 27833 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:54.013273 27949 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:54.033154 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:54.021672 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:53.953714 27842 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:08:54.033742 22825 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:08:54.022826 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:54.014361 27933 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:53.988453 27370 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:54.023293 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:54.029042 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:54.030436 22830 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:53.902683 27893 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:08:54.030908 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:54.030977 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:53.987962 27717 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:54.016072 27866 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:53.966424 27962 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:08:54.038486 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:54.038769 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:54.024458 27838 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:54.018275 28029 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.039239 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:53.967119 28009 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:53.978397 28021 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:08:54.048369 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:54.107901 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:54.107743 27866 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.110176 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:54.020134 27852 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:54.109444 27856 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:08:54.107529 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 68339.5us
I20260709 13:08:54.093662 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:54.017901 28004 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:08:54.032717 27957 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:54.132478 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:54.139511 22753 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:54.127851 22898 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:08:54.019004 27826 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.027156 27910 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.135390 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:54.017766 27985 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:08:54.136873 22824 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:08:54.032943 27833 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:54.109599 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:54.093976 27804 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.136767 27913 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.136734 27856 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:54.098083 28048 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:08:54.035303 27871 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:54.111699 28009 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:54.110178 27866 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:54.143039 27838 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.034288 27973 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.114s
I20260709 13:08:54.034147 27937 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.110s
I20260709 13:08:54.105554 27370 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.034575 28011 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 13:08:54.149750 23445 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
W20260709 13:08:54.150393 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 102089us
I20260709 13:08:54.153117 22899 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
W20260709 13:08:54.187681 28041 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:08:54.110585 28029 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:54.157760 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:54.206816 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49170.6us
I20260709 13:08:54.207397 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:54.158648 28031 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:54.140435 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:54.108877 22831 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:08:54.211305 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:54.212677 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:54.136992 28019 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:54.161507 27975 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.220229 28032 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:54.220715 28032 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.220642 27975 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.221235 28032 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:54.223249 27975 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:54.223610 27993 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.430s	user 0.003s	sys 0.004s
W20260709 13:08:54.224839 28034 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:08:54.225708 28047 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:08:54.226043 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:54.226398 27826 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.165371 27833 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:08:54.170219 28042 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:08:54.188135 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:54.168953 27856 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:54.173933 28009 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:54.182196 27984 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.202965 22905 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:08:54.211350 27804 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:54.041718 23449 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 11 LEARNER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:08:54.212056 28045 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:54.186616 27793 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16.
I20260709 13:08:54.209806 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:54.169440 27913 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.202638 28039 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:54.159135 28023 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:54.182679 27871 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:54.216615 22755 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:08:54.218233 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:54.250794 22753 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:08:54.205689 27838 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:54.219791 28019 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:54.233011 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44963.3us
I20260709 13:08:54.184363 27370 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:54.219177 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:54.251195 22899 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:54.256495 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:54.259361 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:54.280905 22904 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:08:54.281286 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:54.281693 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:54.246557 28039 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.245929 28023 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:54.314235 28023 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.304452 27893 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.315407 27893 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.317406 27893 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:54.304948 28055 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:08:54.305416 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:54.305903 28021 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:54.283504 27985 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:54.329582 27415 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:54.330250 27415 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:54.331030 27415 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:54.331887 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:54.333499 28004 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:54.333945 28004 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:54.302338 28039 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:54.310292 28056 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:08:54.316939 28032 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.311276 27871 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.342906 28032 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:54.333164 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:54.346681 27913 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.346695 27893 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:54.347342 27893 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.347666 27893 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:54.350495 27924 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
I20260709 13:08:54.243604 27984 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.352684 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:54.354472 28019 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.322840 27974 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.355217 27974 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.284518 28040 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:54.321898 27804 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:54.357632 27804 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:54.357993 27804 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:54.337248 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:54.336681 23445 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:54.334913 28004 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.339346 28021 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:54.439867 28004 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:54.352514 27984 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.327598 22900 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:08:54.326730 28011 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:08:54.381867 28032 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:54.383729 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:54.391407 27949 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:08:54.398649 27929 consensus_meta.cc:287] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.000s
I20260709 13:08:54.445310 27929 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.447713 27929 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:54.357105 27974 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.450037 27717 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.451264 22901 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:08:54.332938 27415 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:54.398721 28009 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:54.419162 27844 consensus_meta.cc:287] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.569s	user 0.004s	sys 0.004s
I20260709 13:08:54.422076 27804 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.427234 28019 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.385607 23449 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:54.401049 27856 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.438762 27866 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.511912 22751 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:08:54.462857 22825 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:08:54.501506 22897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.004s	sys 0.005s
W20260709 13:08:54.541290 22897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.004s	sys 0.008s
I20260709 13:08:54.460525 27957 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:54.518064 27980 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.004s	sys 0.004s
I20260709 13:08:54.536899 22751 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:54.520274 22904 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:08:54.550731 22751 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:08:54.461457 27415 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:54.517592 28031 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:54.549952 28058 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:08:54.465083 27804 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.496982 27949 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.565385 27957 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:54.565768 27949 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.526576 28030 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:54.542385 27893 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.476665 27856 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:54.548786 27980 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.004s	sys 0.006s
I20260709 13:08:54.454037 27717 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:54.560165 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 278918us
I20260709 13:08:54.464809 27370 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.563200 27831 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:54.561519 23444 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:54.465590 28019 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.547293 27911 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:54.489607 28037 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:54.462590 28009 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.463135 27844 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.534551 27833 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.531210 28011 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:08:54.559732 28031 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:54.574175 23453 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:54.577695 22898 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:08:54.579530 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.627611 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:54.567358 27924 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:08:54.576947 27985 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:54.557703 27415 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:54.668081 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 85337.2us
I20260709 13:08:54.567618 27973 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:08:54.460958 28029 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.695390 22747 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:54.699570 22749 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:08:54.711835 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:54.668640 28009 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:54.669134 27956 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:54.713299 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 153299us
I20260709 13:08:54.671705 27949 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.667318 27866 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.728070 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:54.730470 23445 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:54.729630 22822 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:54.731624 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:54.733544 22829 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:08:54.670631 28039 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.674549 28021 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:08:54.678480 27871 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:54.750419 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:54.737251 27962 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:08:54.768918 28029 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:54.729174 28009 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.769887 27871 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.798058 27804 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:54.801680 27804 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:54.802608 27804 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.751201 27415 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.805516 27415 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:08:54.805792 28066 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:08:54.730248 27956 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.736073 27949 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:08:54.775250 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45709.9us
I20260709 13:08:54.716048 27973 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 13:08:54.795038 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 47748.8us
W20260709 13:08:54.795639 28063 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:08:54.796562 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 68379.7us
I20260709 13:08:54.800534 27892 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.732854 28070 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
W20260709 13:08:54.795490 28060 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:08:54.816329 22751 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:54.818849 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:54.808609 27949 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.818253 27956 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:54.821403 22751 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:54.833135 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:54.833797 22896 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:08:54.834534 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:54.837231 27506 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:54.837649 27506 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:54.837498 27956 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:54.838534 27506 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.843621 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:54.822429 27949 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:54.853471 28071 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:54.854892 27308 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:54.863417 22896 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:54.828733 28019 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.830304 27833 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.867141 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:54.872009 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:54.872700 22896 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:08:54.877553 28039 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:54.877663 27979 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:54.861297 27844 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:54.879058 28031 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:54.880193 28014 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:54.881335 27694 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:54.883968 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:54.897135 22794 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.958s (SENT) [suppressed 10 similar messages]
I20260709 13:08:54.956338 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:54.963141 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:54.963829 22896 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:54.964789 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:54.967042 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:54.953824 28019 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.954191 27979 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:54.961982 27793 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:55.008342 27694 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:55.008888 27694 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.011368 27694 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.012040 27949 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.014132 27949 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.015136 27793 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.015514 27979 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:54.977298 28019 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:55.017746 27793 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.020028 28019 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:55.011811 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1476) took 1106 ms. Trace:
I20260709 13:08:55.021872 22752 rpcz_store.cc:276] 0709 13:08:53.905453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.917116 (+ 11663us) service_pool.cc:224] Handling call
0709 13:08:53.920423 (+  3307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:53.920818 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:53.920840 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:53.921201 (+   361us) log.cc:844] Serialized 26 byte log entry
0709 13:08:54.857952 (+936751us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b18004ccba0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:08:54.945529 (+ 87577us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:54.946184 (+   655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:54.946274 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:55.011486 (+ 65212us) raft_consensus.cc:1685] finished
0709 13:08:55.011513 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.011740 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":39424,"spinlock_wait_cycles":16505728,"thread_start_us":87376,"threads_started":1,"wal-append.queue_time_us":1048392}
I20260709 13:08:55.020140 27308 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.026000 27308 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.028620 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:55.052940 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 95225.3us
I20260709 13:08:55.127549 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.037019 22896 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:55.130124 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.130251 22751 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:55.135854 22896 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:55.127108 27694 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:55.142552 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 245422us
I20260709 13:08:55.146157 22896 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
W20260709 13:08:55.154590 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.167634 28019 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.168181 28019 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.170192 28019 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:55.127729 27793 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.198316 22900 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:55.177588 28029 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:55.197856 27308 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.183266 27694 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.213846 27842 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:55.214439 27842 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:55.219758 28024 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:08:55.220189 28024 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:55.221048 28024 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.223084 28024 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:55.186579 27793 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.213894 27694 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.216171 28029 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.236486 27793 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.236662 28029 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.239835 27731 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.241061 27731 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:55.241120 27893 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.259588 28031 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.303601 27528 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.323834 27528 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.304649 27308 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:55.327692 27308 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.304960 27674 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:55.331180 27674 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.315629 27940 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.261580 28030 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:08:55.263752 28068 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.478s	user 0.000s	sys 0.001s
I20260709 13:08:55.336477 28030 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:55.333885 27940 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:55.264513 28065 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.529s	user 0.000s	sys 0.012s
I20260709 13:08:55.233604 27842 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.342252 27308 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.345943 27308 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:55.346459 27842 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:55.342465 27940 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:55.343871 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1223) took 1444 ms. Trace:
I20260709 13:08:55.347273 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1213) took 1479 ms. Trace:
I20260709 13:08:55.347843 27308 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.335271 27674 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.349736 28031 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:55.350569 28031 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.347787 22897 rpcz_store.cc:276] 0709 13:08:53.868081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.868187 (+   106us) service_pool.cc:224] Handling call
0709 13:08:53.868433 (+   246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:53.874749 (+  6316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:53.874791 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:53.875319 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:08:54.543534 (+668215us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:54.543574 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:54.543697 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:55.263956 (+720259us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:55.346626 (+ 82670us) raft_consensus.cc:1685] finished
0709 13:08:55.346658 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.347031 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13560,"raft.queue_time_us":12657,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":10406,"spinlock_wait_cycles":26984192,"thread_start_us":577818,"threads_started":1,"wal-append.queue_time_us":790743}
I20260709 13:08:55.347452 22900 rpcz_store.cc:276] 0709 13:08:53.870646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.199505 (+1328859us) service_pool.cc:224] Handling call
0709 13:08:55.199817 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.209282 (+  9465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.209342 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:55.214252 (+  4910us) log.cc:844] Serialized 26 byte log entry
0709 13:08:55.314274 (+100022us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:55.314403 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.314553 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:55.314622 (+    69us) raft_consensus.cc:1685] finished
0709 13:08:55.314633 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.314885 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6371,"raft.queue_time_us":9706,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":68,"thread_start_us":99093,"threads_started":1,"wal-append.queue_time_us":41289}
I20260709 13:08:55.354125 22897 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: []
W20260709 13:08:55.355646 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.355407 27528 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:55.357978 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1854) took 1335 ms. Trace:
I20260709 13:08:55.359975 22897 rpcz_store.cc:276] 0709 13:08:54.020392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.352705 (+1332313us) service_pool.cc:224] Handling call
0709 13:08:55.353306 (+   601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.354489 (+  1183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.354656 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:55.365067 22900 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:55.365777 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1856) took 1344 ms. Trace:
I20260709 13:08:55.366104 22900 rpcz_store.cc:276] 0709 13:08:54.021290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.353771 (+1332481us) service_pool.cc:224] Handling call
0709 13:08:55.364802 (+ 11031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.365526 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.365712 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:55.369545 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:55.377065 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1478) took 1471 ms. Trace:
I20260709 13:08:55.377908 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1999) took 1341 ms. Trace:
I20260709 13:08:55.378381 22822 rpcz_store.cc:276] 0709 13:08:54.036664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.095601 (+ 58937us) service_pool.cc:224] Handling call
0709 13:08:54.225881 (+130280us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b5800144f98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:08:55.377486 (+1151605us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b5400231ed4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:55.377718 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1129891456}
I20260709 13:08:55.378916 27308 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:55.382947 22825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 1997) took 1347 ms (client timeout 1657 ms). Trace:
W20260709 13:08:55.383360 22825 rpcz_store.cc:269] 0709 13:08:54.035001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.035821 (+   820us) service_pool.cc:224] Handling call
0709 13:08:55.382557 (+1346736us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b540008b454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:55.382770 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691993472}
I20260709 13:08:55.385506 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 1995) took 1353 ms. Trace:
I20260709 13:08:55.385859 22827 rpcz_store.cc:276] 0709 13:08:54.032235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.135802 (+103567us) service_pool.cc:224] Handling call
0709 13:08:54.159834 (+ 24032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.230852 (+ 71018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:54.230899 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:54.232405 (+  1506us) log.cc:844] Serialized 26 byte log entry
0709 13:08:54.496900 (+264495us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:54.496937 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.384874 (+887937us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b54002f12d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:55.384933 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:55.385000 (+    67us) raft_consensus.cc:1685] finished
0709 13:08:55.385013 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.385337 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15628,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":2191,"spinlock_wait_cycles":120822656,"thread_start_us":242338,"threads_started":1,"wal-append.queue_time_us":469215}
I20260709 13:08:55.394613 27842 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:55.395391 27842 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.397781 27842 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.397927 22897 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:55.398583 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1859) took 1373 ms. Trace:
I20260709 13:08:55.398921 22897 rpcz_store.cc:276] 0709 13:08:54.025549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.397436 (+1371887us) service_pool.cc:224] Handling call
0709 13:08:55.397771 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.398332 (+   561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.398512 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:55.400508 22897 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:08:55.402762 28031 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.377562 22749 rpcz_store.cc:276] 0709 13:08:53.905921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.983775 (+ 77854us) service_pool.cc:224] Handling call
0709 13:08:53.984083 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.701143 (+717060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.375694 (+674551us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b540032a854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:55.376840 (+  1146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1644836224}
I20260709 13:08:55.401114 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1862) took 1364 ms. Trace:
I20260709 13:08:55.420910 22897 rpcz_store.cc:276] 0709 13:08:54.036308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.399724 (+1363416us) service_pool.cc:224] Handling call
0709 13:08:55.400135 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.400879 (+   744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.401065 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:55.422688 27528 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:55.423512 27528 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.426084 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.426651 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1218) took 1557 ms. Trace:
I20260709 13:08:55.427161 22901 rpcz_store.cc:276] 0709 13:08:53.869219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.013868 (+144649us) service_pool.cc:224] Handling call
0709 13:08:54.026594 (+ 12726us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.451618 (+425024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.426060 (+974442us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b54001b6f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:55.426452 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976071936}
W20260709 13:08:55.431703 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:55.436328 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.437666 28030 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.439194 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3123) took 1560 ms. Trace:
I20260709 13:08:55.439601 22750 rpcz_store.cc:276] 0709 13:08:53.878437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.878995 (+   558us) service_pool.cc:224] Handling call
0709 13:08:53.985436 (+106441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:53.986133 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.438816 (+1452683us) spinlock_profiling.cc:243] Waited 64.2 ms on lock 0x7b540011a8d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:55.439030 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147591040}
I20260709 13:08:55.445633 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 949) took 1428 ms. Trace:
I20260709 13:08:55.446022 23453 rpcz_store.cc:276] 0709 13:08:54.016906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.017045 (+   139us) service_pool.cc:224] Handling call
0709 13:08:54.028203 (+ 11158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:54.579085 (+550882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.445125 (+866040us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b540024ca54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:55.445416 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255218048}
I20260709 13:08:55.447708 27842 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.448367 27842 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.450726 27842 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:55.452243 22898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1858) took 1427 ms (client timeout 1809 ms). Trace:
W20260709 13:08:55.452711 22898 rpcz_store.cc:269] 0709 13:08:54.025008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.140233 (+115225us) service_pool.cc:224] Handling call
0709 13:08:55.451803 (+1311570us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54003eab54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:55.452038 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839222528}
I20260709 13:08:55.454170 28030 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.455058 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1219) took 1585 ms. Trace:
I20260709 13:08:55.455549 22899 rpcz_store.cc:276] 0709 13:08:53.869458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.280310 (+410852us) service_pool.cc:224] Handling call
0709 13:08:55.007865 (+727555us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b5800211598. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6eee 0000000000375939 00007fcb79d4a1e0 00007fcb79e130c4 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:08:55.030208 (+ 22343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.030437 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.030475 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:55.030526 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:55.030581 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.030651 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.454672 (+424021us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b5400071a50. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:55.454846 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65068672}
I20260709 13:08:55.461242 28030 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:55.462015 28030 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.463553 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1221) took 1593 ms. Trace:
I20260709 13:08:55.464146 28030 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.468204 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:55.470084 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2456) took 1642 ms. Trace:
I20260709 13:08:55.470338 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:55.470571 22905 rpcz_store.cc:276] 0709 13:08:53.827583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.828046 (+   463us) service_pool.cc:224] Handling call
0709 13:08:54.202587 (+374541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:54.234462 (+ 31875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.469638 (+1235176us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540034c1d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:55.469878 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860661632}
I20260709 13:08:55.471419 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:55.473493 28031 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:55.474397 28031 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:55.473901 27573 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.474982 28031 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:55.475885 27694 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:55.491062 27694 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:55.491611 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 960) took 1155 ms. Trace:
I20260709 13:08:55.491772 28031 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.492417 27694 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.493973 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.477273 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:55.495169 28039 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:55.496017 27730 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:55.495903 28039 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:55.496788 27730 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.358223 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.498088 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:55.498831 22751 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:55.499354 27730 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.463923 22904 rpcz_store.cc:276] 0709 13:08:53.870031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.281585 (+411554us) service_pool.cc:224] Handling call
0709 13:08:54.281834 (+   249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.557973 (+276139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.463011 (+905038us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b5400274f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:55.463372 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":547848448}
I20260709 13:08:55.499733 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.500881 23451 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 15: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:55.501044 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.502774 28039 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:55.505009 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:55.509184 23446 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:55.487610 27370 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:55.506122 28102 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:55.511185 28102 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:55.511716 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.511852 27817 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.506378 28029 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:55.506632 27528 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:55.492141 23444 rpcz_store.cc:276] 0709 13:08:54.336055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.337036 (+   981us) service_pool.cc:224] Handling call
0709 13:08:55.491135 (+1154099us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400567cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:55.491324 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403117440}
I20260709 13:08:55.508198 27797 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:55.516625 28102 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.518074 27893 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.518383 27985 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:55.518841 27893 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.519399 27871 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.519289 27985 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.538182 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.505158 27856 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:08:55.542095 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.516871 27797 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:55.554373 22897 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:08:55.554869 22897 pending_rounds.cc:85] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 10
I20260709 13:08:55.555100 22897 pending_rounds.cc:107] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:08:55.555390 22897 raft_consensus.cc:2889] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:55.556743 22828 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:08:55.557555 27893 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.562348 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:55.562749 27573 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:55.563090 22830 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:55.566574 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.567790 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.568365 22750 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:55.569692 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1863) took 1532 ms. Trace:
I20260709 13:08:55.537485 27985 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.570060 22897 rpcz_store.cc:276] 0709 13:08:54.037550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.538888 (+1501338us) service_pool.cc:224] Handling call
0709 13:08:55.553782 (+ 14894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.569432 (+ 15650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.569604 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:55.570560 27985 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:55.571544 22897 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:55.571435 27985 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.572182 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1227) took 1138 ms. Trace:
I20260709 13:08:55.572510 22897 rpcz_store.cc:276] 0709 13:08:54.433468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.570843 (+1137375us) service_pool.cc:224] Handling call
0709 13:08:55.571280 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.571974 (+   694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.572139 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:55.573557 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.574398 22749 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:55.581604 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.582373 22752 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:55.582657 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:55.583381 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.584327 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:55.584597 22752 raft_consensus.cc:2410] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:08:55.585677 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.585806 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.586510 22752 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 7.
I20260709 13:08:55.586683 22747 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:55.585861 27856 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.590510 27856 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.590407 27797 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.591746 27797 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:55.593204 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.593629 27910 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.594103 22749 raft_consensus.cc:2410] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:08:55.594226 27910 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:55.595276 27985 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.597669 28009 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:55.598507 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.586688 27822 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:55.606076 27822 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:55.606374 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.537490 28029 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:55.607388 27822 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.608809 27822 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:55.608358 28029 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:55.589306 28082 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.663s	user 0.002s	sys 0.000s
I20260709 13:08:55.611650 23444 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:55.583612 22755 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:55.615864 22827 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:55.615918 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:55.617738 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.618460 22823 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:55.613456 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.620288 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.620149 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:55.625963 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.595286 27506 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:55.651470 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:55.656572 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.663180 27927 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.663818 27862 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.664289 27793 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.664669 27862 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.666946 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:55.667714 27862 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:55.669653 28102 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:55.669857 23449 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:55.675350 22905 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:55.675588 22899 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:08:55.676193 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1224) took 1450 ms. Trace:
I20260709 13:08:55.676492 22899 rpcz_store.cc:276] 0709 13:08:54.225437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.621886 (+1396449us) service_pool.cc:224] Handling call
0709 13:08:55.643441 (+ 21555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.675937 (+ 32496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.676147 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:55.678970 22899 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:55.679798 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:55.657341 22755 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:55.664994 27793 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.688390 27793 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.697180 22896 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:08:55.697575 22896 pending_rounds.cc:85] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 11
I20260709 13:08:55.697850 22896 pending_rounds.cc:107] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:08:55.698112 22896 raft_consensus.cc:2889] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:55.701740 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1481) took 1718 ms. Trace:
I20260709 13:08:55.702098 22753 rpcz_store.cc:276] 0709 13:08:53.983713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.201869 (+218156us) service_pool.cc:224] Handling call
0709 13:08:54.250596 (+ 48727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.251146 (+   550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.701379 (+1450233us) spinlock_profiling.cc:243] Waited 48.5 ms on lock 0x7b54000780d0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:55.701577 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111490816}
W20260709 13:08:55.633504 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48241.9us
I20260709 13:08:55.716280 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:55.716218 27962 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:08:55.722821 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 53969.1us
I20260709 13:08:55.729940 23446 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:55.734747 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:55.753010 22830 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:08:55.754179 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:55.756368 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.757074 23449 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:08:55.760707 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.762066 27910 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:55.663846 27927 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.770597 22753 raft_consensus.cc:2393] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:55.770733 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.771378 22823 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:55.772225 27506 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:55.772807 27506 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.774901 27506 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.775535 27506 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:55.775879 27506 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.776223 27506 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:55.780141 28035 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.780710 28035 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.782769 28035 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.802420 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.803598 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.804060 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.804673 22752 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:08:55.805011 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:55.805505 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.805466 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.806829 22751 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:55.808339 22747 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:55.805025 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.810283 22756 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:55.811504 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:55.813746 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:55.814782 22753 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:55.817971 27956 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:55.818455 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.818722 27956 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.820259 27431 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.820809 27431 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.822490 27956 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.823464 27431 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:55.824463 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1216) took 1955 ms. Trace:
I20260709 13:08:55.842162 22902 rpcz_store.cc:276] 0709 13:08:53.868878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.957519 (+ 88641us) service_pool.cc:224] Handling call
0709 13:08:54.013469 (+ 55950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:54.168275 (+154806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:54.168322 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:54.168378 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:54.168435 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.824063 (+1655628us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400059bd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:55.824118 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.824303 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355540992}
I20260709 13:08:55.843067 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.825309 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3131) took 1921 ms. Trace:
I20260709 13:08:55.827425 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.845371 28045 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.847726 22904 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:55.844012 22828 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:55.848306 22753 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:08:55.843581 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.850406 22902 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:55.828691 28023 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:55.828776 27981 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:08:55.829367 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:55.851931 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.832821 27573 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:55.852427 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:55.852829 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.852582 27573 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.854454 28035 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:55.854737 27573 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:55.834126 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.855379 28037 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.855908 28037 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.856242 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.856899 22905 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:55.827327 27834 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:55.865873 27834 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:55.866230 27834 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:55.866706 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1222) took 1996 ms. Trace:
I20260709 13:08:55.836508 27911 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:55.836846 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:55.867148 23446 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:55.867390 27834 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.869117 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:55.870379 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.867082 22896 rpcz_store.cc:276] 0709 13:08:53.870370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.147008 (+1276638us) service_pool.cc:224] Handling call
0709 13:08:55.147349 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.698641 (+551292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.866373 (+167732us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54003bf6d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:55.866557 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263066496}
I20260709 13:08:55.871171 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:55.871816 27911 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:55.872417 27928 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:55.873047 27928 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:55.873123 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.874078 22747 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:55.874261 27928 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.848516 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1226) took 1434 ms. Trace:
I20260709 13:08:55.869001 27834 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:55.879587 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:55.848326 28045 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.882028 28045 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.851857 27981 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:08:55.835744 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 1.82273e+06us
I20260709 13:08:55.871546 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:55.882889 22825 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:55.883288 22896 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:55.883430 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:55.884276 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.884932 22896 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:55.847292 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.830031 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.885704 22823 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:55.847807 22748 rpcz_store.cc:276] 0709 13:08:53.903329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.926724 (+ 23395us) service_pool.cc:224] Handling call
0709 13:08:53.926976 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:53.987864 (+ 60888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.824887 (+1837023us) spinlock_profiling.cc:243] Waited 54.7 ms on lock 0x7b54002a9754. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:55.825142 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125879680}
I20260709 13:08:55.882681 27981 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:55.885171 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:55.887439 27981 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.888341 27573 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:55.861717 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.889927 27694 raft_consensus.cc:993] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:08:55.890688 28045 raft_consensus.cc:1081] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:08:55.891765 27981 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:55.876174 22904 rpcz_store.cc:276] 0709 13:08:54.414412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.846597 (+1432185us) service_pool.cc:224] Handling call
0709 13:08:55.847177 (+   580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.848301 (+  1124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.848472 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:55.890594 22871 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:55.876226 27528 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:55.879794 28032 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.886072 23449 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:55.900915 22904 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:08:55.855100 28035 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:55.769624 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53500.7us
I20260709 13:08:55.857223 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:55.891983 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:55.838083 27927 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:55.864986 28037 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.877393 22905 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:55.900266 27528 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.882529 27848 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:55.901911 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:55.906766 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 52538.9us
I20260709 13:08:55.903899 27573 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:55.912163 23453 raft_consensus.cc:2410] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:08:55.906958 27848 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:55.916734 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:55.917058 27848 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.918282 27848 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:55.919183 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:55.919884 27985 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:55.920043 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:55.920286 27985 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:55.907891 27573 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.920710 27985 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:55.920992 27573 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:55.917632 28037 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:55.922312 28037 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.922688 28037 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:55.922881 27717 raft_consensus.cc:2764] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:08:55.925094 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.933204 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.937376 22748 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:08:55.937688 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:55.906811 27528 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:55.937136 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.942006 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:55.934059 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.931640 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:55.941417 23444 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:55.937376 27838 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:55.949788 27838 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:55.950721 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58871.1us
I20260709 13:08:55.951143 27717 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:55.914898 27370 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:55.951740 27717 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:55.953743 22902 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:08:55.956319 23449 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:55.959679 27717 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:55.963653 22652 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33010 (request call id 226) took 16867 ms (client timeout 15000 ms). Trace:
I20260709 13:08:55.965801 22752 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:55.966521 28035 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:55.967645 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.968796 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.969103 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:55.969959 22823 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:55.970754 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:55.973063 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.974416 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.931373 28032 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:55.975132 22752 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:55.976424 28032 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:55.978361 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:55.980587 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.981207 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:55.982402 22747 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:08:55.982834 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.984723 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.986831 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.989589 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:55.990326 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.990633 28024 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:08:55.990891 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:55.991998 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.992363 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:55.993046 27831 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:55.990819 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:55.993861 22905 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:08:55.994228 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.994958 23451 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:55.994899 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:55.995810 22905 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:55.993451 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:55.997812 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.998831 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:55.999107 27910 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:55.999465 22905 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:56.000216 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:08:56.000823 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 93709.7us
I20260709 13:08:56.000733 22905 raft_consensus.cc:2376] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11. Re-sending same reply.
I20260709 13:08:56.003180 22871 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:56.003851 27370 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:56.004336 27370 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:56.004768 27370 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:55.993523 27831 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:55.928144 27431 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:56.008603 27831 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:56.008690 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:56.009589 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.000401 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:56.015796 22752 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 13:08:56.064850 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.011051 27431 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.054189 27797 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.074173 22905 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:56.075255 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:56.078353 27924 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.078917 27924 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.084327 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.084256 28037 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.084990 23449 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:56.085362 22751 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:56.075022 27797 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.073057 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.062012 27694 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:56.091007 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:56.094122 23450 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:56.081079 27924 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:56.099671 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:56.069067 27431 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:56.103672 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 83269.4us
I20260709 13:08:56.093235 27822 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:56.115892 22905 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:08:56.096819 27694 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.117949 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:56.127126 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:56.108867 27956 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:56.095057 28032 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:08:56.115350 27822 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:56.145687 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.143747 22902 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:08:56.127987 27694 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:56.156809 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:56.158140 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:56.119311 27797 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:56.149704 28113 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:08:55.964057 22652 rpcz_store.cc:269] 0709 13:08:39.197982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.207611 (+  9629us) service_pool.cc:224] Handling call
0709 13:08:41.157380 (+1949769us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54000ab4d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d65090 00007fcb74e03f3a 00007fcb7abba1cc 00007fcb7acb837e 00007fcb791792a4 00007fcb79179232 00007fcb79179161 00007fcb791790dc 00007fcb79178392 00007fcb7313648c 00007fcb731358c6
0709 13:08:55.963375 (+14805995us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:50.580227 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:50.583703 (+  3476us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:50.583755 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:08:50.583782 (+    27us) tablet.cc:662] Decoding operations
0709 13:08:50.585455 (+  1673us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:50.585558 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:50.585598 (+    40us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:08:50.588182 (+  2584us) tablet.cc:701] Row locks acquired
0709 13:08:50.598199 (+ 10017us) write_op.cc:260] PREPARE: finished
0709 13:08:50.617998 (+ 19799us) write_op.cc:270] Start()
0709 13:08:50.635062 (+ 17064us) write_op.cc:276] Timestamp: P: 1783602530605897 usec, L: 0
0709 13:08:50.641665 (+  6603us) op_driver.cc:348] REPLICATION: starting
0709 13:08:50.740111 (+ 98446us) log.cc:844] Serialized 4776 byte log entry
0709 13:08:51.269262 (+529151us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:08:51.600426 (+331164us) op_driver.cc:464] REPLICATION: finished
0709 13:08:51.728866 (+128440us) write_op.cc:301] APPLY: starting
0709 13:08:51.728978 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:52.115844 (+386866us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:52.115875 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:52.394860 (+278985us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:52.464595 (+ 69735us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:08:52.473878 (+  9283us) write_op.cc:312] APPLY: finished
0709 13:08:52.569570 (+ 95692us) log.cc:844] Serialized 113 byte log entry
0709 13:08:52.651661 (+ 82091us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:52.654336 (+  2675us) write_op.cc:454] Released schema lock
0709 13:08:52.659840 (+  5504us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:52.680809 (+ 20969us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:52.681705 (+   896us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":48136,"reported_tablets":41,"spinlock_wait_cycles":662432000,"child_traces":[["op",{"apply.queue_time_us":116706,"apply.run_cpu_time_us":37779,"apply.run_wall_time_us":941180,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54287,"num_ops":12,"prepare.queue_time_us":165299,"prepare.run_cpu_time_us":24766,"prepare.run_wall_time_us":447512,"raft.queue_time_us":175485,"raft.run_cpu_time_us":17386,"raft.run_wall_time_us":146860,"replication_time_us":947096,"spinlock_wait_cycles":32195712,"thread_start_us":608296,"threads_started":5,"wal-append.queue_time_us":464888,"wal-append.run_cpu_time_us":36093,"wal-append.run_wall_time_us":2533970}]]}
I20260709 13:08:56.167111 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:56.146111 27956 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.146684 27927 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.129542 28035 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:08:56.148303 27893 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.147578 28045 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:56.118667 27985 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.158764 27822 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:56.122370 27911 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.163977 22902 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:56.213178 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:56.218642 22750 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:08:56.182601 27956 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:56.105265 27838 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.163029 27834 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:08:56.163038 27910 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:08:56.229508 22905 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
W20260709 13:08:56.234705 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88184.7us
I20260709 13:08:56.253844 28019 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:08:56.254565 28114 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:08:56.254645 28019 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:56.258687 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 69720.6us
I20260709 13:08:56.259531 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:56.260308 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:56.260933 27693 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.186198 27817 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:56.270253 28102 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.270918 28102 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:56.273272 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.275290 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:56.291221 28035 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.291946 28031 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:56.292500 28031 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:56.292668 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1865) took 2106 ms. Trace:
I20260709 13:08:56.293011 28031 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:56.293095 22901 rpcz_store.cc:276] 0709 13:08:54.186329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.631910 (+1445581us) service_pool.cc:224] Handling call
0709 13:08:55.673595 (+ 41685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.674174 (+   579us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.674221 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:55.674839 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 13:08:55.895297 (+220458us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:55.895339 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.895417 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:56.292461 (+397044us) raft_consensus.cc:1685] finished
0709 13:08:56.292478 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:56.292619 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12380,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":159,"thread_start_us":220043,"threads_started":1,"wal-append.queue_time_us":220489}
I20260709 13:08:56.291802 28035 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.297775 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.299314 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1864) took 2113 ms. Trace:
I20260709 13:08:56.299726 22903 rpcz_store.cc:276] 0709 13:08:54.186023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.823560 (+1637537us) service_pool.cc:224] Handling call
0709 13:08:55.823972 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.824395 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.824430 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:55.824907 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:08:56.145689 (+320782us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:56.145730 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.145790 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:56.298954 (+153164us) raft_consensus.cc:1685] finished
0709 13:08:56.298976 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:56.299113 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":502,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":320361,"threads_started":1,"wal-append.queue_time_us":403787}
I20260709 13:08:56.301946 27834 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:08:56.302171 28099 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:08:56.302343 27834 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:56.303333 27834 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.304512 27834 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:56.305914 28111 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:08:56.312839 22905 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:56.349141 27928 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.218782 28045 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:56.224471 28029 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:08:56.274524 27817 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:08:56.417326 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.248921 27893 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:56.218505 27927 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:56.427420 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.432548 27928 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:56.417110 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 102115us
I20260709 13:08:56.368027 28045 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:56.456161 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:56.448506 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:56.450162 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:56.457523 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 198088us
I20260709 13:08:56.461957 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:56.463727 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 338032us
I20260709 13:08:56.343623 28030 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:56.472285 23449 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:56.406776 28037 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.290551 22654 catalog_manager.cc:5697] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 9 to 11, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 11 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:56.432246 28102 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:56.310089 27693 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:56.487955 22751 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:56.492792 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:56.492892 22654 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 7 to 8, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 8 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:56.502135 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46049.1us
I20260709 13:08:56.509634 27693 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:56.523736 27956 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.530570 27927 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:56.536664 27985 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.538961 27928 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:56.541654 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.549934 27911 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:56.553208 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:56.554786 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 68399.3us
W20260709 13:08:56.556063 22825 consensus_meta.cc:287] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.673s	user 0.006s	sys 0.013s
I20260709 13:08:56.556491 22825 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:56.562950 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:08:56.564622 22904 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.579s	user 0.007s	sys 0.011s
I20260709 13:08:56.565414 22904 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:08:56.487289 27962 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:56.488523 28036 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:56.582229 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:56.587185 22904 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:56.528324 22654 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 11, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 11 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.585284 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.580161 28023 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.590744 28023 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.592746 28023 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.569612 27717 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:56.582573 27985 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.593694 27985 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:56.595522 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1228) took 2151 ms. Trace:
I20260709 13:08:56.595650 27985 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:56.595892 22897 rpcz_store.cc:276] 0709 13:08:54.443871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.573186 (+1129315us) service_pool.cc:224] Handling call
0709 13:08:55.639785 (+ 66599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.735839 (+ 96054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.735884 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:55.755832 (+ 19948us) log.cc:844] Serialized 26 byte log entry
0709 13:08:56.125572 (+369740us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:08:56.125624 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.126292 (+   668us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:56.595344 (+469052us) raft_consensus.cc:1685] finished
0709 13:08:56.595365 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:56.595473 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":15723,"thread_start_us":291596,"threads_started":1,"wal-append.queue_time_us":760254}
I20260709 13:08:56.596968 22902 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:08:56.599851 27506 consensus_meta.cc:287] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.823s	user 0.008s	sys 0.026s
I20260709 13:08:56.600273 27506 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.601454 28036 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.602662 23449 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:56.603567 28036 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.612553 27506 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:56.614967 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.634310 27793 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:56.634963 27793 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.637574 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1225) took 2402 ms. Trace:
I20260709 13:08:56.638548 28031 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.637950 22898 rpcz_store.cc:276] 0709 13:08:54.234971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.848972 (+1614001us) service_pool.cc:224] Handling call
0709 13:08:55.849296 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.850040 (+   744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.850073 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:55.850733 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 13:08:56.099485 (+248752us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:56.099536 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.099621 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:56.637370 (+537749us) raft_consensus.cc:1685] finished
0709 13:08:56.637389 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:56.637513 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":104,"thread_start_us":247738,"threads_started":1,"wal-append.queue_time_us":754264}
I20260709 13:08:56.640565 28031 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.641755 28035 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.642491 28035 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.643010 28035 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.578083 28037 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:08:56.632774 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:56.651084 22902 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:56.676520 22748 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:56.679744 22896 consensus_meta.cc:287] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.766s	user 0.005s	sys 0.014s
I20260709 13:08:56.680147 22896 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:56.680409 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:56.649510 27848 consensus_meta.cc:287] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.731s	user 0.008s	sys 0.004s
I20260709 13:08:56.618175 27985 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:56.689409 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:56.691167 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:56.681046 22752 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:56.645654 27893 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:56.694005 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.692101 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:56.648043 28019 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.645181 28035 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.697925 23450 raft_consensus.cc:2410] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:08:56.699499 22654 catalog_manager.cc:5697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 11, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:56.646564 22904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43998
I20260709 13:08:56.708181 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.708786 23447 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:56.710202 22654 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 13, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 13 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:56.717605 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:56.707103 27573 consensus_meta.cc:287] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.786s	user 0.007s	sys 0.012s
I20260709 13:08:56.724752 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:56.727200 22753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.016s	sys 0.019s
W20260709 13:08:56.729209 22753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.017s	sys 0.019s
I20260709 13:08:56.729763 23446 raft_consensus.cc:1217] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Deduplicated request from leader. Original: 10.10->[13.11-14.12]   Dedup: 14.12->[]
I20260709 13:08:56.729923 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.730525 22654 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:56.723712 27793 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:56.749830 23446 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: [13.11-14.12]
I20260709 13:08:56.733196 22902 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:08:56.757743 23444 consensus_meta.cc:287] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.808s	user 0.017s	sys 0.003s
W20260709 13:08:56.758419 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 498138us
I20260709 13:08:56.763415 23444 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
W20260709 13:08:56.760610 22899 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.023s	sys 0.043s
I20260709 13:08:56.722611 28019 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:56.765949 22899 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.023s	sys 0.043s
I20260709 13:08:56.762405 22654 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 6 to 10, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 10 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:56.770241 22654 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 14, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 14 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.737766 27956 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:56.726171 28045 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.782018 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:56.775405 28019 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.784180 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.783627 22654 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 14 to 15, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 15 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.786480 28019 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.786072 22654 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 13 to 16, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 16 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.786980 28019 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.787461 28019 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:56.787917 22654 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 13, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 13 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:56.777496 27797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.016s	sys 0.028s
W20260709 13:08:56.789665 27797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.016s	sys 0.028s
I20260709 13:08:56.789770 27793 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.790253 27793 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.792770 27793 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:56.794551 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:56.796010 28019 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:56.796708 28019 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.797405 28019 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:56.802119 28129 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:56.802527 22654 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 10, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 10 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:56.803925 22654 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 14, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 14 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:56.805107 28029 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:56.805693 28029 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.805641 22654 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 7 to 9, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 9 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.807825 28029 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:56.807438 22654 catalog_manager.cc:5697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:56.809116 28029 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:56.810290 28029 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.812798 28029 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:56.773967 27822 consensus_meta.cc:287] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 13:08:56.731876 27573 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:56.816980 23419 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:56.815420 27981 consensus_meta.cc:287] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.923s	user 0.010s	sys 0.010s
I20260709 13:08:56.851770 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:56.853578 22828 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:56.852366 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:56.855576 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:56.856104 22753 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:08:56.848747 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:56.859131 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:56.808734 22654 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 13, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 13 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:56.861629 22823 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:56.863260 23451 raft_consensus.cc:2410] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:08:56.863747 22898 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:08:56.859856 22750 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:08:56.868207 22654 catalog_manager.cc:5697] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 4 to 9, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 9 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:56.868440 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:56.872839 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.875442 22748 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:08:56.876636 22827 raft_consensus.cc:2410] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:08:56.873474 23450 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:56.876588 22654 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:56.848690 27962 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:56.883044 28019 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:56.883850 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:56.883453 22654 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 12 to 15, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 15 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.884485 22829 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:08:56.884756 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2005) took 1025 ms. Trace:
I20260709 13:08:56.885118 22825 rpcz_store.cc:276] 0709 13:08:55.859530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.879304 (+ 19774us) service_pool.cc:224] Handling call
0709 13:08:56.884593 (+1005289us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b54002c9a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:56.884685 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5872000}
I20260709 13:08:56.885653 22654 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
W20260709 13:08:56.886586 27431 consensus_meta.cc:287] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.763s	user 0.010s	sys 0.008s
I20260709 13:08:56.887044 27431 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.887358 22654 catalog_manager.cc:5697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:56.891938 27817 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.898438 23453 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
I20260709 13:08:56.899067 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:56.901139 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:56.929726 22825 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:08:56.874413 28129 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:56.847191 27822 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:56.888872 22654 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 15, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 15 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:56.878875 27962 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.907428 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:56.932276 22756 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:56.889034 27431 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:56.925231 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 466931us
I20260709 13:08:56.951293 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:56.950946 22654 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 14, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 14 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.923520 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:56.953107 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:56.948628 27962 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:56.962972 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.964144 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:56.924634 28031 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:56.965487 22825 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:56.967346 22749 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:56.966320 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:56.968261 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 463917us
I20260709 13:08:56.969967 22752 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 13:08:56.969988 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:56.974534 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.977015 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:56.981658 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:56.982607 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 427190us
I20260709 13:08:56.981698 22654 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 11, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:56.984990 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:57.022330 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:56.966262 27817 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:57.027164 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:57.060942 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:08:56.970140 27838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.016s	sys 0.016s
I20260709 13:08:56.989245 22754 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:56.980736 28134 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:57.069504 22654 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 8 to 11, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:08:57.071898 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:56.985464 27528 consensus_meta.cc:287] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.985s	user 0.009s	sys 0.013s
W20260709 13:08:56.988123 27694 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.776s	user 0.001s	sys 0.039s
I20260709 13:08:57.056527 28023 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:56.980134 28037 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:57.073629 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:56.987095 27431 consensus_queue.cc:1059] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
W20260709 13:08:57.080307 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 129074us
I20260709 13:08:57.081066 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.084245 22829 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:57.084748 22654 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 12 to 14, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 14 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:08:56.975513 28031 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.075364 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:57.093921 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1236) took 1083 ms. Trace:
I20260709 13:08:57.095166 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.094720 22903 rpcz_store.cc:276] 0709 13:08:56.010541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.309552 (+299011us) service_pool.cc:224] Handling call
0709 13:08:56.349647 (+ 40095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:56.349955 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:56.750371 (+400416us) log.cc:844] Serialized 14 byte log entry
0709 13:08:57.093589 (+343218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:57.093639 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:57.093697 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:57.093758 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:57.093874 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14417,"prepare.queue_time_us":264437,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":703755,"threads_started":2}
I20260709 13:08:57.056350 27985 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:08:57.075161 27730 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:08:57.097080 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 69585.9us
I20260709 13:08:57.083091 27910 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:08:57.101711 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 307248us
I20260709 13:08:57.087949 28045 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.102411 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:57.102326 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2481) took 1112 ms. Trace:
I20260709 13:08:57.104730 22904 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:57.108505 22904 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:57.105258 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:57.087416 28023 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:57.086994 27838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.016s	sys 0.018s
W20260709 13:08:57.099776 22869 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.027s	sys 0.044s
I20260709 13:08:57.101480 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1231) took 2229 ms. Trace:
W20260709 13:08:57.115315 22869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.027s	sys 0.044s
I20260709 13:08:57.111210 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.103106 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:57.117619 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:57.118678 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:57.120160 27717 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.120586 27717 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:57.113916 28023 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:57.094478 27573 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:57.116250 22897 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:57.127372 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.097148 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:57.114861 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.130750 22828 raft_consensus.cc:2410] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:08:57.131839 22824 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:57.108176 22901 rpcz_store.cc:276] 0709 13:08:55.990145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.294734 (+304589us) service_pool.cc:224] Handling call
0709 13:08:56.295046 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:56.295236 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:56.680022 (+384786us) log.cc:844] Serialized 14 byte log entry
0709 13:08:57.101812 (+421790us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:57.101887 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:57.101968 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:57.102088 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:57.102258 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6689,"prepare.queue_time_us":241945,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":9777,"thread_start_us":773867,"threads_started":2}
I20260709 13:08:57.098529 27834 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:57.137368 22756 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:08:57.099355 27730 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.086596 27528 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:57.103646 27910 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:57.091595 27694 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.144805 22902 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:57.144798 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.179064 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.179149 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.179422 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:57.143410 27730 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:57.180663 27370 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.175s	user 0.013s	sys 0.009s
I20260709 13:08:57.145943 28140 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:57.181692 28140 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:57.146344 28135 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.146873 27804 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:08:57.147666 28130 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:08:57.147883 28133 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:08:57.151754 28023 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.152668 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.153126 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:57.201906 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:57.155122 27717 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:57.156560 28132 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.397s	user 0.002s	sys 0.000s
I20260709 13:08:57.157219 27822 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.157606 27910 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:57.143844 28029 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:08:57.158514 27831 consensus_meta.cc:287] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.097s	user 0.003s	sys 0.019s
I20260709 13:08:57.160274 28031 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.165792 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.166419 28137 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:57.166656 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.213850 22824 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:08:57.168128 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1527) took 1006 ms. Trace:
I20260709 13:08:57.170763 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1426) took 1181 ms. Trace:
I20260709 13:08:57.215987 23453 rpcz_store.cc:276] 0709 13:08:55.989155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.991677 (+  2522us) service_pool.cc:224] Handling call
0709 13:08:57.170561 (+1178884us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b5400730354. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:57.170702 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2071827328}
I20260709 13:08:57.184020 27431 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:57.136863 28037 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.174394 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.181114 27370 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.177630 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.217746 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.223285 22896 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:57.153111 22752 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:57.146081 28019 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:57.185460 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.182984 28140 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.224694 22901 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:57.232426 22750 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:57.237607 27983 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:08:57.195859 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:57.197844 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.230448 27797 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:08:57.242609 28023 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:57.244132 27717 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:57.244547 22754 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:57.248934 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.233704 27962 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.251737 27962 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:57.253403 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 113741us
I20260709 13:08:57.254313 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:57.255129 28030 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:57.255275 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:57.255532 28030 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:57.256403 28030 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:57.199069 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:57.222661 27431 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.258720 27431 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:57.137084 27834 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.259426 27834 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:57.199008 22797 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:08:57.261915 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 62976.3us
I20260709 13:08:57.262754 28031 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:57.262744 28036 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:57.263463 28031 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:57.263873 28036 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:57.264521 28036 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.263890 28031 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:57.265480 28029 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:57.267599 28036 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.272523 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.115484 22899 rpcz_store.cc:276] 0709 13:08:54.871968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.680690 (+808722us) service_pool.cc:224] Handling call
0709 13:08:55.681074 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.747370 (+ 66296us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:08:56.181190 (+433820us) log.cc:844] Serialized 14 byte log entry
0709 13:08:57.096679 (+915489us) log.cc:844] Serialized 14 byte log entry
0709 13:08:57.101162 (+  4483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:57.101211 (+    49us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:08:57.101262 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:57.101336 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:57.101433 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6687,"prepare.queue_time_us":550224,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":31873,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":151,"thread_start_us":1219364,"threads_started":3,"wal-append.queue_time_us":761707}
I20260709 13:08:57.273876 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.199959 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:57.205461 22900 raft_consensus.cc:2368] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13.
I20260709 13:08:57.205475 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:57.207965 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.198616 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:57.274492 28037 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:57.275080 22903 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:57.275604 23449 raft_consensus.cc:2410] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:08:57.181726 27694 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.172087 28073 raft_consensus.cc:3055] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:57.213020 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:57.276628 28073 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:57.215667 22749 rpcz_store.cc:276] 0709 13:08:56.161728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.161879 (+   151us) service_pool.cc:224] Handling call
0709 13:08:56.166130 (+  4251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:56.967812 (+801682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:57.167908 (+200096us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b54003381d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:57.168067 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1836652416}
I20260709 13:08:57.212951 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2488) took 1092 ms. Trace:
I20260709 13:08:57.177137 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2487) took 1114 ms. Trace:
W20260709 13:08:57.219628 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 117286us
I20260709 13:08:57.220238 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.278273 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.278756 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:57.195289 27848 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:57.279682 23447 raft_consensus.cc:2410] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:08:57.279515 27848 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.196233 28135 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.280777 27924 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:57.280997 22749 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:57.111799 28134 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:57.238200 28127 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.307s	user 0.002s	sys 0.000s
I20260709 13:08:57.256403 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:57.272850 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.296543 22829 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:57.273752 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.202541 23450 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:57.186892 22871 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:57.274581 22897 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:57.278831 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 58704.9us
I20260709 13:08:57.281203 27924 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.219336 22825 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:57.179790 22748 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
W20260709 13:08:57.291008 28132 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:57.281646 22902 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:57.277681 22905 rpcz_store.cc:276] 0709 13:08:56.062294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.555636 (+493342us) service_pool.cc:224] Handling call
0709 13:08:56.562762 (+  7126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:56.597765 (+ 35003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:56.597795 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:56.598231 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:08:57.026697 (+428466us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:57.026734 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:57.026795 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:57.176926 (+150131us) raft_consensus.cc:1685] finished
0709 13:08:57.176954 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:57.177082 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118219,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":6202,"thread_start_us":424589,"threads_started":1,"wal-append.queue_time_us":513582}
I20260709 13:08:57.277046 22751 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:08:57.274580 22824 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
W20260709 13:08:57.308034 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 80188.2us
I20260709 13:08:57.277611 22898 rpcz_store.cc:276] 0709 13:08:56.120484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.639642 (+519158us) service_pool.cc:224] Handling call
0709 13:08:56.639936 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:56.872848 (+232912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:57.212735 (+339887us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400184a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:08:57.212902 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508398464}
I20260709 13:08:57.194023 22904 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.221604 28032 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:57.215909 27831 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.294963 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:57.299611 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.221516 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:57.313993 23445 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:57.316403 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:57.281736 28135 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:57.305343 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:57.306326 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50011.4us
I20260709 13:08:57.281594 27848 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:57.277089 27730 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.309305 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:57.293349 22753 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
W20260709 13:08:57.311793 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 124959us
I20260709 13:08:57.294354 28102 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:57.317807 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.277813 28073 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.325657 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2482) took 1335 ms. Trace:
I20260709 13:08:57.307430 27924 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:57.333254 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.330605 22900 rpcz_store.cc:276] 0709 13:08:55.990463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.992090 (+  1627us) service_pool.cc:224] Handling call
0709 13:08:57.322575 (+1330485us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540034b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:57.325567 (+  2992us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2787794816}
I20260709 13:08:57.330847 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:57.307195 27694 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:57.310492 28032 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:57.346112 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.349671 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.350322 22749 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:57.341792 22750 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:57.336167 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:57.351037 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:57.307178 27717 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:57.325124 28036 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:57.322394 27981 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:57.363907 28036 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:57.352418 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.366201 22752 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:57.345465 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.338956 27797 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:57.342728 27983 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:08:57.358913 28145 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:08:57.358719 27834 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.359310 27370 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:57.351238 28129 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:57.359772 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.360039 28137 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:57.361716 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:57.363358 27730 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:57.368392 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:57.335031 28135 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.051s
I20260709 13:08:57.332404 27962 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:57.373735 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:57.427220 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 58904.2us
I20260709 13:08:57.427716 22905 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:08:57.427939 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:57.428778 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.393083 28139 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:57.429796 28139 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:57.430711 28139 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.393852 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.432690 22751 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:57.394366 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:57.427953 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:57.396139 27834 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:57.458418 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.384428 28029 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:08:57.387681 28145 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.390463 27431 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:57.398705 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.400876 28073 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.368074 27911 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:57.401790 28032 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.402926 27528 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:08:57.404161 28134 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.335981 28102 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.376643 27928 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:57.406899 27956 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:57.408025 27983 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.408892 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:57.410411 27804 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:57.351256 28147 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:08:57.412472 27831 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.414211 22900 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:57.419957 22754 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
W20260709 13:08:57.369300 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:57.494539 27928 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.503753 27928 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:57.487337 27431 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:57.414011 28129 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.389012 28036 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.433946 22896 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:57.505244 28129 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:57.463289 22898 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:08:57.508673 22898 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:08:57.494768 27956 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.509752 27956 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:57.465659 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.434165 27848 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:08:57.490327 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.511055 27431 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.511641 22904 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.511889 22755 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:57.470161 27717 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:57.495497 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:57.462985 28145 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:57.498850 22899 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:57.503023 28102 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.515627 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:57.489753 27911 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.523932 27928 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.524422 27911 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:57.525933 27928 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.494298 28032 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:57.527951 22902 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:57.533726 22896 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:08:57.499387 27528 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:57.544322 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:57.541916 22752 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
W20260709 13:08:57.561937 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 237265us
W20260709 13:08:57.562412 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:57.563089 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:57.562892 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 240854us
W20260709 13:08:57.565955 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 345854us
I20260709 13:08:57.511668 27848 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:57.573309 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 237223us
I20260709 13:08:57.545214 27928 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:08:57.566663 22719 connection.cc:256] RPC call timeout handler was delayed by -1.2126799999999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2141s for a response
I20260709 13:08:57.580587 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:57.534924 27370 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.563342 28140 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:57.587015 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:57.513140 27983 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.595120 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:57.595805 22752 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:57.602156 22902 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:57.565011 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:57.574743 27911 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:57.637192 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50411.5us
I20260709 13:08:57.637945 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:57.586555 27817 consensus_meta.cc:287] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.002s
I20260709 13:08:57.641100 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.637933 28134 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:57.582259 27928 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.640796 22797 leader_election.cc:400] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
I20260709 13:08:57.652083 22797 leader_election.cc:403] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:08:57.585969 28154 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:57.652661 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 90015.3us
I20260709 13:08:57.657933 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.587177 27793 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:57.656068 22751 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:57.654316 22756 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:57.665195 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.665818 22824 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:08:57.580165 27528 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.675532 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.621871 27981 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:57.679782 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:57.622287 27694 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:57.657953 27928 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:57.681607 22823 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:08:57.681532 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:57.682729 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:57.683476 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 167931us
I20260709 13:08:57.683604 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:57.677914 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:57.593746 27370 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.670254 28135 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.646983 28153 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:57.704192 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 139270us
I20260709 13:08:57.756333 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:57.763846 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:57.727998 22752 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:57.728586 22828 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:57.765357 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:57.671325 27804 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:57.703475 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:57.728526 22825 raft_consensus.cc:2410] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:08:57.756415 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:57.674650 28134 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:57.652123 27911 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.771749 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:57.666610 27956 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.670619 28030 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.764396 27793 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.798219 22897 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:08:57.672446 28140 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.696394 27431 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.077s
W20260709 13:08:57.803403 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:57.669761 28154 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.696964 28031 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.684500 27848 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.686569 27910 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:57.811450 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 143114us
W20260709 13:08:57.813686 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 110312us
I20260709 13:08:57.841878 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:57.779237 28164 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:57.764773 28135 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.814126 27962 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:57.850739 27962 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.850499 28129 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.851267 27962 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:57.803480 27573 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:57.830345 28166 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:57.810195 27793 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.855545 27793 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:57.859375 22823 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:57.865435 22749 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:57.877936 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:57.879691 22755 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
W20260709 13:08:57.885051 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 202491us
I20260709 13:08:57.890772 27910 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:57.891381 27910 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:57.837778 27981 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.894259 28031 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:57.887194 22822 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:08:57.847306 28164 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.853196 28129 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:57.921218 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 57790.1us
W20260709 13:08:57.923465 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 159682us
I20260709 13:08:57.860368 27431 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:57.956158 22794 leader_election.cc:400] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13.
W20260709 13:08:57.956241 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 298369us
I20260709 13:08:57.956760 22794 leader_election.cc:403] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:57.957341 23416 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:57.968612 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.921624 28164 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:57.946024 27431 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:57.982726 23419 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.885768 27694 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:57.978053 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:57.976518 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:57.937304 28167 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:57.895141 28134 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:57.994746 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:57.955195 27910 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:08:57.958572 28153 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:08:57.976939 28129 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:57.975894 27983 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:57.985139 28035 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
W20260709 13:08:58.009155 28170 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 13:08:57.999747 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 43769.4us
I20260709 13:08:57.987262 28171 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:57.992731 27848 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:58.010710 23419 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:57.992479 27831 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:08:58.019078 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:58.020433 28030 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:58.022012 27831 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:58.022614 27831 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:58.023578 27831 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.025682 28129 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:58.026270 28129 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:58.027168 28129 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:58.034133 28140 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:58.010049 28184 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:58.035611 27956 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.074600 28158 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13.
W20260709 13:08:58.074831 28168 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:08:58.076052 28154 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.009421 28178 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
W20260709 13:08:58.078464 27924 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.001s
I20260709 13:08:58.078963 27924 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.018412 28167 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:08:58.080470 28160 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:08:58.080981 27924 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.081807 23450 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:58.078377 27962 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:58.091416 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:58.095455 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.097245 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.097808 22755 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:08:58.038811 28139 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:58.103958 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:58.109724 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.111899 22756 raft_consensus.cc:2368] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16.
I20260709 13:08:58.104561 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:58.113168 22752 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 17: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:58.110682 22754 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:08:58.103549 28177 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.116394 28137 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:58.116976 28154 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:58.070726 27431 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.122816 27834 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:58.123555 28152 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.356s	user 0.003s	sys 0.000s
W20260709 13:08:58.124279 28145 consensus_meta.cc:287] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.002s
I20260709 13:08:58.127643 28147 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:08:58.132180 28073 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.665s	user 0.006s	sys 0.003s
I20260709 13:08:58.133875 28135 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:58.066402 28171 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.176004 27911 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:58.138456 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 120877us
I20260709 13:08:58.143448 27962 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.144795 22797 leader_election.cc:400] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16.
I20260709 13:08:58.068733 27981 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:58.202071 22797 leader_election.cc:403] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:08:58.174070 22749 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:58.147513 28137 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:58.174201 28154 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:58.168901 28032 consensus_meta.cc:287] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.602s	user 0.003s	sys 0.007s
I20260709 13:08:58.177676 28045 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:58.138046 27928 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.148162 28177 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.169281 28147 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:08:58.185421 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.147226 28166 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:58.204493 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:58.181612 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 186982us
W20260709 13:08:58.209240 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 64575.3us
I20260709 13:08:58.169407 27834 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.221637 27834 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.211498 27693 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:58.225245 28177 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.225728 28154 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.206545 28137 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:58.205922 28045 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:58.184089 28145 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.205374 27981 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:08:58.215271 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:58.229137 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:58.213241 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 335386us
I20260709 13:08:58.226776 28171 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:58.237990 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.240732 28145 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:58.241524 22905 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:58.238353 28179 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:58.242578 27928 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:58.244421 28045 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.245558 28045 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:58.246637 28164 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.247601 28137 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.248627 28164 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.250000 28137 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:58.253813 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 275877us
I20260709 13:08:58.260052 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.263666 22756 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:58.265693 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.269284 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:58.269972 22756 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:58.271195 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:58.274253 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.274701 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:58.275307 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.274801 22749 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:58.271744 28102 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:58.279551 28147 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.280771 28147 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:58.280730 28073 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:58.281780 28073 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.282109 28073 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:58.286068 27370 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:58.289584 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.290314 22750 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:58.295192 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:58.296144 22756 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:58.293120 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:58.289623 27528 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:58.305588 28035 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:58.306322 28035 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.306794 28035 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:58.238010 28166 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.314656 28139 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:58.320628 22794 leader_election.cc:400] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
W20260709 13:08:58.320792 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62449.1us
I20260709 13:08:58.298498 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:58.326527 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 50305.1us
I20260709 13:08:58.323693 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:58.328802 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:08:58.329559 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.330623 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:58.344631 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.343637 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:58.344873 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.254467 27981 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.347208 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.348182 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.348986 22828 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:08:58.349288 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.308947 28073 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.350207 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.363680 22822 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:58.364382 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.334165 27370 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:58.347854 22826 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:58.368510 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:58.369576 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.370071 22755 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:58.371655 27983 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:58.320490 28035 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.372046 27983 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:58.372967 27983 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.373939 28035 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:58.374183 27983 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:58.367501 27817 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:58.375519 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:58.334678 27831 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:58.381268 27431 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:58.385470 22905 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:08:58.385756 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:58.329254 28193 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:58.386579 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:58.386871 28193 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.387318 28193 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:58.388109 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:58.388227 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.387223 23451 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:08:58.388885 22897 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:58.329386 27528 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:58.389688 28194 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:58.389878 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.389629 27528 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.389638 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:58.391101 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.391839 22901 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:58.392311 27528 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:58.392666 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.393234 22903 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:08:58.329869 28184 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:58.396220 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.396972 22902 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:58.389000 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:58.387424 23450 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:08:58.397950 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.398577 22903 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:08:58.390199 28194 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.399001 28073 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.399144 28194 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:58.386927 22900 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:08:58.346168 28032 raft_consensus.cc:2764] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:08:58.343132 28187 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:58.401338 28187 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.401666 28187 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:58.402690 22900 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:58.403692 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.404346 22897 raft_consensus.cc:2368] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11.
I20260709 13:08:58.329360 28164 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 13:08:58.406031 28134 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:58.408681 28129 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:58.408277 28134 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:58.409348 27804 consensus_meta.cc:287] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.000s
I20260709 13:08:58.409737 27804 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
I20260709 13:08:58.404529 27693 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16.
I20260709 13:08:58.406031 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.410960 22753 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:08:58.411193 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 421161us
W20260709 13:08:58.412869 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48593.2us
I20260709 13:08:58.413056 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:58.413620 22747 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:58.414078 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.415124 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:58.396454 28184 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.442422 22901 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:58.421252 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:58.421027 22897 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:08:58.420003 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.418980 22756 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:58.446776 22749 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:58.417233 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:58.434465 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.455513 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:58.457834 22903 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
W20260709 13:08:58.461715 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51707.6us
I20260709 13:08:58.461633 28045 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:58.464660 22824 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:08:58.464754 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57593us
I20260709 13:08:58.465184 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:58.465754 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:58.469611 28179 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:58.469672 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.470068 28179 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.470434 28179 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:58.470752 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:58.471709 28102 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:08:58.472350 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:58.474529 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.490383 22825 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:58.491251 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.492272 22756 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:58.493065 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:58.496062 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:58.499392 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.500109 22904 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:58.501546 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.502745 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.506003 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:58.511567 22904 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:08:58.520823 23444 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:08:58.425058 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.420382 28129 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:58.526628 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:58.528443 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:58.529866 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.530797 22905 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:58.532217 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:58.524956 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.536401 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:58.381556 27831 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.443116 27528 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:58.547571 22826 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:58.540525 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.529072 23453 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:58.429754 28147 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:58.548596 22904 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:58.553740 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.554556 22824 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:58.505801 28186 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:58.558127 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:58.558589 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 141427us
I20260709 13:08:58.559433 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.529618 27693 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.550050 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:58.564491 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:58.566397 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:58.567106 23451 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:58.568717 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.568854 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.570005 22900 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:58.571128 22828 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:58.541463 27831 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:58.420074 27370 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.572890 22899 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:58.574394 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:08:58.576133 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.579595 22824 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:58.581167 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:58.589766 23453 raft_consensus.cc:2368] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11.
I20260709 13:08:58.558348 28186 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.590404 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.590615 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:58.555938 27928 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:58.591012 22826 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:58.603154 22905 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
W20260709 13:08:58.614358 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55027.4us
I20260709 13:08:58.614714 23444 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:58.616842 22904 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
W20260709 13:08:58.619917 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.589058 28167 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:08:58.622830 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.591130 28134 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:58.629431 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:58.576612 27693 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.558288 28157 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:08:58.607847 27910 raft_consensus.cc:3055] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:58.591981 28035 raft_consensus.cc:2764] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:08:58.648140 22822 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:08:58.649137 23451 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:58.649556 22824 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:08:58.604781 28186 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:58.665130 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.624296 28102 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:58.672061 22747 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:58.686596 22899 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
W20260709 13:08:58.669957 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 119956us
I20260709 13:08:58.707158 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.703439 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:58.720655 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.721495 22904 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:58.645776 28179 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.725274 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.727842 22904 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:08:58.679957 27910 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:58.722865 28147 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.732367 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.737581 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:58.712191 28201 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:08:58.748070 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.749219 27910 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.751701 27910 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:58.750676 28149 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:08:58.718909 27848 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.753515 27848 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.754443 27693 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.755596 27848 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:58.756083 28193 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.757431 28179 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.757815 27370 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:58.759714 28194 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.761256 27528 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:58.724079 28160 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:08:58.762799 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.746556 27817 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:58.764825 27817 raft_consensus.cc:2764] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:08:58.765440 27817 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:58.765499 28102 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:58.765976 27817 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:58.766947 27817 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.726217 28145 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:08:58.769604 28145 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:58.769997 28145 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:58.773479 28036 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:08:58.726161 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:58.713392 28167 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:58.752471 28187 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.784425 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.785634 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:58.785759 22904 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:08:58.730130 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:58.811991 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.812424 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.812593 22822 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:08:58.812970 22747 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 7.
W20260709 13:08:58.804417 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 74374.6us
I20260709 13:08:58.780122 28035 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:08:58.819634 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 44334.6us
I20260709 13:08:58.765990 28102 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:58.814143 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:58.814687 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:58.822576 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:58.818748 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:58.823890 22750 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:58.780229 28036 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:58.826517 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.835839 22904 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:58.832231 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:58.805588 28192 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:58.833942 22822 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:08:58.834609 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:58.837683 28167 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:08:58.840016 22748 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:58.841585 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.842103 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:58.845737 22827 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:08:58.846860 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.848867 23419 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:58.773521 28147 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:08:58.852638 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.823171 28035 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:58.855448 22904 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:08:58.857549 22824 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:08:58.863456 28207 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:08:58.863850 28036 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.865041 28036 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:58.856698 27834 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:58.866005 27834 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.841498 28194 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:58.870254 22827 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:08:58.844750 28167 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.894126 28167 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:58.847114 28073 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:58.830288 27981 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:08:58.897742 28073 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:58.859809 28035 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:58.898620 28073 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.878335 28187 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.887184 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:58.888682 23452 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:08:58.902653 22750 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:08:58.904835 22904 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:08:58.894143 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:58.911015 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:58.911715 22827 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:08:58.902472 22831 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 8.
I20260709 13:08:58.914032 28147 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:58.887740 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:58.885542 27834 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:58.913918 28186 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.933816 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:58.934633 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:58.935276 28186 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.936754 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:58.937919 22904 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:58.938884 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:58.939903 22750 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:58.940842 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.943662 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.937995 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:58.945699 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:58.946753 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:58.946946 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 184193us
I20260709 13:08:58.947512 22752 raft_consensus.cc:2410] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:08:58.948146 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.948573 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.902257 28193 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:58.949755 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:58.881110 28145 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.947595 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:58.926478 27910 raft_consensus.cc:2764] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:08:58.951727 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:58.938802 28192 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:58.942322 28147 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:58.956532 22897 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:08:58.958006 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:58.954890 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:58.943909 28186 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:58.944521 27528 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:58.943325 27693 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:08:58.985824 28194 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:58.954361 27981 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:58.988569 27831 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:58.956653 28192 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.956063 28179 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:58.989704 28192 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:58.952670 28045 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:58.995617 28035 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.996614 22749 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:58.944670 27694 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:08:58.991032 28191 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:58.997632 28191 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:58.970556 28140 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:58.999270 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:58.952335 27848 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:08:59.006909 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.007701 27804 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:08:59.008176 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.970484 28214 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:08:59.005436 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50615.3us
I20260709 13:08:59.010291 28214 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:59.002663 28194 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:59.013113 28194 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.005388 27983 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:08:59.003304 28139 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:08:59.016232 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.015940 28139 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.018697 28139 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.020542 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:59.020802 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:59.021085 23452 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:59.021541 23444 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:59.021992 27528 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:08:59.022851 27528 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.025285 28214 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.025645 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:59.026839 28129 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.027343 28129 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.028002 23452 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:59.003710 27848 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.029649 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.030418 22831 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:08:59.010535 27694 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:08:59.031181 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.031304 27694 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:59.032357 27694 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.034587 28171 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.023130 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:59.035095 28171 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.029666 27848 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:59.036638 27831 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.025717 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:08:59.038313 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 80627.5us
I20260709 13:08:59.039191 22748 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:59.040302 22794 leader_election.cc:400] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11.
I20260709 13:08:59.040652 22794 leader_election.cc:403] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:08:58.978905 22827 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:08:59.046068 23449 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:59.047216 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:08:59.046736 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 95272.8us
I20260709 13:08:59.049904 22748 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:59.050448 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:59.051159 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:59.052924 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.057372 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:08:59.062907 22827 raft_consensus.cc:2410] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:08:59.052246 22748 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:08:59.008198 27804 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:59.071375 22750 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:59.048331 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:59.070785 22897 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:59.072052 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.079746 22829 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:59.080111 22829 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:59.072795 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:59.073985 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.071679 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:08:59.076774 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.089855 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:08:59.089968 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.090739 23444 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:59.091418 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.092252 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:08:59.093936 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.094416 22831 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:08:59.094800 22752 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:08:59.080973 22829 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.109612 22829 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:59.102489 22754 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:59.107980 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:08:59.114859 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.111481 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:59.046489 28036 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.070756 28179 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11.
I20260709 13:08:59.092850 27804 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:59.128451 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.091352 28166 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:59.145378 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.146468 22831 raft_consensus.cc:2410] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:08:59.131785 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.132061 23453 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:59.090523 28045 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:08:59.148231 22826 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:59.136129 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:59.021226 28167 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.100272 27694 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:59.101953 27983 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:59.149555 22822 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:08:59.129920 28073 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:08:59.037182 28171 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:59.150589 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:59.118157 28129 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:59.100481 28227 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:59.108747 28187 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:08:59.163591 28134 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:59.145242 27370 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:59.126524 28186 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:08:59.185993 22896 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:59.101711 27834 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.188481 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:59.158078 28231 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:59.117393 28137 consensus_queue.cc:1175] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c45e1805f049467697a7358fb566de2f" responder_term: 12 status { last_received { term: 12 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 12 index: 13 } }
I20260709 13:08:59.129653 27431 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:08:59.117168 27693 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:08:59.131587 27981 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
W20260709 13:08:59.179597 28234 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:08:59.148548 28153 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:08:59.157675 28102 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.185292 28129 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:08:59.188498 27983 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:59.235247 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:59.188627 27370 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.167768 28036 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.194204 28240 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:59.240294 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 121876us
I20260709 13:08:59.236059 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:59.247002 22901 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:59.249575 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:08:59.250289 22905 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:59.253365 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 100474us
I20260709 13:08:59.256356 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:59.241245 28145 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:59.245494 27370 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:59.247334 28191 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:59.256908 28223 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:08:59.229902 28239 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:08:59.254611 28035 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:59.220461 28231 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.244841 28129 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.300954 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:08:59.299158 28233 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 13:08:59.319784 28247 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:08:59.321187 28036 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.348985 22905 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:08:59.327373 22898 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:08:59.364423 22905 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:08:59.380420 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:59.381215 22754 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:08:59.382118 22905 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:08:59.391273 22749 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:59.383671 22897 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:08:59.329968 27834 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:59.360170 28225 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:08:59.358652 28229 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:08:59.331290 28231 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.382910 28254 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:59.411834 28244 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:08:59.409878 28228 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:59.428368 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:59.432061 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:59.310503 27983 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.437601 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:59.442090 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.442925 22827 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:08:59.444186 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.444610 22831 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:59.444965 22824 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:59.446466 28246 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.200s	user 0.003s	sys 0.011s
I20260709 13:08:59.449657 28167 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:08:59.449759 28249 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:08:59.430732 27693 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.462203 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:59.454228 22824 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:08:59.451447 22749 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
W20260709 13:08:59.464288 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.440047 28035 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:08:59.440065 27817 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:59.458297 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.429821 28254 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:08:59.477133 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.461560 22829 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:08:59.438171 28214 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:59.469818 22827 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:59.474189 22826 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:08:59.475272 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.417588 28192 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.442616 28129 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:59.480124 22748 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:08:59.480558 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:59.481869 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:08:59.484117 27848 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.455195 28102 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:08:59.489395 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.456085 28187 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:59.479596 27817 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:59.500358 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:08:59.510421 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:59.499835 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 353076us
I20260709 13:08:59.467667 27693 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.525434 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:08:59.525751 22903 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:08:59.523684 27910 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:59.531859 27804 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.510834 28214 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:59.521716 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:08:59.522192 22751 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:59.501345 28032 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:08:59.538129 27910 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:08:59.538465 28254 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.538537 27910 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:59.539048 27910 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:08:59.538872 28214 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.544225 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1435) took 1158 ms. Trace:
I20260709 13:08:59.544656 23445 rpcz_store.cc:276] 0709 13:08:58.386226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.387874 (+  1648us) service_pool.cc:224] Handling call
0709 13:08:59.544024 (+1156150us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b5400554454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:08:59.544166 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1779220224}
I20260709 13:08:59.542132 28179 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:08:59.520462 28045 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:08:59.545549 28262 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:08:59.500507 28140 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.578193 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:08:59.574008 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.574585 23450 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:59.537822 28032 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:59.574990 22901 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:08:59.575317 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:08:59.583814 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:59.584270 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:08:59.588492 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63146.3us
I20260709 13:08:59.592618 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:59.587239 28140 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:08:59.594175 28102 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:59.599050 22794 leader_election.cc:400] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 10. Current term is 11.
I20260709 13:08:59.603803 28102 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:59.610530 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:59.610911 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:08:59.594336 28186 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:59.599869 28140 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:08:59.599327 28179 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:08:59.613864 28261 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:59.599195 28157 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:08:59.603612 28193 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:08:59.628101 28140 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:59.628546 28140 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:59.628269 27817 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.629449 28140 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.630064 27848 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:08:59.630669 28256 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.093s	user 0.004s	sys 0.001s
I20260709 13:08:59.626336 28032 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.639930 28261 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:08:59.630636 27370 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.617123 28139 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:08:59.632347 27431 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:08:59.589785 28073 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
W20260709 13:08:59.634663 28255 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:08:59.634323 27693 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.639930 28157 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:59.602757 28153 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:08:59.625406 28254 raft_consensus.cc:2764] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:08:59.645731 28192 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:59.602567 28195 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:59.597625 28227 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:08:59.654702 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 65113us
I20260709 13:08:59.656265 28261 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:59.657269 27848 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:59.656548 28186 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.655285 22754 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:59.668532 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:59.674508 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:59.666603 28157 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:59.679678 28035 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:59.681335 27693 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:08:59.680922 28157 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.683960 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.684221 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.684499 22829 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:59.658499 27804 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:59.687302 22753 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:08:59.688867 22823 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:59.666482 27431 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:59.694485 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:08:59.679612 28266 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:08:59.696841 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:59.697418 28267 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:59.697454 28232 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:59.702350 27831 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:59.702589 28232 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:59.698271 22831 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:59.702927 27831 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:59.706065 27831 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.707165 23447 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:59.709147 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:59.709383 22824 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:59.666795 27834 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:08:59.702688 28261 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.711927 22831 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:08:59.718710 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.719307 22831 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:08:59.719398 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.698060 22829 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:08:59.682358 28157 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:59.690023 28186 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.656994 27910 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.727311 23451 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:08:59.733006 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:08:59.744356 27804 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:08:59.714700 23419 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:59.674454 28134 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:08:59.750406 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:59.754577 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:59.754173 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:59.711611 28261 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:08:59.761924 27910 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:59.763082 22825 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:08:59.765609 23452 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:08:59.713807 28231 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.702318 28267 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:08:59.744897 27804 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.725672 28035 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.715548 28166 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:08:59.687129 27848 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.772986 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:08:59.798595 28232 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.801553 28102 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.770936 28177 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:59.745800 28227 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:08:59.770355 28257 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:08:59.808573 28166 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:59.818483 28137 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:59.788148 27693 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:08:59.805217 28231 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.808097 28035 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.850106 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:08:59.835353 28232 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:59.834004 28266 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:08:59.838061 28102 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:59.839715 27981 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.863933 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:08:59.869462 22749 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:08:59.874387 27848 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.874918 27848 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:08:59.875510 27848 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:59.876430 27848 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.879379 22828 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:08:59.879737 22828 pending_rounds.cc:85] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 11
I20260709 13:08:59.879944 22828 pending_rounds.cc:107] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:08:59.880159 22828 raft_consensus.cc:2889] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:59.881377 28251 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 13:08:59.884033 28251 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:59.842676 27983 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:08:59.866321 28266 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.889873 27983 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:59.890211 27983 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:08:59.891296 28266 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.891004 27983 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.892370 27983 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:59.839182 27431 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:08:59.894948 28035 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.859251 27370 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:08:59.901516 28186 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:08:59.867837 27981 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:08:59.831060 28179 consensus_queue.cc:1175] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c45e1805f049467697a7358fb566de2f" responder_term: 17 status { last_received { term: 14 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: [16.14-16.14]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:08:59.914830 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 159231us
I20260709 13:08:59.907495 23445 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:08:59.923740 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:08:59.917970 28268 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:08:59.949093 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 214807us
I20260709 13:08:59.951931 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:08:59.943734 28137 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:08:59.953739 22824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:08:59.968007 22824 raft_consensus.cc:606] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:08:59.968375 22824 raft_consensus.cc:614] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:59.868765 28267 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:08:59.994321 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:08:59.996317 22753 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:08:59.973426 28137 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:08:59.955976 28035 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
W20260709 13:09:00.012959 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.009742 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:08:59.968129 28266 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:09:00.015074 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.033687 22824 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
W20260709 13:09:00.017079 28271 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:00.017230 27981 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.018414 28231 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.019403 28035 consensus_queue.cc:1243] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:09:00.022562 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.059269 22830 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:00.025898 28140 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.026300 22753 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:09:00.029747 28272 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:09:00.047295 28166 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.063263 28281 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:00.063808 28281 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.064754 28166 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.066362 28281 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:00.067776 28284 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:09:00.068638 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:00.077911 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.080787 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.084002 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.084769 22753 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:00.086129 22830 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:09:00.086759 22830 raft_consensus.cc:606] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.087170 22830 raft_consensus.cc:614] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:09:00.093431 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:00.082046 28244 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:09:00.081108 28035 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:00.101785 22753 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
W20260709 13:09:00.116513 28286 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.127s	user 0.000s	sys 0.002s
I20260709 13:09:00.100903 27817 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:00.124303 27817 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.100757 28035 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.096964 28177 raft_consensus.cc:697] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:00.127923 28281 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.130538 28281 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:00.132134 28285 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:09:00.132477 28281 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.132504 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2852) took 1103 ms. Trace:
I20260709 13:09:00.134181 22822 rpcz_store.cc:276] 0709 13:08:59.040481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.042034 (+  1553us) service_pool.cc:224] Handling call
0709 13:09:00.132347 (+1090313us) spinlock_profiling.cc:243] Waited 60.6 ms on lock 0x7b5400081e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:00.132457 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139361152}
I20260709 13:09:00.134824 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1248) took 1719 ms. Trace:
I20260709 13:09:00.133776 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1891) took 1186 ms. Trace:
I20260709 13:09:00.135870 22904 rpcz_store.cc:276] 0709 13:08:58.947264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.947373 (+   109us) service_pool.cc:224] Handling call
0709 13:08:58.947619 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:58.947793 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:58.980627 (+ 32834us) log.cc:844] Serialized 14 byte log entry
0709 13:09:00.133374 (+1152747us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:00.133438 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:00.133501 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.133578 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:00.133716 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":896855,"prepare.queue_time_us":37775,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":898762,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":255470,"threads_started":1,"wal-append.queue_time_us":342649}
I20260709 13:09:00.135248 22902 rpcz_store.cc:276] 0709 13:08:58.415885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.416014 (+   129us) service_pool.cc:224] Handling call
0709 13:08:58.416371 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:58.493840 (+ 77469us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:58.744278 (+250438us) log.cc:844] Serialized 14 byte log entry
0709 13:08:59.148851 (+404573us) log.cc:844] Serialized 14 byte log entry
0709 13:08:59.227776 (+ 78925us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:59.235145 (+  7369us) log.cc:844] Serialized 26 byte log entry
0709 13:08:59.235307 (+   162us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:59.235549 (+   242us) log.cc:844] Serialized 14 byte log entry
0709 13:09:00.132777 (+897228us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b1800262b40. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb78ccf58f 00007fcb71d0e571 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd
0709 13:09:00.134359 (+  1582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.134545 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:00.134597 (+    52us) raft_consensus.cc:1685] finished
0709 13:09:00.134617 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:00.134761 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":922158,"prepare.queue_time_us":270345,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":4818,"raft.queue_time_us":1092268,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":15516,"spinlock_wait_cycles":7322880,"thread_start_us":655200,"threads_started":4,"wal-append.queue_time_us":197363}
W20260709 13:09:00.139050 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 882686us
I20260709 13:09:00.139811 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:00.140640 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.141641 28242 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.139226 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1446) took 1573 ms. Trace:
I20260709 13:09:00.142122 28242 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.142650 23450 rpcz_store.cc:276] 0709 13:08:58.565717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.565850 (+   133us) service_pool.cc:224] Handling call
0709 13:08:59.574020 (+1008170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:59.579528 (+  5508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.137206 (+557678us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b5400554450. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:00.139177 (+  1971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220346624}
I20260709 13:09:00.144310 28242 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.144836 28242 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:00.145696 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.146381 22903 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:00.147168 27793 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:00.147541 27793 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:00.148315 27793 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.110028 28137 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.150594 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.151986 22899 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:00.158564 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.159322 22897 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:00.161008 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.161561 22899 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:00.162611 28186 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:00.164733 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.165819 22829 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:09:00.166118 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.165812 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:00.166842 23450 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:00.167761 28191 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:00.168169 28191 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.168540 28191 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:00.149256 27793 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.129523 28035 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:00.139088 28235 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.886s	user 0.001s	sys 0.009s
I20260709 13:09:00.184552 28161 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:00.184947 28161 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:00.186172 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:00.186910 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:00.187595 22822 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:09:00.187675 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.188372 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:00.162531 28214 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:00.189781 28214 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:00.175052 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.203773 22899 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:00.177918 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.189792 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1898) took 1078 ms. Trace:
I20260709 13:09:00.205066 22896 rpcz_store.cc:276] 0709 13:08:59.110949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.122977 (+ 12028us) service_pool.cc:224] Handling call
0709 13:09:00.189626 (+1066649us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5400107554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:00.189743 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25841024}
I20260709 13:09:00.190011 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.170689 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.191324 22828 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:09:00.211148 22828 pending_rounds.cc:85] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 11
I20260709 13:09:00.211198 22904 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:00.211552 22828 pending_rounds.cc:107] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:09:00.211925 22828 raft_consensus.cc:2889] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:00.242956 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1268) took 1191 ms. Trace:
I20260709 13:09:00.244216 28239 consensus_queue.cc:1243] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:09:00.204881 28242 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:00.231532 28161 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:00.249894 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:00.260288 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 87182.9us
I20260709 13:09:00.260624 22753 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:00.241371 28177 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.231530 28214 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.250373 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.269730 22901 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:09:00.269987 22899 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:09:00.252100 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.253919 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.252422 22900 rpcz_store.cc:276] 0709 13:08:59.054426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.057438 (+  3012us) service_pool.cc:224] Handling call
0709 13:08:59.099940 (+ 42502us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:59.152411 (+ 52471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:59.152439 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:59.163676 (+ 11237us) log.cc:844] Serialized 52 byte log entry
0709 13:08:59.427288 (+263612us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:59.436446 (+  9158us) log.cc:844] Serialized 14 byte log entry
0709 13:09:00.242566 (+806120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.242660 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:00.242722 (+    62us) raft_consensus.cc:1685] finished
0709 13:09:00.242741 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:00.242901 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":847677,"prepare.queue_time_us":8982,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":895652,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":3203,"thread_start_us":263225,"threads_started":1,"wal-append.queue_time_us":279859}
I20260709 13:09:00.271242 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.263020 22905 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:09:00.271960 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.264519 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.248745 27910 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:09:00.270166 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 232804us
I20260709 13:09:00.271837 22900 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:00.273784 22904 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:00.255474 28253 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.284513 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.285200 22825 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:00.284224 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.287724 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.280174 28036 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:00.277124 27910 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:00.294790 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.295131 27910 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:00.280364 28242 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:00.295858 28214 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:00.296478 28242 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.297530 28282 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:09:00.299256 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.258168 28252 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:00.306699 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.290506 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.309639 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:00.311107 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.311921 22903 raft_consensus.cc:2393] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:00.312753 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.313910 28248 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.317448 27528 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:00.317968 27528 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.318281 27817 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.265916 28187 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:00.320063 27528 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:00.323336 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 71322.9us
I20260709 13:09:00.313283 22903 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.313197 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.324409 22754 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:09:00.326213 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.326274 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.327482 22900 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:09:00.328799 22900 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:00.277779 28161 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:00.335603 22905 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:09:00.335805 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.258239 28102 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:09:00.339284 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.339303 22896 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:00.289932 28073 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.358448 28073 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.359961 23453 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:00.362736 28073 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.347088 22900 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:00.348416 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.351560 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:00.301296 28147 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.370340 22905 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:09:00.372092 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:00.375284 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.377856 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.331441 28036 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:00.333076 28154 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:00.380565 22754 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:00.399291 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.399982 22905 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:00.346638 22903 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:00.381727 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:00.349615 28145 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:00.404985 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.371304 28137 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:00.405895 22903 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:00.395612 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.397589 22825 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:00.397697 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.396102 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.406912 22823 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:00.405187 22905 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:00.407474 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.411163 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.338243 28187 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.412830 23444 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:00.414435 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.394210 28239 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:00.424454 23447 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:00.306082 28253 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.425557 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.428578 22754 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:00.428803 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.401158 28243 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.431705 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.405081 28139 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.404137 27528 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.432600 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:00.411605 28137 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.415315 28252 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:00.408968 27370 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:00.448287 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.441403 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.444082 23444 raft_consensus.cc:606] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.449468 23444 raft_consensus.cc:614] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:00.422174 28145 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:00.433341 28166 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:00.454032 22900 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:00.425251 28192 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:00.400810 28147 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.445182 28167 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:00.468041 22905 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:00.444931 28137 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:00.469810 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.471349 22905 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:00.472833 22903 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:00.448697 27528 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:00.478366 28296 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:00.448336 28139 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.480424 28139 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:00.448976 28252 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:00.480993 28252 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.482306 28191 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.484350 28191 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:00.485941 28304 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:09:00.486673 28147 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.453718 28278 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:00.451292 28257 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:00.488910 23451 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 18: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:00.439697 28129 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:09:00.439814 28282 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:09:00.494391 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.495754 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.485239 28303 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:00.497699 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 47217.7us
I20260709 13:09:00.497444 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.490394 28278 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:00.503760 23419 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:00.489926 28167 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:00.504922 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.506088 28277 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:00.506592 28277 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:00.507038 28277 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:00.515717 22905 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:09:00.519047 27793 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:00.520704 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.521564 22825 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:00.496223 28035 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:00.526250 22898 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:00.526659 28035 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:00.527834 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.530189 22747 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:00.497527 28147 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:09:00.501037 28294 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:09:00.445947 28243 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.538221 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.516155 28252 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:00.558678 28252 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:00.516960 28277 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.560498 28243 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.560678 28277 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.494853 28257 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:00.554818 28299 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:09:00.471745 28259 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:09:00.561774 28167 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.523509 28278 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:00.564482 28301 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:09:00.566520 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.567313 23450 raft_consensus.cc:2410] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 15 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:09:00.536499 27793 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:00.569694 27793 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.570047 27793 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.570222 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:00.572384 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.573127 22755 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:00.573495 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.573911 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.574239 22898 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:00.577076 22753 raft_consensus.cc:2410] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:09:00.581948 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:00.585510 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:00.588690 22898 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:00.600063 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.600871 23449 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:00.602059 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.530462 27983 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:09:00.574015 28253 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.611281 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:00.610817 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.504971 28303 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:00.603219 28282 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:00.603989 27370 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:00.611344 28277 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:00.628806 22754 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:09:00.619344 28253 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:00.631625 22825 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:09:00.633036 22829 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:00.634007 22825 pending_rounds.cc:85] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 6
I20260709 13:09:00.634272 22825 pending_rounds.cc:107] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:09:00.634490 22825 raft_consensus.cc:2889] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:00.639123 27910 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:00.624253 28282 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.642591 28282 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.642854 28194 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:09:00.647127 28302 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.207s	user 0.000s	sys 0.002s
I20260709 13:09:00.633934 28253 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.634586 28262 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:00.631323 28277 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:00.665880 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.666136 28303 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.666611 27910 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.655046 28253 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:00.675546 28248 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:00.695168 28277 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.662190 28262 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:00.699417 28262 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.701684 27804 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.703799 22823 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:09:00.704141 22823 pending_rounds.cc:85] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 8
I20260709 13:09:00.704409 22823 pending_rounds.cc:107] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:09:00.703969 28253 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.704609 22823 raft_consensus.cc:2889] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:00.706175 28282 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.710752 28253 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.711532 28282 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:00.711792 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:00.712440 23450 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:09:00.717959 28145 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:00.719074 23450 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:00.719333 27528 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.719244 28278 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.687561 27793 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.748471 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.750602 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.724565 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.729964 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.731516 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.731746 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.732174 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.733065 22899 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:00.734586 22898 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
W20260709 13:09:00.760375 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.767678 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.771325 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.772058 22898 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:00.772094 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:00.773003 22829 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.774891 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.778084 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.784513 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.785254 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.785629 22831 raft_consensus.cc:2410] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 14 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:09:00.746127 28278 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.775575 27370 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:00.791729 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.792515 27848 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:00.793025 27848 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:00.784226 28102 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.773962 28314 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:09:00.793918 27848 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.794134 28102 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:00.794786 28314 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:00.795113 27793 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.767949 28187 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.796340 28145 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:00.784237 28262 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.797482 28262 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.799487 28262 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.799988 27528 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:00.801393 28314 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:00.801476 28166 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:00.804010 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.802701 28314 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.804606 22755 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:00.807477 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.793453 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.813562 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.814383 22829 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:00.815671 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:00.814502 27793 raft_consensus.cc:2764] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:09:00.817016 27981 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:00.820071 23445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:00.813418 28314 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:00.820922 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.820943 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.819355 28278 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:09:00.836581 22903 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.838217 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.839504 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.840598 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.821975 22831 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.843003 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.847498 22831 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:00.847885 22903 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:00.683161 28137 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:00.788751 28191 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:00.824291 23445 raft_consensus.cc:606] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.812083 28145 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:00.820683 27370 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:00.822726 22898 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:00.812227 27528 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:00.850003 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.848104 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:00.853211 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.853874 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.851068 22755 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:00.854486 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.855274 22748 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.855511 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.850243 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:00.859951 28191 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:00.860453 28191 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:00.857239 28248 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:00.861349 28191 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.867002 28200 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.867273 28240 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:00.867868 28240 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:00.869814 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.870867 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:00.871577 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.872056 28161 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:00.872262 28261 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:00.872605 28161 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:00.872753 28261 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.873490 28232 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:00.874066 28232 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:00.874949 28232 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.878895 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.873237 28261 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:00.881515 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.867600 28200 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.883513 28200 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.883610 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.873649 28161 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.849572 28253 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:00.888368 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:00.888366 22748 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:00.891256 22747 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:00.858664 28129 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:00.897514 23419 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:00.899684 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.902817 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.904167 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.904853 22904 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:00.910235 22897 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:09:00.887104 27431 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:00.913899 27431 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:00.914729 27431 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.916128 27431 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:00.917552 28177 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:00.852547 28166 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:09:00.917809 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:00.923122 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:00.924211 22825 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:00.924592 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:00.930019 22898 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:09:00.851962 22871 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:00.937153 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49042.2us
I20260709 13:09:00.896399 28248 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:00.934286 28193 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:00.909638 28157 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:00.943332 28248 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.890703 28282 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:00.868994 28240 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:00.895524 28287 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:00.921278 28231 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:00.927923 28195 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:00.910305 28167 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.919907 28192 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.919517 28137 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.962965 22904 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:00.922519 28187 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:09:00.911464 28200 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:09:00.961351 28193 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
W20260709 13:09:00.958220 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 106324us
I20260709 13:09:00.980314 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.982537 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.982888 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.983925 22899 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:00.986446 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:00.987118 22900 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:00.956470 28231 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.989862 23445 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:09:00.994417 22829 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:00.988329 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:00.991204 22896 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:00.995664 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:00.961283 27528 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:00.963647 28167 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.012488 22750 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:01.014405 28167 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:00.967617 28195 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:00.963546 28192 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.011837 28231 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:01.041710 28261 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.043745 28261 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:00.972250 28262 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:09:01.016721 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:01.046307 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:01.046530 22824 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:00.999346 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:01.033689 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:01.047379 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:01.033370 28177 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
W20260709 13:09:01.048451 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49218us
I20260709 13:09:01.041163 28192 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:01.018411 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:01.047405 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:01.052596 22829 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:01.056030 27848 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:01.056614 27848 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:01.056901 22824 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:01.057528 27848 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.033727 28195 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.038200 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:00.987638 27793 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.036873 28287 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:00.991600 22904 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:09:00.961236 28267 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:09:01.062122 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:00.963199 28214 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:09:01.080483 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:01.069275 28261 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:01.071094 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 145255us
I20260709 13:09:01.080754 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:01.088531 22830 raft_consensus.cc:2410] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:09:01.089016 23445 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:01.090816 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:01.101450 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:01.091451 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:01.107232 22750 raft_consensus.cc:2410] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:09:01.067091 28242 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.031s	sys 0.072s
I20260709 13:09:01.112214 22871 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:01.113142 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:01.075722 28192 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:01.114850 28147 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:09:01.120796 28309 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.648s	user 0.000s	sys 0.002s
I20260709 13:09:01.088608 28035 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.076833 28287 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:01.124171 22871 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:01.112722 28242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.032s	sys 0.074s
I20260709 13:09:01.111690 28313 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:01.082774 27793 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.087659 28166 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:09:01.087401 28167 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:09:01.116726 28045 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:01.108904 28261 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.161301 28287 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.162076 28167 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:09:01.165376 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 54293.2us
I20260709 13:09:01.131287 28303 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:01.177273 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 191059us
I20260709 13:09:01.162497 28167 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:01.188525 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:01.149776 28313 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.145349 28187 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:01.162551 28261 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:01.178088 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:01.146128 28248 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:01.179328 28298 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:01.225723 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.230954 28339 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:01.238365 27793 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:01.141844 28035 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.254408 28232 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:01.263928 23445 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 13:09:01.263041 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 83653.1us
I20260709 13:09:01.215062 28313 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:01.189522 28166 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:09:01.270565 28232 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:01.272230 27793 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:01.181288 28231 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:01.255002 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 66523.8us
I20260709 13:09:01.188671 28334 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 13:09:01.299576 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:01.274729 28341 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:09:01.315407 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:01.316134 22896 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:01.318106 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:01.318929 22902 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:01.255458 28167 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:01.265517 27817 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:01.329946 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:01.336172 22897 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:01.292644 28045 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.339522 22754 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:01.258126 28187 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.335875 27817 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:01.307757 28248 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.305565 28314 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.355335 28287 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:01.356208 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38478 (request call id 3184) took 1097 ms. Trace:
I20260709 13:09:01.370216 22753 rpcz_store.cc:276] 0709 13:09:00.259021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.271592 (+ 12571us) service_pool.cc:224] Handling call
0709 13:09:01.356003 (+1084411us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54004060d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:01.356146 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699125632}
I20260709 13:09:01.325848 27528 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.359721 28102 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:01.373744 22896 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:01.260954 28298 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.379989 28298 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:01.326148 28267 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:09:01.393646 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:01.262300 28339 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.327929 27793 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.391939 22897 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:01.392481 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:01.367066 28167 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:01.362668 27817 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:01.352084 28045 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.401950 28300 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:09:01.375458 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1611) took 1191 ms. Trace:
I20260709 13:09:01.418459 22751 rpcz_store.cc:276] 0709 13:09:00.184497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.184688 (+   191us) service_pool.cc:224] Handling call
0709 13:09:00.185065 (+   377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:00.192497 (+  7432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:00.192526 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:00.231778 (+ 39252us) log.cc:844] Serialized 52 byte log entry
0709 13:09:00.364020 (+132242us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:00.391443 (+ 27423us) log.cc:844] Serialized 14 byte log entry
0709 13:09:00.391632 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.391717 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:01.120215 (+728498us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:01.365396 (+245181us) raft_consensus.cc:1685] finished
0709 13:09:01.375205 (+  9809us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:01.375389 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6976,"prepare.queue_time_us":27675,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":7805,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":18352,"spinlock_wait_cycles":16353280,"thread_start_us":79540,"threads_started":1,"wal-append.queue_time_us":164462}
I20260709 13:09:01.420158 28193 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:01.420529 28193 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:01.420898 28193 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:01.414321 22899 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:09:01.396023 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:01.398800 23451 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:01.399269 22897 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:01.412142 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:01.400578 28298 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.397831 22753 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:01.413296 28171 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:09:01.414302 28338 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:09:01.460911 28329 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:09:01.463641 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 164129us
I20260709 13:09:01.396754 28342 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:01.369498 28248 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.373895 28314 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.386466 27528 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.489002 28102 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:01.464025 28298 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.490054 28333 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:01.491600 22825 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:09:01.500523 22753 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:09:01.501788 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:01.502807 27431 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:01.475306 28342 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:01.510329 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:01.478314 27804 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:01.528090 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:01.530655 23447 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:01.532423 22900 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:01.534134 22901 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:01.532475 27431 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.535780 27431 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.538092 27431 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:01.540246 22825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:09:01.541100 22825 raft_consensus.cc:606] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:01.546366 23447 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:01.541913 22825 raft_consensus.cc:614] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:01.553568 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:01.423614 28232 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.474974 28300 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:09:01.584131 27793 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:01.585853 28187 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:01.588266 28353 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:01.588765 28353 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.589116 28353 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:01.590083 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 88364us
W20260709 13:09:01.595384 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 54410us
I20260709 13:09:01.596244 22794 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:01.605067 27431 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:01.607614 22747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49614
I20260709 13:09:01.611264 22747 raft_consensus.cc:606] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Received request to transfer leadership
I20260709 13:09:01.611589 22747 raft_consensus.cc:614] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:01.614411 28337 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:09:01.614910 28366 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:09:01.615366 28330 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:09:01.619925 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:01.621291 28328 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:09:01.621243 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:01.648772 22755 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:09:01.590044 28035 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:01.651682 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 98204.8us
W20260709 13:09:01.653422 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.624617 28231 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:01.657488 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 191885us
I20260709 13:09:01.574167 27804 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:01.699047 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 77865.4us
I20260709 13:09:01.647951 28257 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:01.521117 28333 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:01.716809 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58477.9us
I20260709 13:09:01.718351 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:01.667310 28266 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:01.663857 28365 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.697178 27694 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.721788 28187 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
W20260709 13:09:01.727027 28335 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:09:01.727119 28261 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:01.727623 28354 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:09:01.727872 28345 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:09:01.728210 28354 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:01.712220 28179 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:01.731693 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:01.737588 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.719264 28355 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:01.799456 28355 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.802156 28355 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:01.791294 28364 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:09:01.792309 28351 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:09:01.793217 28363 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:09:01.729125 28339 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:01.796509 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 63893.3us
I20260709 13:09:01.804004 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:01.797034 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 83959.2us
I20260709 13:09:01.722765 28266 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:01.734627 28231 raft_consensus.cc:2764] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:09:01.728967 28261 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:09:01.760131 28045 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:01.738101 28179 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:01.831739 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:01.832942 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:01.833506 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:01.834081 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:01.882143 22826 raft_consensus.cc:2410] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:09:01.658624 28342 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.838634 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:01.732486 27694 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.804055 28351 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:01.907582 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:01.804841 28314 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:01.826982 28371 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:01.940454 23447 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:01.908135 28355 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:09:01.908381 28298 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:01.947767 28370 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.948303 28370 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:01.949514 28102 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:01.950865 28102 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:09:01.949865 28359 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:09:01.952384 28353 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.955473 28333 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:01.961499 28248 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:01.729082 28354 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.969059 28371 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:01.733181 28365 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.994216 28179 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:01.944396 28351 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.078596 28351 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:02.083160 28193 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.000s
I20260709 13:09:02.083740 28193 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:02.084218 27817 consensus_meta.cc:287] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.625s	user 0.007s	sys 0.003s
I20260709 13:09:02.061152 28371 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.086885 28371 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:01.961217 28287 consensus_meta.cc:287] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.000s
I20260709 13:09:02.093158 27694 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:02.111207 28313 consensus_meta.cc:287] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.802s	user 0.011s	sys 0.000s
I20260709 13:09:02.111817 28313 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.112684 28370 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.114449 28313 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.118616 28353 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.084856 27817 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:02.130373 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:02.139552 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 304798us
I20260709 13:09:02.173540 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.174487 23449 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:02.178326 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.180181 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:01.829852 28339 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:02.193518 28193 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.194381 28193 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:02.196240 27817 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:02.194471 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 355903us
I20260709 13:09:02.213819 27910 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.214365 27910 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.218439 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.219075 22751 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:02.219676 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.221551 27910 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.222146 22747 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:09:02.222523 22747 pending_rounds.cc:85] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 10
I20260709 13:09:02.222764 22747 pending_rounds.cc:107] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:09:02.223004 22747 raft_consensus.cc:2889] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:02.226413 22747 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:02.226624 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.226786 22747 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:02.227604 22747 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.228435 22747 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:02.232007 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2651) took 1502 ms. Trace:
I20260709 13:09:02.232484 22755 rpcz_store.cc:276] 0709 13:09:00.729372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.858384 (+129012us) service_pool.cc:224] Handling call
0709 13:09:01.648509 (+790125us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:01.649282 (+   773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.231823 (+582541us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b540016b050. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:02.231951 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663393536}
I20260709 13:09:02.236726 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.237344 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:02.237490 23447 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:02.242877 22755 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:02.253669 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:02.255324 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.256701 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.257256 22823 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:02.254093 27910 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:02.258440 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:02.258220 27910 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.259910 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.260835 22756 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 13:09:02.261440 22702 auto_rebalancer.cc:893] Removing replace marker failed: Replica 6a5191aaff76469bbf6c4735dc9e6509 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } } pending_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:09:02.261896 22702 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:09:02.262167 22702 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:09:02.240365 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:02.267701 28102 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:02.268895 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:02.269665 28378 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:02.270076 28378 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:02.270437 28378 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:02.276607 22755 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:02.277719 27910 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.283798 28378 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:02.284221 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 782315us
I20260709 13:09:02.286675 28192 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:09:02.290565 28167 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.291090 28167 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.291620 28361 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:02.293447 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.294214 22828 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:02.278776 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:02.304535 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.305104 22747 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:09:02.240160 27804 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.292155 28361 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.320828 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.321168 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.326526 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.327312 22750 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:02.307426 28147 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:02.333536 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.334299 22900 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:02.330578 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.337693 28378 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.342227 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.342360 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.342784 22900 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:02.343439 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.343811 22902 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:02.311967 28102 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:02.343133 23444 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:02.345463 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.349210 22831 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:02.356225 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.356616 23450 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:09:02.348726 28102 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.358739 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.358917 28102 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:02.361632 22826 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:02.359527 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.358718 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.360651 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.361855 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:02.528272 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:02.534981 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:02.359381 22750 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:02.532658 22900 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:02.533304 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:02.362224 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.398839 22756 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:02.430868 28266 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.538981 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:02.399089 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.399473 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.536581 22756 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:02.545133 22905 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:09:02.348693 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.536165 22902 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
W20260709 13:09:02.545970 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.547370 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:02.423413 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.408434 23447 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:02.434490 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:02.547133 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:02.547875 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.545528 22756 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.551038 22756 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:02.552992 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.555751 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:02.556470 22899 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:02.356639 28243 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.557390 22902 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:02.557304 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:02.560446 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:02.359587 28166 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:02.407835 28167 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.566310 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:02.550691 28392 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:09:02.432981 28139 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:02.550410 28382 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:09:02.550176 28381 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:09:02.408701 28354 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:02.547115 27981 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:02.344108 28147 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.582866 22823 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:02.583721 28102 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.570979 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:02.586784 22899 raft_consensus.cc:2393] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 8: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:02.587807 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.588510 22899 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:09:02.550449 28303 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.589638 28303 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.592154 28303 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.551013 28373 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:02.593001 28373 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.593487 28373 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:02.593353 28139 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.563019 27910 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:02.594492 27910 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.594838 27910 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:02.551074 28393 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.571000 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:02.563306 28166 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.596478 28166 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:02.596315 28393 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.599150 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.554059 28371 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:02.563740 28243 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.600253 28371 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.601742 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.601917 28243 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:02.565266 28266 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:02.602499 28371 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.602593 22797 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:02.603304 28193 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:02.604148 28193 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:02.605082 28193 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:02.605506 22905 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:02.606314 28380 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:02.606799 28380 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:02.607197 28380 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:02.607919 28214 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:02.608358 28214 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:02.609226 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.609163 28214 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.610571 28214 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:02.610615 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.595816 28139 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.581717 27981 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.611996 28342 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:02.572130 27817 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:02.584221 28147 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.605578 28376 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:02.615051 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.619796 22831 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:09:02.623992 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.625206 22900 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:02.557871 28339 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:02.605443 28193 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.625594 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.578173 28361 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.640350 22828 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:02.617872 22899 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:02.626387 22756 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:02.572253 28333 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:09:02.628322 28383 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:09:02.620777 28370 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:02.644047 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.651472 22900 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
W20260709 13:09:02.655526 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.629841 28394 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:02.706789 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:02.639642 27817 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:02.711278 22899 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:02.714525 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63891.4us
I20260709 13:09:02.643390 28376 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:02.617482 28354 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:02.642508 28193 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:02.646988 28147 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.731019 22900 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:02.715518 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.644984 28370 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:02.642022 28339 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.735301 28147 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:02.737984 28357 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.013s	user 0.016s	sys 0.032s
I20260709 13:09:02.739060 28147 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.712647 27817 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:02.727777 28102 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:02.730757 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 174214us
I20260709 13:09:02.752766 28252 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.753239 28396 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:02.753844 22900 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:09:02.753994 28396 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.756525 28395 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:02.756946 28395 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.757272 28395 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:02.759097 28177 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:02.761543 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.749974 27817 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:02.763767 28354 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:02.765072 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.753293 28252 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.767583 28252 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:02.765769 22755 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:02.768671 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:02.767621 27981 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.770304 28257 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:02.770682 28257 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.770985 28257 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:02.771782 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:02.772454 22824 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:02.735874 28370 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:02.773777 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.777511 28252 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:02.777886 28252 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.778451 28252 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:02.778972 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3197) took 1639 ms. Trace:
I20260709 13:09:02.779847 28166 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.780949 22902 raft_consensus.cc:2368] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17.
I20260709 13:09:02.779342 22748 rpcz_store.cc:276] 0709 13:09:01.139891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.140045 (+   154us) service_pool.cc:224] Handling call
0709 13:09:01.164830 (+ 24785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.372341 (+207511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:01.372384 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:01.372434 (+    50us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:01.372482 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.778759 (+1406277us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54000780d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:02.778807 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:02.778919 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476730624}
I20260709 13:09:02.785807 22899 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:02.717365 28394 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.787063 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.788976 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.790526 22899 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:02.798413 23445 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 13:09:02.807569 23449 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:09:02.777875 28257 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.810724 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.811080 28257 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.815231 28193 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.816336 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:02.816955 22748 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:09:02.818648 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.819319 28354 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:02.819743 28354 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.820124 28354 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:02.821830 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:02.826498 23448 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:09:02.828369 28354 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.830024 28193 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.831271 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:02.832134 22748 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:02.831354 23445 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:02.832913 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1298) took 1705 ms. Trace:
I20260709 13:09:02.833390 22904 rpcz_store.cc:276] 0709 13:09:01.111864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.113833 (+  1969us) service_pool.cc:224] Handling call
0709 13:09:01.115345 (+  1512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.147387 (+ 32042us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:01.187574 (+ 40187us) log.cc:844] Serialized 14 byte log entry
0709 13:09:01.187971 (+   397us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:01.188018 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:01.188074 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.233645 (+1045571us) spinlock_profiling.cc:243] Waited 8.7 ms on lock 0x7b180049b640. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e522 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:02.817134 (+583489us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b5400002dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:02.817182 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:02.817293 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65882,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":1532,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":21081,"spinlock_wait_cycles":91368448}
I20260709 13:09:02.835641 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:02.836318 22904 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 8.
I20260709 13:09:02.796530 28134 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:02.847318 22748 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:09:02.848238 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:02.853039 23449 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:02.848130 28129 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:02.864599 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 229449us
I20260709 13:09:02.853348 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.787847 27981 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:09:02.852473 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:02.871057 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:02.865391 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:02.815271 28365 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.879251 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.881477 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:02.882032 22824 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:02.849009 28192 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:02.790298 28376 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.881825 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:02.850534 28134 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:02.872136 28372 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:02.874445 28232 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:02.872437 28166 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:02.945771 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:02.949250 22825 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:02.884318 28193 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:02.865127 28373 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:02.887542 28231 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:02.962958 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:02.945034 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.954259 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:02.965559 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:02.967211 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:02.987681 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.872292 28129 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:02.885506 28365 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:02.906909 28147 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.011294 23448 raft_consensus.cc:2368] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14.
W20260709 13:09:03.032375 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 78228.2us
I20260709 13:09:03.035373 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:03.012871 22824 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:03.032997 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:02.953910 28385 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:09:03.011914 22825 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
W20260709 13:09:02.962265 28384 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:09:03.019479 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57220.7us
I20260709 13:09:02.907042 28192 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.036551 23445 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:03.050038 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:02.968691 28372 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:03.058400 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:03.007498 28266 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:03.084116 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 213135us
I20260709 13:09:02.963258 28354 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:02.950629 28243 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 13:09:02.987335 28214 raft_consensus.cc:2764] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:09:03.013823 28361 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:03.042702 28397 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:09:03.089658 22830 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:03.033598 28365 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:02.954564 28380 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.019217 28193 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.032238 28232 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:03.042867 28382 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:09:03.036396 28102 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.036439 28390 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:02.971176 28166 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
I20260709 13:09:03.041476 28147 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.113796 28303 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:03.056105 28353 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.063251 28281 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:03.116245 28388 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:09:03.127327 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 92031.6us
I20260709 13:09:03.058704 28409 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:03.041045 28167 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.079s
I20260709 13:09:03.148947 28167 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.149379 28372 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.090577 28339 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.150943 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:03.152235 28406 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.198s	user 0.000s	sys 0.000s
I20260709 13:09:03.152449 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2535) took 2252 ms. Trace:
W20260709 13:09:03.152942 28401 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:09:03.153391 28129 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:03.153949 28129 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:03.154150 28404 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:09:03.113129 28266 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.116115 28403 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.086063 28253 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:03.125054 28365 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:03.144716 27694 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.126969 28179 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.142455 28353 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.149454 28167 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.152856 22898 rpcz_store.cc:276] 0709 13:09:00.899551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.901252 (+  1701us) service_pool.cc:224] Handling call
0709 13:09:00.901587 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:00.931126 (+ 29539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.152232 (+2221106us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b54001b6f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:03.152375 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47914368}
I20260709 13:09:03.125042 28354 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.143668 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:03.155902 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 97554.4us
I20260709 13:09:03.062652 28134 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:03.177242 28400 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:09:03.138530 28394 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.134191 28102 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.137516 28371 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:03.182242 22868 leader_election.cc:400] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14.
I20260709 13:09:03.182539 23449 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:03.187551 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:03.210644 22868 leader_election.cc:403] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:03.210793 22899 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:03.190096 28380 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:03.190928 28373 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.132918 28193 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.175673 28403 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:03.194561 28402 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:09:03.194682 28395 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.196599 28266 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.145879 28370 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.199913 27817 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:03.204360 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.223237 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.223570 28193 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.223830 23446 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:09:03.224056 28193 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:03.224766 28415 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:09:03.224613 28193 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.224860 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:03.226519 28408 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 13:09:03.205088 28423 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:09:03.205188 27910 consensus_meta.cc:287] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 13:09:03.228593 27910 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.229362 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2551) took 1398 ms. Trace:
I20260709 13:09:03.124665 28361 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.229846 22901 rpcz_store.cc:276] 0709 13:09:01.831074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.831226 (+   152us) service_pool.cc:224] Handling call
0709 13:09:01.831588 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:02.434352 (+602764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.434413 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:02.434890 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:09:02.791267 (+356377us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:02.791312 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.791393 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.229080 (+437687us) raft_consensus.cc:1685] finished
0709 13:09:03.229108 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.229280 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":467586,"raft.queue_time_us":480515,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":3723,"thread_start_us":496596,"threads_started":2,"wal-append.queue_time_us":571926}
I20260709 13:09:03.216080 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.233294 28361 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.211907 28371 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:03.234843 28371 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:03.243036 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1077) took 1717 ms. Trace:
I20260709 13:09:03.243323 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1054) took 2648 ms. Trace:
I20260709 13:09:03.243526 22830 rpcz_store.cc:276] 0709 13:09:01.525949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.526529 (+   580us) service_pool.cc:224] Handling call
0709 13:09:01.526839 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.120237 (+1593398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.242780 (+122543us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54000adcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:03.242964 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3593132800}
I20260709 13:09:03.180576 28179 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.145479 28281 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.243747 22822 rpcz_store.cc:276] 0709 13:09:00.594773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.594914 (+   141us) service_pool.cc:224] Handling call
0709 13:09:00.595183 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:00.595545 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:00.595584 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:00.610683 (+ 15099us) log.cc:844] Serialized 26 byte log entry
0709 13:09:03.040916 (+2430233us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b180040d4a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:09:03.144205 (+103289us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:03.144268 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.144366 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.243034 (+ 98668us) raft_consensus.cc:1685] finished
0709 13:09:03.243060 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.243252 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4405,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":9206,"spinlock_wait_cycles":8099072,"thread_start_us":101448,"threads_started":1,"wal-append.queue_time_us":2596985}
I20260709 13:09:03.177855 27694 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.247881 28353 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.201893 28354 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.249830 28403 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:03.154413 28129 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.250166 28354 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.271306 28281 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.271413 28395 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:03.271854 27804 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:03.272245 27528 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:03.272363 27804 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.192903 28252 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.274503 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:03.284549 22899 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:09:03.285492 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:03.289475 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.290241 22903 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:03.291903 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.293206 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.294090 22903 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 8.
I20260709 13:09:03.181435 28231 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:03.223966 27817 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.298664 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.220319 28253 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:09:03.244308 28102 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.304721 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.303053 28281 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.308163 28129 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:03.308069 28281 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.310285 28193 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.224632 28214 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.310878 28281 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.312129 28426 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:03.212981 28232 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.296051 28395 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:03.318611 22899 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:03.305109 28354 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.305764 28373 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.333012 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.333209 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.333827 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:03.334519 27804 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:03.334966 28252 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.347419 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.347592 22899 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:03.319952 28395 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:03.311407 28214 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.354683 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.351891 28428 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.355814 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.355940 28428 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.357746 28393 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:03.341408 28134 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:09:03.353989 28231 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.360383 28428 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.360705 28214 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.362524 28419 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.367309 28419 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.350119 28395 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:03.348251 28373 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.379819 28409 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:03.382025 28262 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:03.345832 28354 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.392442 28373 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.407130 28354 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.388635 28262 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:03.408840 28262 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:03.411043 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.409480 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:03.395849 27694 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:03.411626 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.415160 22897 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:03.422394 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:03.423161 22755 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:03.423065 28428 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.425559 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:03.425760 28428 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.426265 22749 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:03.412647 28214 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.427263 28214 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.427680 28253 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14.
I20260709 13:09:03.427724 28428 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.417766 28393 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.411895 28262 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:03.441907 28393 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.444470 28393 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:03.429562 28409 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.445711 28409 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.449435 28409 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:03.456555 28365 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:03.459697 28365 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:03.451992 28214 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.450932 27910 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:03.530884 22897 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:03.531813 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:03.531996 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.532433 22905 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:03.532213 28395 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.532604 22897 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:09:03.533262 22899 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:03.533483 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.534014 22897 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:03.473842 28371 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.536437 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 998) took 2470 ms. Trace:
I20260709 13:09:03.479872 28419 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:03.494102 28431 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:09:03.536814 23451 rpcz_store.cc:276] 0709 13:09:01.063366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.064700 (+  1334us) service_pool.cc:224] Handling call
0709 13:09:01.065078 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.422753 (+357675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.533742 (+2110989us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540018c4d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:03.533900 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767030528}
I20260709 13:09:03.539417 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:03.541630 23444 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:03.535555 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:03.544473 27910 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:03.544920 27910 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:03.540751 28371 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:03.549293 23444 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:03.553572 22905 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:03.554294 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.555063 22901 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:03.555258 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:03.556023 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:03.557794 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:03.558812 22754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 2656) took 2634 ms (client timeout 3128 ms). Trace:
I20260709 13:09:03.559983 22756 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:03.561537 22901 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:03.561802 22905 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:03.562543 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:03.563248 22901 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:03.563302 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.564558 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.565088 22905 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:03.565953 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.567571 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:03.568116 22905 raft_consensus.cc:2393] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 15: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:09:03.559208 22754 rpcz_store.cc:269] 0709 13:09:00.924253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.924394 (+   141us) service_pool.cc:224] Handling call
0709 13:09:03.558633 (+2634239us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b54003cfad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:03.558754 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952584576}
I20260709 13:09:03.560760 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:03.555147 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:03.573536 28171 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:03.574028 28171 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:03.574931 28171 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.577445 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.578056 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.578456 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:03.579008 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.579412 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.572003 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:03.581755 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:03.582757 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:03.584141 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 2049) took 1327 ms. Trace:
I20260709 13:09:03.585388 22905 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:09:03.586052 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1927) took 1052 ms. Trace:
I20260709 13:09:03.586398 22905 rpcz_store.cc:276] 0709 13:09:02.533695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.583709 (+1050014us) service_pool.cc:224] Handling call
0709 13:09:03.583976 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.585829 (+  1853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.586004 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:03.585012 22794 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:03.584522 22826 rpcz_store.cc:276] 0709 13:09:02.256651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.257825 (+  1174us) service_pool.cc:224] Handling call
0709 13:09:02.258064 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:02.536695 (+278631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.583921 (+1047226us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54001fa9d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:03.584086 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237775744}
I20260709 13:09:03.588656 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:03.589243 23448 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:03.590499 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:03.590792 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.591632 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.593497 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.595546 22826 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:03.596951 22901 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:03.598642 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.601191 22831 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:03.602182 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:03.603252 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.605006 22901 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:09:03.570642 28214 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:03.587401 28419 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:03.609172 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.610560 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2566) took 1008 ms. Trace:
I20260709 13:09:03.619077 28073 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:03.588332 28365 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:03.601850 28232 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:03.625589 27910 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.629601 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:03.765902 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 136388us
I20260709 13:09:03.669816 22901 rpcz_store.cc:276] 0709 13:09:02.601912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.602418 (+1000506us) service_pool.cc:224] Handling call
0709 13:09:03.604061 (+  1643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.610155 (+  6094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.610504 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:03.767091 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:04.014655 23448 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:03.601830 27694 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:04.017599 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:04.002980 22719 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.792s (SENT) [suppressed 2 similar messages]
I20260709 13:09:03.896960 28161 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:03.821811 28232 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:03.941033 28440 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:09:04.047806 28427 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:04.048596 22897 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:04.049252 28373 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:04.049957 28393 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:03.841703 28351 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:03.959329 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.868577 28231 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:04.063794 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:03.937448 28371 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.678215 28419 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.759815 28365 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:03.758104 28073 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:04.068271 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:04.071601 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:04.061364 27834 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:03.877909 28252 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.072718 28427 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.085692 28252 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.087349 28427 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.088191 28252 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:04.010803 28395 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.088829 28252 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:03.833949 28253 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:03.775172 28394 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:04.090422 28394 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:04.090277 28253 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.091542 28394 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:03.828550 28193 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:04.063760 27694 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:04.075527 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:04.076061 28373 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:03.774715 28266 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:03.774895 28214 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:03.815856 28334 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:09:03.978852 28434 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 13:09:04.078329 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.081442 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:04.081970 28365 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:04.084620 28073 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:04.092996 28253 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.098138 28193 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.080847 28419 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:04.105291 28192 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.089521 28252 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.096725 23445 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:09:04.067250 28161 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.110797 27694 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:04.114404 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:04.081782 28371 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.136451 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1924) took 1603 ms. Trace:
I20260709 13:09:04.136973 22903 rpcz_store.cc:276] 0709 13:09:02.532944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.300398 (+767454us) service_pool.cc:224] Handling call
0709 13:09:03.302187 (+  1789us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:03.304091 (+  1904us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:03.304134 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:03.304651 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:09:03.433847 (+129196us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:03.433892 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.433954 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:04.136183 (+702229us) raft_consensus.cc:1685] finished
0709 13:09:04.136210 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.136379 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1289,"raft.queue_time_us":3359,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":9763,"thread_start_us":128747,"threads_started":1,"wal-append.queue_time_us":392762}
I20260709 13:09:04.113837 28334 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.112052 28266 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.130911 28253 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.121532 28214 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.140110 28427 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:04.165691 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2568) took 1543 ms. Trace:
I20260709 13:09:04.166200 22898 rpcz_store.cc:276] 0709 13:09:02.621933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.164696 (+1542763us) service_pool.cc:224] Handling call
0709 13:09:04.165067 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:04.165270 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:04.165306 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:04.165378 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:04.165430 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.165496 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.165620 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:04.168008 22898 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:04.168610 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2570) took 1407 ms. Trace:
I20260709 13:09:04.168970 22898 rpcz_store.cc:276] 0709 13:09:02.761675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.167314 (+1405639us) service_pool.cc:224] Handling call
0709 13:09:04.167624 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:04.168367 (+   743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.168555 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:04.187898 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2573) took 1348 ms. Trace:
I20260709 13:09:04.188544 22898 rpcz_store.cc:276] 0709 13:09:02.839783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.172168 (+1332385us) service_pool.cc:224] Handling call
0709 13:09:04.172479 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:04.172698 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:04.172734 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:04.172786 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:04.187543 (+ 14757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.187652 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.187826 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:04.078883 28393 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.209160 22897 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:04.211591 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:04.080013 28351 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.128770 28354 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:04.161175 28253 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.080719 28231 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.220319 28354 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.235567 22826 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:09:04.230865 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 227258us
I20260709 13:09:04.231133 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:04.214646 28192 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:04.175213 28427 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.233418 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:04.234151 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:04.270927 28409 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:04.271477 28409 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:04.268622 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 199110us
I20260709 13:09:04.231067 28252 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.284121 28192 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.297608 28192 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.302098 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:04.302807 23446 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:04.300295 28365 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.265327 28262 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:04.307091 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 239216us
I20260709 13:09:04.308609 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.308789 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:04.318958 22822 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:04.316026 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.217509 28232 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.317565 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.319132 22828 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:04.309755 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.321633 22827 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:04.322539 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.326992 22830 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:04.326419 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.286949 27694 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:04.373986 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.374614 22822 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:09:04.381354 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.375617 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:04.318058 28214 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.333245 27910 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.383985 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.384249 27910 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.377240 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.374547 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:04.387293 22828 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:04.323267 28262 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.390988 27910 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.429040 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:04.430074 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 105896us
I20260709 13:09:04.372112 28161 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:04.438068 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.434962 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:04.394482 28433 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:09:04.461025 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 64676.7us
I20260709 13:09:04.386046 28192 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:04.462067 22897 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:04.459864 22830 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
W20260709 13:09:04.462488 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 138730us
I20260709 13:09:04.375454 27694 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.463930 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.466811 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.471172 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:04.474613 28445 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:09:04.456073 28193 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:04.431694 28240 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:04.432159 28334 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:04.422979 28214 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:04.438038 28266 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:04.484277 28193 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.485601 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.456840 28354 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:04.490521 22897 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: [11.11-13.12]
I20260709 13:09:04.469823 28460 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:04.482880 28351 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.486325 28393 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:04.516922 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 45811.9us
I20260709 13:09:04.471975 28192 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.520267 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:04.526142 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:04.489575 28214 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.510130 28465 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:04.541451 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:04.518153 28231 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:04.495826 28193 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:04.542197 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.497053 28449 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:09:04.592389 28161 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:04.594034 28451 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
W20260709 13:09:04.647760 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.520503 28427 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:04.532802 28441 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
W20260709 13:09:04.681872 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 174595us
I20260709 13:09:04.514616 28460 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:04.534327 28253 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:04.532472 28457 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:09:04.734189 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:05.198401 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.004s	sys 0.007s
W20260709 13:09:05.198882 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.004s	sys 0.008s
W20260709 13:09:05.199436 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 767843us
I20260709 13:09:04.839854 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1306) took 2285 ms. Trace:
W20260709 13:09:05.208050 23417 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.002s	sys 0.006s
W20260709 13:09:05.208482 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 587217us
W20260709 13:09:05.208364 23417 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.002s	sys 0.006s
I20260709 13:09:04.704779 28365 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.209926 22897 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:05.208956 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.218825 22904 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:05.220009 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.220916 22904 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:05.224030 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1926) took 2690 ms. Trace:
I20260709 13:09:05.226933 22899 rpcz_store.cc:276] 0709 13:09:02.533456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.348508 (+815052us) service_pool.cc:224] Handling call
0709 13:09:03.354412 (+  5904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.533632 (+179220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.205087 (+1671455us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b540034cbd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:05.223953 (+ 18866us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400790784}
I20260709 13:09:05.201895 22905 rpcz_store.cc:276] 0709 13:09:02.555051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.601434 (+1046383us) service_pool.cc:224] Handling call
0709 13:09:03.602757 (+  1323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:03.603083 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:03.603109 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:03.603754 (+   645us) log.cc:844] Serialized 52 byte log entry
0709 13:09:04.153879 (+550125us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:04.320765 (+166886us) log.cc:844] Serialized 14 byte log entry
0709 13:09:04.329092 (+  8327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.329195 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:04.839623 (+510428us) raft_consensus.cc:1685] finished
0709 13:09:04.839646 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.839787 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7403,"prepare.queue_time_us":300361,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":12654,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":41439,"thread_start_us":305980,"threads_started":2,"wal-append.queue_time_us":615199}
W20260709 13:09:04.835863 28463 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.316s	user 0.001s	sys 0.000s
W20260709 13:09:04.845046 28419 consensus_meta.cc:287] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.715s	user 0.007s	sys 0.000s
W20260709 13:09:05.270038 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.265983 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:05.269358 22902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 1931) took 2680 ms (client timeout 2008 ms). Trace:
W20260709 13:09:05.272670 22902 rpcz_store.cc:269] 0709 13:09:02.534586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.560231 (+ 25645us) service_pool.cc:224] Handling call
0709 13:09:05.209578 (+2649347us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54003e9754. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:05.214629 (+  5051us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503931136}
W20260709 13:09:04.800915 28454 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.407s	user 0.000s	sys 0.001s
I20260709 13:09:05.023598 28193 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.274713 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:05.275100 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.818120 28460 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:04.833223 28214 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.202895 28192 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.276178 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:05.263231 28073 consensus_meta.cc:287] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 1.138s	user 0.010s	sys 0.012s
I20260709 13:09:05.309160 22822 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:05.315863 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:05.308987 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.309643 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:05.315125 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:05.210091 28465 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:04.838308 28459 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.437s
I20260709 13:09:05.321271 22830 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:05.313674 28473 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:05.322242 28473 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.324335 28473 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.324968 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:04.841303 28449 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.443s
W20260709 13:09:05.326890 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 128517us
I20260709 13:09:04.709261 28161 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.329485 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.335350 28214 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.337270 28214 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.337906 28465 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.340204 28214 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.340512 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1942) took 1271 ms. Trace:
I20260709 13:09:05.341013 22897 rpcz_store.cc:276] 0709 13:09:04.069164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.492545 (+423381us) service_pool.cc:224] Handling call
0709 13:09:05.209717 (+717172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:05.210343 (+   626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.340341 (+129998us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b54003e9750. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:05.340461 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1648563456}
I20260709 13:09:05.310483 28468 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:05.314548 28193 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.274899 28365 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:09:05.275691 28419 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.346096 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:05.326674 27694 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.348910 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:05.350255 28167 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:05.344543 28240 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.862s	user 0.012s	sys 0.000s
I20260709 13:09:05.351801 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:05.355249 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.358144 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:05.358541 22896 raft_consensus.cc:2410] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:09:05.359309 22897 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:05.360577 22823 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:05.360819 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:05.328253 28073 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.358953 28248 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.352339 28214 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:05.372763 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.347820 28193 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.375612 28193 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.354542 28231 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:05.373651 22897 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:05.379138 23447 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:05.377760 28390 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.380756 28390 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.383157 28390 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:05.426685 23416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.000s	sys 0.008s
W20260709 13:09:05.428406 23416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.001s	sys 0.009s
W20260709 13:09:05.429484 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 677889us
I20260709 13:09:05.427879 28484 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:05.428789 28487 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:05.433172 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.373441 28214 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:05.446069 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:05.429961 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 226836us
I20260709 13:09:05.376199 28193 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:05.378757 28231 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:05.429797 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69033us
I20260709 13:09:05.438894 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1087) took 1838 ms. Trace:
I20260709 13:09:05.440076 28487 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.442832 28073 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.454423 22905 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:09:05.493479 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:05.494326 22902 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:05.427613 28457 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:05.486568 28465 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:05.704473 28465 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.705376 28457 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.707856 28465 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.628296 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.709489 28465 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.627534 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.710346 28465 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:05.712855 28465 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:05.346930 28468 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.583228 28147 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.714414 28147 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:05.715267 28147 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.453279 28167 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.715827 28468 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.596544 28154 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:05.493235 28487 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.716825 28167 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:05.717588 28154 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.471532 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2672) took 2821 ms. Trace:
W20260709 13:09:05.619001 28442 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.314s	user 0.016s	sys 0.030s
I20260709 13:09:05.725446 22756 rpcz_store.cc:276] 0709 13:09:02.650310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.650452 (+   142us) service_pool.cc:224] Handling call
0709 13:09:03.559684 (+909232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.560313 (+   629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.471301 (+1910988us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54003cfad0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:05.471463 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1951343488}
W20260709 13:09:05.726202 28442 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.421s	user 0.016s	sys 0.030s
W20260709 13:09:05.521570 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72624.9us
I20260709 13:09:05.577229 22897 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:05.575425 22899 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:09:05.734500 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:05.735423 28147 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:05.612345 22902 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:09:05.582588 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2567) took 2960 ms. Trace:
I20260709 13:09:05.737490 28154 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.737388 22903 rpcz_store.cc:276] 0709 13:09:02.621702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.138900 (+1517198us) service_pool.cc:224] Handling call
0709 13:09:04.139245 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:04.144165 (+  4920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:04.144207 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:04.144258 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:04.144308 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.582337 (+1438029us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b54002dd7d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:05.582390 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.582529 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10862848}
I20260709 13:09:05.578023 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.573246 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:05.454119 28214 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:05.741942 22896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1303) took 4330 ms (client timeout 1521 ms). Trace:
I20260709 13:09:05.742141 28376 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:05.742481 22896 rpcz_store.cc:269] 0709 13:09:01.411805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.411930 (+   125us) service_pool.cc:224] Handling call
0709 13:09:05.741465 (+4329535us) spinlock_profiling.cc:243] Waited 3.93 s on lock 0x7b540034d0d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:05.741698 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9034912640}
I20260709 13:09:05.742918 28376 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.744364 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2581) took 2335 ms. Trace:
I20260709 13:09:05.487237 28231 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:05.744809 22898 rpcz_store.cc:276] 0709 13:09:03.408310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.191046 (+782736us) service_pool.cc:224] Handling call
0709 13:09:04.191318 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:04.222285 (+ 30967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:04.222337 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:04.222434 (+    97us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:04.222465 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.743828 (+1521363us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b540034bcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:05.743907 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.744146 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70695296}
I20260709 13:09:05.747972 28495 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:05.749117 28154 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:05.750348 28147 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:05.750846 28147 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.751278 28147 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:05.701453 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.017s	sys 0.013s
W20260709 13:09:05.753544 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.017s	sys 0.013s
W20260709 13:09:05.754014 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 553463us
I20260709 13:09:05.755203 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:05.376811 28248 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.596671 28262 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.759172 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2585) took 2263 ms. Trace:
I20260709 13:09:05.702180 28277 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.626585 28323 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:05.761901 28376 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.762012 28323 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.764699 28323 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:05.765590 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.568866 28419 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.459453 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2671) took 2837 ms. Trace:
I20260709 13:09:05.686157 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:05.700975 28499 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:05.769830 28499 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.770675 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.771466 22755 raft_consensus.cc:2410] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:09:05.772158 28499 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.772305 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.759653 22901 rpcz_store.cc:276] 0709 13:09:03.496055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.276535 (+780480us) service_pool.cc:224] Handling call
0709 13:09:04.276804 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:05.363241 (+1086437us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:05.363469 (+   228us) log.cc:844] Serialized 14 byte log entry
0709 13:09:05.615401 (+251932us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b1800437c20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:05.637244 (+ 21843us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b1800437c20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:05.701445 (+ 64201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:05.701597 (+   152us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:05.701731 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.758620 (+ 56889us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54003bfbd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:05.758683 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.758935 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":203162,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":68,"raft.queue_time_us":10587,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":2508491264,"thread_start_us":273481,"threads_started":2,"wal-append.queue_time_us":202863}
I20260709 13:09:05.480372 28192 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:05.773499 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 239988us
I20260709 13:09:05.617092 28195 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:05.585480 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2896) took 1211 ms. Trace:
I20260709 13:09:05.777640 22823 rpcz_store.cc:276] 0709 13:09:04.373863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.376960 (+  3097us) service_pool.cc:224] Handling call
0709 13:09:05.585274 (+1208314us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b5400081e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:05.585425 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2235236096}
I20260709 13:09:05.777920 28195 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.622464 28348 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:05.767848 22747 rpcz_store.cc:276] 0709 13:09:02.621919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.622050 (+   131us) service_pool.cc:224] Handling call
0709 13:09:02.622294 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:02.768588 (+146294us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:03.058861 (+290273us) log.cc:844] Serialized 14 byte log entry
0709 13:09:03.133688 (+ 74827us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:03.133736 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:03.133783 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.459195 (+2325412us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54000776d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:05.459252 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.459389 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27011,"prepare.queue_time_us":56928,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":15877,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":1710,"spinlock_wait_cycles":335490304,"thread_start_us":333733,"threads_started":2,"wal-append.queue_time_us":243304,"wal-append.run_cpu_time_us":1311,"wal-append.run_wall_time_us":1053075}
I20260709 13:09:05.779815 28348 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.780431 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.476083 28371 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.584290 28370 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.590165 28373 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.470521 28396 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:05.635624 28437 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.403s	user 0.013s	sys 0.034s
I20260709 13:09:05.782617 22747 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:05.482400 22826 rpcz_store.cc:276] 0709 13:09:03.600305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.600452 (+   147us) service_pool.cc:224] Handling call
0709 13:09:03.600710 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:04.236104 (+635394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.438658 (+1202554us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54003813d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:05.438839 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1459320704}
I20260709 13:09:05.785303 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3202) took 2363 ms. Trace:
I20260709 13:09:05.785816 22748 rpcz_store.cc:276] 0709 13:09:03.421629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.421805 (+   176us) service_pool.cc:224] Handling call
0709 13:09:03.422216 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:03.423575 (+  1359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:03.423619 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:03.430738 (+  7119us) log.cc:844] Serialized 26 byte log entry
0709 13:09:03.599849 (+169111us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:09:03.599887 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.599950 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:05.635077 (+2035127us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:05.784631 (+149554us) raft_consensus.cc:1685] finished
0709 13:09:05.784671 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.785054 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":909,"raft.queue_time_us":6232,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":2369,"spinlock_wait_cycles":6428928,"thread_start_us":162136,"threads_started":1,"wal-append.queue_time_us":739819}
I20260709 13:09:05.788275 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:05.685180 22749 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:09:05.687932 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.789985 22750 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 8.
I20260709 13:09:05.790671 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.791455 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.791707 22749 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:05.793000 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.793669 28240 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:05.794164 28240 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.794269 22750 raft_consensus.cc:2410] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:09:05.794585 28240 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:05.795356 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.796826 22750 raft_consensus.cc:2410] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:09:05.689169 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.797835 22751 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:05.773989 28192 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.690518 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.800582 22752 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
W20260709 13:09:05.479598 28472 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:09:05.801098 28192 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.573686 28484 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:05.802541 28484 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.803845 28277 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.691486 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.805747 22754 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:09:05.724583 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.730091 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.804986 28484 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.808180 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:05.793040 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.809442 28396 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:05.809816 28449 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:05.810031 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:05.810561 28449 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.812904 28449 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:05.814136 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 240810us
I20260709 13:09:05.815331 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:05.815708 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:05.816704 28396 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:05.816957 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.817195 28396 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:05.817683 28396 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:05.819121 28277 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.822674 28348 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.824000 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:05.825585 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.826875 28494 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:05.827080 28240 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.827634 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.828368 22905 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:05.748818 28495 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.762579 28161 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.764617 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.773011 22755 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
W20260709 13:09:05.576920 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 249620us
I20260709 13:09:05.454128 28193 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.868456 28193 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:05.631387 28179 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.871323 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:05.576577 28200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.005s	sys 0.024s
W20260709 13:09:05.872494 28200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.171s	user 0.005s	sys 0.025s
I20260709 13:09:05.872311 22749 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:05.719306 28487 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.780637 28195 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.610514 28409 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.873875 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.784026 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.874516 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.875020 22747 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 11.
I20260709 13:09:05.875272 22756 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:09:05.876328 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.876955 22753 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:09:05.877933 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.878640 22754 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
W20260709 13:09:05.883570 22720 connection.cc:256] RPC call timeout handler was delayed by -3.0231276240000398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:09:05.884325 28487 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:05.884907 28487 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.885892 23453 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:05.888180 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.888665 28262 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:05.888845 23445 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:05.815358 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:05.829394 28240 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:05.844949 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.846786 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:05.854426 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.830351 28484 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.891664 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.892402 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.856154 28277 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:05.858078 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.892702 28484 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.895450 22896 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:05.895635 28459 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:05.867018 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.896477 28459 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.872375 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:05.892163 23449 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:05.898836 28459 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.900255 22905 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:05.903415 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.891666 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.891944 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.845139 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:05.906225 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 59690.1us
I20260709 13:09:05.889285 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.906036 22754 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:05.908541 28200 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:05.910584 23446 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:05.910995 23446 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:05.911298 28167 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:05.912042 23446 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.913182 23446 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:05.913914 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:05.914763 23447 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:05.915732 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.916361 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:05.917029 23451 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:05.917369 28248 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.918820 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.921586 28484 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:05.923338 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.928138 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:05.928331 23446 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:09:05.931358 23446 pending_rounds.cc:85] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 11
I20260709 13:09:05.931965 23446 pending_rounds.cc:107] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:09:05.932672 23446 raft_consensus.cc:2889] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:05.935905 28396 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.936935 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:05.888870 28487 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.937959 22901 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:05.941614 28262 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:05.907997 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:05.931466 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:05.862720 28499 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.948319 22750 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:09:05.967116 28214 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.968479 22756 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:05.970680 28426 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:05.973546 28214 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:05.973230 28499 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:05.988400 22748 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:05.991806 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 84014.6us
I20260709 13:09:05.997107 28473 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:06.040560 28473 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:06.056857 28473 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.059026 28459 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:06.057051 28231 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.064782 22753 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:06.065034 23449 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:06.083801 22751 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:06.094944 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:06.139467 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.158147 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.184041 22753 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
W20260709 13:09:06.185115 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 123390us
I20260709 13:09:06.187533 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.188431 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:06.190416 22751 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:05.945031 28147 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:06.192610 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.182480 28499 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.199571 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.200872 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.170573 28495 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.193686 28262 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:06.217741 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 393953us
I20260709 13:09:06.219008 22748 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:06.222285 22902 raft_consensus.cc:2368] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14.
I20260709 13:09:06.223613 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.225828 22750 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:09:06.226951 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.226030 22756 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:06.227572 22750 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:06.228343 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.233997 28231 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.197956 28193 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.235278 22747 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:06.237037 28193 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.239399 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.240140 22747 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:06.240878 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:06.241843 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 426597us
I20260709 13:09:06.243989 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.244269 22754 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:09:06.242517 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 372417us
I20260709 13:09:06.238015 28396 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.251312 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.250669 28167 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:09:06.252545 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.254745 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:06.258867 22902 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:09:06.259020 28167 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:06.259502 28167 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:06.260015 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.262986 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.269260 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.269338 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.272225 22756 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:06.272711 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.263751 28167 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:06.273958 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.276836 22753 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:06.280423 22748 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:06.282497 22751 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:06.270733 28147 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:06.286566 22898 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:09:06.286547 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:06.281427 22653 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 11 to 14, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 14 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.291272 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:06.286350 28395 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:06.296296 22754 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:09:06.296561 22751 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:06.283442 28282 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:09:06.299932 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 50018.6us
I20260709 13:09:06.299150 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:06.302450 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.305186 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:06.305001 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:06.306416 22901 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:06.298286 28282 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:06.311566 22751 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: []
W20260709 13:09:06.313822 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47269us
I20260709 13:09:06.308488 28495 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:06.319743 22898 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:06.317037 28495 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:06.321491 28495 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:06.322831 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.323702 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:06.323546 22653 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 10, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 10 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.324488 28449 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:06.325047 28449 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:06.317119 28200 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:06.326149 28449 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.331288 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:06.332687 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.325997 22653 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 11 to 12, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 12 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:06.334962 22747 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:06.337275 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.338374 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:06.339954 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:06.341291 28348 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:06.342250 28277 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:06.342458 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.342626 28277 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:06.337924 22754 raft_consensus.cc:2376] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14. Re-sending same reply.
I20260709 13:09:06.344295 28277 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:06.344317 22654 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 335) took 16876 ms (client timeout 15000 ms). Trace:
I20260709 13:09:06.345185 28262 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:06.351011 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.345412 28277 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:06.351948 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.359921 22901 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:09:06.362200 22750 raft_consensus.cc:2410] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:09:06.367482 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.348517 28167 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.368324 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:06.369151 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 220796us
I20260709 13:09:06.369254 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:06.370040 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.369290 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:06.372912 28200 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:06.375573 28171 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:06.348699 28195 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:09:06.368271 28395 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:06.381424 28396 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:06.413465 28147 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:06.414616 22653 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 13, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 13 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:06.418644 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 76336.9us
I20260709 13:09:06.392119 28420 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:06.392309 28171 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:06.422925 28167 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:06.423369 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 54191.2us
W20260709 13:09:06.424065 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54435.4us
I20260709 13:09:06.425499 28427 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:09:06.447000 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 75210.5us
I20260709 13:09:06.471312 28420 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.473714 28420 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.474277 28420 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:09:06.347925 22654 rpcz_store.cc:269] 0709 13:08:49.516268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.525635 (+  9367us) service_pool.cc:224] Handling call
0709 13:09:06.344081 (+16818446us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:58.126465 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:58.127171 (+   706us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:58.127217 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:08:58.127243 (+    26us) tablet.cc:662] Decoding operations
0709 13:08:58.250875 (+123632us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:58.250968 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:58.250998 (+    30us) tablet.cc:685] Acquiring locks for 26 operations
0709 13:08:58.255510 (+  4512us) tablet.cc:701] Row locks acquired
0709 13:08:58.255559 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:08:58.255726 (+   167us) write_op.cc:270] Start()
0709 13:08:58.255877 (+   151us) write_op.cc:276] Timestamp: P: 1783602538255664 usec, L: 0
0709 13:08:58.255899 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:08:58.512901 (+257002us) log.cc:844] Serialized 10152 byte log entry
0709 13:08:59.226734 (+713833us) op_driver.cc:464] REPLICATION: finished
0709 13:08:59.583774 (+357040us) write_op.cc:301] APPLY: starting
0709 13:08:59.592427 (+  8653us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:59.648887 (+ 56460us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:59.649017 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:00.028476 (+379459us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:00.028991 (+   515us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=52,mrs_lookups=0
0709 13:09:00.029044 (+    53us) write_op.cc:312] APPLY: finished
0709 13:09:00.035979 (+  6935us) log.cc:844] Serialized 228 byte log entry
0709 13:09:00.080736 (+ 44757us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:00.094044 (+ 13308us) write_op.cc:454] Released schema lock
0709 13:09:00.103771 (+  9727us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:00.145635 (+ 41864us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:00.145663 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":34554,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":311483,"apply.run_cpu_time_us":36393,"apply.run_wall_time_us":593748,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112807,"num_ops":26,"prepare.queue_time_us":802351,"prepare.run_cpu_time_us":15389,"prepare.run_wall_time_us":608566,"raft.queue_time_us":301021,"raft.run_cpu_time_us":29011,"raft.run_wall_time_us":313578,"replication_time_us":945946,"spinlock_wait_cycles":75520,"thread_start_us":706325,"threads_started":4,"wal-append.queue_time_us":308703,"wal-append.run_cpu_time_us":19302,"wal-append.run_wall_time_us":2194530}]]}
I20260709 13:09:06.487550 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.489444 28167 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:06.490061 28167 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:06.490576 28167 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:06.495234 28495 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:06.439607 28395 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.489073 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:09:06.515370 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 97567.8us
W20260709 13:09:06.517462 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 87954.5us
I20260709 13:09:06.500928 28167 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.430414 28282 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.522348 22653 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.522253 28147 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:06.536460 28348 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:06.536919 28348 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:06.537281 28427 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:06.537909 28348 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.539379 28348 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:06.527160 28495 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:06.542580 28193 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:06.550833 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:06.535216 28147 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:06.552171 22653 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 12, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 12 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:06.560514 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.561254 28167 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.565973 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:06.566589 28231 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.570680 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.570876 28231 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:06.571707 28495 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:06.572425 28495 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:06.573638 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:06.574075 28395 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.574668 28395 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:06.574946 22871 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:06.576476 28167 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:06.577037 28193 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:06.573078 28231 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:06.577594 28193 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.567986 22653 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 13 to 17, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:06.578157 28193 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:06.579455 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.580150 28521 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:06.559648 28147 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:06.585562 28521 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:06.585999 28521 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:06.587489 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:06.589543 28510 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:06.591162 28510 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:06.577601 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.582131 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:06.595010 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:06.583297 22653 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 13 to 17, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 17 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:06.584183 28195 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:06.600282 28195 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.602777 28195 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:06.609933 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:06.609930 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 89372.5us
I20260709 13:09:06.611630 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.613515 28473 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:06.613905 28473 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:06.614235 28473 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:06.614255 28262 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:06.614883 28262 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:06.615978 28262 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.618626 28231 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:06.619143 28231 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.619552 28231 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:06.622081 28449 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:06.622502 28449 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:06.612998 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:06.623517 28449 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.624267 28459 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:06.624912 28449 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:06.625911 28427 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:06.626408 28427 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:06.626856 28427 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:06.628103 28147 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.606211 22653 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 8 to 10, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 10 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:06.633253 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:06.635735 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.625236 28282 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:06.640338 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:06.637724 28395 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:06.668009 28395 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:06.668464 28395 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:06.645613 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:06.674238 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:06.680464 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 45019.2us
I20260709 13:09:06.647869 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:06.635023 22653 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 11 to 12, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 12 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.649631 28147 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:06.629824 28457 consensus_queue.cc:1175] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3b626c41600d4737a178ebab8a7ebfc2" responder_term: 10 status { last_received { term: 10 index: 10 } last_committed_idx: 9 last_received_current_leader { term: 10 index: 10 } }
I20260709 13:09:06.653913 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:06.654388 23446 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:09:06.654650 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 78453.6us
I20260709 13:09:06.644083 28214 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:06.637027 28262 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:06.657768 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:06.658135 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:06.659713 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:06.661533 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:06.663547 28427 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.667330 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:06.683209 23451 raft_consensus.cc:2410] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 15 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:09:06.649019 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:06.609944 28521 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:06.688899 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 94639.2us
I20260709 13:09:06.706007 22653 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 11, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 11 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.701974 23453 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:06.687494 28499 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:09:06.710009 23447 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
W20260709 13:09:06.689393 28514 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:06.707265 28525 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:06.715247 28525 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:06.717658 28525 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:06.709889 28420 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:06.723902 22653 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:06.723479 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 111534us
I20260709 13:09:06.728425 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.731748 22752 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:06.735733 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.739274 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.739382 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.740140 22749 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:06.740572 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.741150 22747 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:06.739923 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.742973 22755 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:06.745795 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.746956 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.748472 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:06.743278 22755 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:06.743193 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:06.759411 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.760357 28526 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:06.760720 28526 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:06.761066 28526 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:06.749998 28519 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:09:06.737130 28495 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:09:06.765036 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:06.769874 28171 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:06.770426 28171 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.770890 28171 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:06.774823 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 69427.5us
I20260709 13:09:06.778252 28526 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.778993 28499 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:06.782210 28499 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:06.782595 28499 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:06.783398 22755 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:06.784350 22755 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:06.786072 28526 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.786482 28420 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:06.796901 28147 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:06.797428 28147 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:06.797866 28147 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:06.798856 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:06.793747 28473 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.804203 22750 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:06.803649 22653 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 7 to 9, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 9 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.808301 28193 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.808871 28348 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:06.810680 28193 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.811244 28193 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:06.800431 28427 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:06.838697 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 75183.8us
I20260709 13:09:06.839007 22755 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:09:06.839385 22755 pending_rounds.cc:85] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 10
I20260709 13:09:06.840328 22755 pending_rounds.cc:107] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:09:06.840538 23445 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:06.843039 22749 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:09:06.841841 22755 raft_consensus.cc:2889] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:06.846235 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:06.847123 28524 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:06.847548 28524 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:06.847915 28524 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:06.848134 28526 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:06.848776 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:06.856937 22747 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:06.804955 28231 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.841710 28167 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.783398 28499 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.864868 22797 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:06.849207 28529 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:09:06.848904 28526 consensus_queue.cc:1243] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:09:06.871721 28521 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.871448 28495 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:06.874413 28473 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:06.875274 28495 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:06.875669 28495 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:06.876534 28495 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.877832 28495 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:06.879815 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.882532 28519 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:06.882923 28519 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:06.883776 28519 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.884313 28231 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.884981 28519 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:06.885454 22751 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:06.861790 28167 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:06.900314 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 47760.9us
I20260709 13:09:06.903045 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.900571 28529 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:06.904935 28529 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:06.905970 28529 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:06.885141 28499 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:06.910559 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:06.899744 22794 leader_election.cc:400] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14.
I20260709 13:09:06.912369 22794 leader_election.cc:403] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:06.916395 28526 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:06.919937 28524 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:06.926282 28524 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:06.928056 28525 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:06.929556 28533 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:06.930011 28533 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:06.930362 28533 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:06.912107 28147 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.937075 28420 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:09:06.937593 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:06.937794 28147 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.938454 28521 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:06.940070 22826 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:06.940223 28521 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:06.941476 28427 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:06.941244 28534 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:06.943758 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:06.952105 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:06.956228 28537 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.956461 28540 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:06.956951 28540 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.956763 28537 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.959003 28540 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.959158 28537 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.961949 22831 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:09:06.961951 28525 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:06.947706 28427 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:06.966645 28427 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.967489 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:06.968358 22828 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:06.946401 28395 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:06.971794 28395 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.943717 28167 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:06.983552 28420 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14.
I20260709 13:09:06.984931 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:06.988858 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:06.989933 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:06.990134 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:06.992892 22826 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:06.993862 28525 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:06.985687 28521 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:06.987357 22828 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:06.965111 28526 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:07.020172 28542 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:07.020787 28542 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.021378 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.021971 22901 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:07.023279 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.023519 28542 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:07.023823 22901 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:07.024044 28542 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.024511 28542 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:07.025027 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.025439 28167 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:07.025923 28167 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.026450 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.027881 28167 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:07.028767 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.028909 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.029359 22902 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:07.029414 22898 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:07.027851 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.030304 22897 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:07.030540 22901 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:07.032023 28542 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:07.033910 22905 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:06.983239 28524 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:07.035089 22901 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:09:07.036437 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.036763 22823 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:07.037164 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.994314 28525 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:07.040210 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 84092.5us
I20260709 13:09:07.042217 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.044770 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:07.045869 23450 raft_consensus.cc:2410] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 19 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:09:07.046352 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.046929 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.048310 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.048647 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:07.049444 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.051774 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:07.052445 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:07.035832 28524 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:07.054389 22896 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:07.055660 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.054055 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:06.988858 28171 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.058635 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:07.059664 23450 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:07.059203 23447 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:09:07.061617 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.062494 22902 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:07.042786 28525 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:07.066260 22901 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:09:07.068454 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:07.058797 28524 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:07.072432 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.076259 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:07.076926 22827 raft_consensus.cc:2410] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:09:07.089027 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.091823 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.061203 28526 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:07.095913 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.098129 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.100201 22904 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:07.116752 22747 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:07.069773 28533 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:07.116582 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:07.122822 22898 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:07.128168 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.128672 28533 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:07.130067 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:07.109999 28535 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:07.135396 22829 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:07.141505 22824 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:09:07.148299 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 349484us
I20260709 13:09:07.149658 28524 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.151490 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.152958 28524 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:07.154367 28171 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:07.130436 28473 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:09:07.155846 28473 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:07.156215 28473 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:09:07.165455 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 74385.2us
I20260709 13:09:07.166236 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:07.173360 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 47862.4us
I20260709 13:09:07.173668 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:07.174661 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.174852 28499 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:07.175315 28499 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:07.175676 28499 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:07.176091 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:07.157019 28473 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.179024 28473 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:07.176051 28427 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:07.186811 28282 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:07.187386 28282 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:07.187727 28282 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:07.190994 28348 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:07.192149 28449 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:07.192528 28449 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:07.174669 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.199060 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:07.200343 22749 raft_consensus.cc:2368] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20.
I20260709 13:09:07.193639 22797 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.205955 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:07.206269 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:07.206629 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:07.207044 22749 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:07.208403 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:07.221879 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.224170 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.225904 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:07.222992 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:07.238809 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:07.238157 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:07.246093 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.207911 28526 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:07.220572 28262 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:07.247998 22896 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:07.166973 28510 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
W20260709 13:09:07.254347 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:07.255437 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 80827.3us
I20260709 13:09:07.245563 28524 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:07.215489 28427 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:09:07.269548 28527 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:09:07.248873 28526 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:07.272794 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:07.262943 28520 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:07.243716 28529 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:07.264957 28459 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:07.277056 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:07.277067 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 83496.3us
I20260709 13:09:07.265825 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.273553 22898 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:07.194293 28449 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.249197 28262 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:07.285657 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:07.286242 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.288139 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:07.301759 22719 connection.cc:256] RPC call timeout handler was delayed by -1.247491255s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 13:09:07.302146 22719 connection.cc:256] RPC call timeout handler was delayed by -1.1727912549999928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 13:09:07.302420 22719 connection.cc:256] RPC call timeout handler was delayed by -1.1286912550000352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
I20260709 13:09:07.304294 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.305183 28541 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:07.305601 28541 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:07.305956 28541 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:07.272007 28526 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:07.307121 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.308383 28231 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:07.308743 28231 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:07.309088 28231 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:07.310088 23453 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:09:07.311127 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.311666 23450 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:07.274664 28520 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:07.289697 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:07.333082 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 127190us
I20260709 13:09:07.333808 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.334913 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.291462 28534 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:07.292447 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.292730 23447 raft_consensus.cc:2410] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:09:07.313078 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.340603 23444 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:07.260179 28524 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.326611 23446 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:07.327615 23452 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
W20260709 13:09:07.337642 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 45358.9us
I20260709 13:09:07.277263 28459 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:07.298377 28527 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:07.319329 28262 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:07.362140 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:07.362545 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 123813us
I20260709 13:09:07.363266 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:07.363987 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.367465 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:07.339090 28520 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:07.358063 28561 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.377827 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.341308 28171 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:09:07.386549 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 79509.8us
I20260709 13:09:07.344842 28395 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:07.392042 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.357676 28524 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:07.385300 22898 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:07.359684 28459 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:07.359884 28527 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:09:07.402838 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 56392.6us
I20260709 13:09:07.369632 28396 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:07.405375 22871 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:07.405807 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:07.407570 22871 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:07.400662 28495 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:07.409436 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:07.393193 28171 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.412730 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:07.413967 22896 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:07.402119 28519 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:07.415629 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.371392 28534 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:09:07.420728 22871 leader_election.cc:400] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20.
I20260709 13:09:07.421732 22871 leader_election.cc:403] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:09:07.429505 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet ab1914222f984a8f9db2e3956a97bad0: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:07.410770 28525 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.439344 23444 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:07.443219 23450 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:07.410998 28167 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:07.413132 28535 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:07.404790 28533 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:09:07.415561 28171 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:07.466893 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60908.3us
I20260709 13:09:07.487671 28396 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:07.521158 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.416966 28395 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:09:07.530640 28559 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 13:09:07.494338 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.525794 28534 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:07.558494 22896 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:07.524228 28262 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:07.531461 28554 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:09:07.525004 28544 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:07.657012 28167 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:07.665428 22900 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 22: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:07.666312 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:07.666072 28167 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:07.666651 28262 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:07.658468 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 253263us
I20260709 13:09:07.667241 28262 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:07.667728 28262 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:07.658474 28579 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:07.433267 28525 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:07.441787 28564 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:07.669787 28564 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.670181 28564 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:07.670353 22896 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:07.659456 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:07.672943 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.488128 28526 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.674711 28526 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:07.472386 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51826.4us
I20260709 13:09:07.571043 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.398741 28561 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:07.470954 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 128365us
I20260709 13:09:07.601612 28499 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:07.617475 28231 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:07.623569 28558 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.354s	user 0.006s	sys 0.000s
I20260709 13:09:07.567633 28527 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:07.630700 28565 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 13:09:07.638160 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 274939us
I20260709 13:09:07.905766 28262 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:07.906438 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:07.907635 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:07.412861 28537 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
W20260709 13:09:07.909766 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 511874us
I20260709 13:09:07.910663 28573 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:07.911108 28573 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:07.467083 28195 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:07.911376 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:07.911490 28573 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:07.911964 28195 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:07.912349 28195 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:07.912492 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:07.912735 28195 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:07.905184 28171 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:07.915165 22900 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:07.668787 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.598372 28282 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.916241 28560 consensus_queue.cc:1059] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:07.917250 28560 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:07.917733 28282 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:07.672281 28396 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20.
I20260709 13:09:07.676194 28395 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:07.919087 23446 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:07.919412 28395 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:07.633203 28541 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.921039 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.921955 22828 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:07.922699 28541 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:07.777093 28557 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.505s	user 0.004s	sys 0.016s
I20260709 13:09:07.923148 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.925083 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.915756 28562 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:09:07.927407 28562 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:07.927743 28562 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:07.928520 28562 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:07.836637 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 168220us
I20260709 13:09:07.897380 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.930011 28195 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.931202 28541 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:07.931849 28541 consensus_queue.cc:1243] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:09:07.925081 28282 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:09:07.933749 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:07.934633 28541 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:07.934056 28282 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:07.935137 28541 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:07.935232 28282 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:07.936154 28541 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.936746 28282 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.938143 28282 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:07.941093 22903 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:07.942680 22903 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:07.942961 28277 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:07.943341 28277 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:07.944185 28277 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.945291 28277 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:07.949906 28556 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.679s	user 0.004s	sys 0.009s
I20260709 13:09:07.932091 28195 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:07.953545 22897 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:09:07.953121 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:07.954185 28544 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:07.954602 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.954803 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:07.955482 28395 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:07.955951 28395 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:07.956826 28395 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.957924 28395 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:07.897956 28564 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:07.898504 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.899179 28569 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:07.900519 28524 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.901162 28231 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:07.675751 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:07.971292 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 295585us
I20260709 13:09:07.972011 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.972431 28524 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:07.964982 28519 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:07.973109 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:07.902308 28459 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.973875 28519 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:07.975448 28459 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:07.902832 28499 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:07.581419 28510 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.975821 28569 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.977001 28167 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:07.977686 28167 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:07.978329 28167 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:07.978632 28569 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:07.976732 28510 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.982043 28510 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:07.486145 28563 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.256s
I20260709 13:09:07.909085 22899 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:07.918195 28568 raft_consensus.cc:3055] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:07.922979 28561 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:07.960664 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48239.5us
I20260709 13:09:07.960340 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:07.962278 28427 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:09:07.967098 28171 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:07.967625 28572 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:07.967146 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.963251 28262 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:07.975045 28519 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:07.999543 28427 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:07.999950 28427 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:08.002310 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.003427 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.003975 22751 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:08.006618 28537 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.007117 28537 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:07.999572 28262 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:08.009826 28262 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:08.011566 28555 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.748s	user 0.006s	sys 0.000s
I20260709 13:09:08.013495 28572 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:08.014423 28572 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:08.016741 28231 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.017213 28231 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:07.985980 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:07.986722 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.024930 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:07.997092 23445 pending_rounds.cc:85] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 11
I20260709 13:09:07.991385 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.988406 28568 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:07.996031 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:07.963599 28579 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:09:07.903275 28520 consensus_meta.cc:287] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.502s	user 0.015s	sys 0.003s
I20260709 13:09:08.030010 23445 pending_rounds.cc:107] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:09:08.039096 23445 raft_consensus.cc:2889] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:08.032186 22756 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
W20260709 13:09:08.031927 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.025539 22754 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:08.040206 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:07.993908 28499 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:08.041581 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:07.997453 28562 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:08.045694 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:08.044744 22719 leader_election.cc:336] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 2.008s (SENT)
I20260709 13:09:08.000778 28427 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:08.052067 22868 leader_election.cc:336] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.565s (SENT)
I20260709 13:09:08.056248 23450 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:08.052028 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.053128 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.060225 22902 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:09:08.060371 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:08.061214 22898 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:08.037529 28420 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:09:08.063175 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.064271 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.065002 22897 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:08.065562 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.066241 22900 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:08.066565 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.067078 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.067893 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.068522 22831 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:08.068885 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.069527 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.069947 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.070171 28231 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:08.048840 28449 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:08.070585 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.070815 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.071528 22903 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:08.072293 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.059543 28195 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:08.068972 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.073276 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.078577 22903 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 22: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:08.073561 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.079982 22901 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:08.074433 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.075022 22828 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:08.075826 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.076012 22899 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:08.077056 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:08.070595 22830 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:08.089941 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.076597 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:08.101768 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:08.103091 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.047314 28459 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:08.108912 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:08.047004 28579 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:08.111598 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.095722 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.115386 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.115598 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.116318 22901 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:08.116556 22900 raft_consensus.cc:2393] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 9: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:08.060058 28167 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.057999 28499 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:08.117733 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.125830 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.044283 28520 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.043581 28535 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:09:08.114975 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:08.132401 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.133107 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:08.053633 28396 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:08.169292 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:08.169595 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.134807 22902 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:09:08.132354 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:08.137914 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45076.9us
I20260709 13:09:08.172709 22900 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:08.140133 23452 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:08.176257 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.177372 22901 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:09:08.142740 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.179122 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:08.179378 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.182379 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.182888 22902 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: []
W20260709 13:09:08.184827 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 140089us
I20260709 13:09:08.188241 23444 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:08.190186 22756 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:08.191649 22900 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:08.187523 22827 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:08.167313 28427 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:08.197584 22750 pending_rounds.cc:85] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 7
I20260709 13:09:08.197971 22750 pending_rounds.cc:107] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:09:08.198269 22750 raft_consensus.cc:2889] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:08.200832 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:08.206642 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.141397 28167 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:08.217602 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 165543us
I20260709 13:09:08.226867 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.242305 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.175889 28396 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:08.247960 22756 pending_rounds.cc:85] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 6
I20260709 13:09:08.248287 22756 pending_rounds.cc:107] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:09:08.248603 22756 raft_consensus.cc:2889] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:08.250772 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.251400 22828 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:09:08.250794 22826 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:08.256245 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:08.217870 28510 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:08.268105 22825 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:08.275875 22898 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:08.210470 28473 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:09:08.208473 28561 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:09:08.289651 22829 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:08.265020 28262 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:08.258953 28564 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:08.294616 22822 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:08.274868 28277 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:08.299441 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.311630 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:08.311674 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 112996us
I20260709 13:09:08.314393 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.316650 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.317483 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.318255 22824 raft_consensus.cc:2410] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:09:08.318725 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.322078 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.274049 28569 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:09:08.258109 28499 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.290944 28579 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.335950 28598 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:08.290763 28528 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:09:08.344914 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.346179 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.346253 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.351157 22751 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:08.314227 28277 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:08.321169 28171 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.356233 28577 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 13:09:08.357200 28595 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:09:08.361512 28521 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:08.363269 22830 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:08.364688 22831 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:08.355472 22752 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:09:08.366564 22828 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:08.336777 28570 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:08.421545 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 146063us
I20260709 13:09:08.426942 22831 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:09:08.369232 28541 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:08.433324 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 182977us
I20260709 13:09:08.369503 28521 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:08.436389 22828 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:08.438148 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.417112 28520 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:08.443136 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 354184us
I20260709 13:09:08.445883 23452 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:08.369531 28171 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:08.447193 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 102347us
I20260709 13:09:08.448166 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.448060 28277 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.438839 28231 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.454278 22823 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
W20260709 13:09:08.469877 28593 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.149s	user 0.001s	sys 0.001s
W20260709 13:09:08.470552 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.471725 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.428352 28570 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:08.473495 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.477945 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.478545 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.481935 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.478682 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.483016 22902 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:08.487638 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.489508 22825 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:09:08.489849 22825 pending_rounds.cc:85] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 13
I20260709 13:09:08.489876 28570 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:08.490247 22825 pending_rounds.cc:107] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:09:08.490553 22825 raft_consensus.cc:2889] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:08.495218 28573 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.000s
I20260709 13:09:08.495658 28573 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.472402 28525 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:08.496687 23450 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14. Ops: []
W20260709 13:09:08.497838 28395 consensus_meta.cc:287] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.003s
I20260709 13:09:08.459043 28231 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.499833 28231 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:08.501490 28600 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:09:08.503861 22903 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:08.504727 22824 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:08.479156 28624 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:08.509624 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 43718.4us
I20260709 13:09:08.513443 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.515241 28603 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:09:08.516141 28603 consensus_queue.cc:1243] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:09:08.516611 28603 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:08.516961 28603 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:08.518326 28615 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:09:08.519408 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.521915 28619 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:08.523694 22897 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:08.524266 22896 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:08.526293 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:08.526577 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:08.527073 22750 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:08.527884 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.528560 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.532630 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.534334 28573 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:08.501251 28262 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:08.538182 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.538684 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:08.530160 28231 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:09:08.555925 23446 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:08.552711 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.570389 22750 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:08.553660 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.581543 22902 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:08.584055 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:08.585259 28592 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:09:08.502238 28521 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.536540 28564 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:09:08.588076 28603 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:08.615767 28631 consensus_queue.cc:1059] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:09:08.616307 28613 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:09:08.615131 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:08.614866 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.622735 22902 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:08.620002 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 92177.3us
W20260709 13:09:08.619937 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.624698 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.595209 28262 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:08.591835 28499 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:08.631013 28632 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:08.635246 22750 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:09:08.642918 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.642609 22902 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:08.645797 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.650213 28597 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:08.650722 28597 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.652875 28597 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:08.627940 28631 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:08.628309 28521 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:08.666710 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.632855 28262 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:08.670414 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.670550 28262 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:08.671540 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.672225 22829 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:08.672861 28167 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:08.674172 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:08.675659 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.676102 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.676987 22827 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:08.678053 22827 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:08.681329 28171 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:08.682253 22827 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:09:08.688668 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:08.700878 22797 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.702765 28583 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.703409 28583 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.705619 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:08.691154 28231 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:08.707474 28231 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:08.709623 28634 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:08.705027 28568 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.711110 28568 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:08.712982 28510 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:08.713436 28510 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:08.714389 28510 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:08.715802 28629 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:09:08.715989 28510 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:08.722169 28583 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:08.723421 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.723606 28231 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:08.691609 28631 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:08.727782 22822 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:09:08.731189 28541 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:08.709884 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.737603 28396 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:08.741592 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.745432 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.750998 28524 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:08.751382 28524 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:08.752197 28524 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:08.753078 28524 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:08.765380 28628 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:08.753445 28631 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.754935 28544 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:08.769786 22754 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:08.754768 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:08.755419 28171 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:08.756048 28396 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:08.779515 28167 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:08.780381 28529 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:08.781515 28537 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:08.757688 28527 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:08.781765 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:08.785471 28634 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:08.788357 28624 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:08.788882 28632 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:08.757123 28495 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:09:08.790956 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:08.770292 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 69511.7us
I20260709 13:09:08.770926 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:08.772554 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:08.795185 23446 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:08.795372 23444 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 12.
I20260709 13:09:08.794423 22747 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:08.796723 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:08.794489 22831 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:08.801000 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.798540 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:08.803866 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:08.805465 23419 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:08.807503 28277 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:08.808204 28573 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:08.808794 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.808739 28573 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.811022 28573 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:08.811866 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:08.813633 28277 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:08.814409 28612 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:08.816306 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:08.801967 28535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.015s	sys 0.006s
I20260709 13:09:08.817214 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:08.817178 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:08.820751 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 175047us
I20260709 13:09:08.819939 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:08.819854 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:08.804396 28563 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:08.805531 28167 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:08.827489 28630 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:09:08.840758 28167 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:08.805003 28449 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:08.841799 28449 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.842245 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:08.842981 28195 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:08.800477 28562 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:09:08.843451 28195 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:08.846719 28541 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.847056 28195 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.847219 28541 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:08.827800 28147 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:08.828007 28611 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:09:08.828163 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:08.800325 28529 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:09:08.819922 28544 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:08.824342 28348 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:08.830137 28262 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:08.830554 28634 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:08.831457 28528 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:08.832969 28573 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:08.833176 28638 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:08.826287 28561 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
W20260709 13:09:08.889871 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43904.9us
I20260709 13:09:08.833804 28560 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:08.835151 28568 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:08.835958 28603 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:08.816062 28420 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:09:08.838603 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:08.838791 28459 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:08.821062 28535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.016s	sys 0.006s
I20260709 13:09:08.856225 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:08.858095 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.859179 23450 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:08.858882 28147 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.860608 28495 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:08.860869 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:08.884269 22828 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:08.887338 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.886716 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.888670 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.902824 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:08.891400 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 96325.1us
I20260709 13:09:08.826429 28632 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:09:08.893476 22824 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:08.893697 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.917672 22903 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:08.898337 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:08.860154 28348 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.898753 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.859217 28544 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:08.892511 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.886036 28537 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:09:08.861719 28231 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:08.903849 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.900288 28612 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:08.925554 28231 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:08.907346 28420 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:08.925972 28231 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:08.926154 28420 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:08.927098 28420 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.928256 28420 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:08.908875 28599 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:08.892053 28560 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:08.937530 22905 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
W20260709 13:09:08.938089 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46404.2us
I20260709 13:09:08.899549 28459 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:08.853718 28624 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:09:08.915653 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57006.4us
I20260709 13:09:08.892385 28449 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:08.918674 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:08.892736 28541 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:08.901921 28147 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:08.930827 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.937489 28528 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:08.952052 28562 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:08.919377 28171 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:08.919533 28348 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:08.921983 28544 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:09.001489 22905 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:09.002914 28282 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:09.003301 28282 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:09.004123 28282 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.005918 28537 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:08.964864 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.962790 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:08.960701 28262 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:08.972301 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:08.893265 28579 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:08.937145 28529 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:08.976352 28611 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:09.020102 28262 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:09.016245 28579 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:09.021530 28533 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:09.021965 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:09.022615 22823 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:08.978370 28473 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:08.932929 28599 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:08.929551 28277 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:09:08.989990 28395 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:08.990180 28459 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:08.991740 22898 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:08.992874 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:08.993539 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:08.996829 28528 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:09:08.993563 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 18a127a20cfa4496bad51cb634bd0d0f: Illegal state: Replica 6a5191aaff76469bbf6c4735dc9e6509 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:09.032276 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:08.962893 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44265.8us
I20260709 13:09:09.027407 28395 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:09.008142 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 105650us
I20260709 13:09:09.012903 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:09.017784 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45547.6us
W20260709 13:09:09.019986 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57284.3us
I20260709 13:09:09.026365 28473 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:09.027511 22752 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:09:09.030689 22900 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:09.031109 22903 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:09.038564 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:09.005259 28282 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:09.042228 28395 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.043504 22901 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:09:09.044420 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:09.038708 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:09.021548 28579 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.066929 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:09.036442 28449 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.068452 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:09.069852 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:09.071887 22903 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:09:09.072206 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:09.075083 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:09.067977 28518 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.061781 28171 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:09.078462 28171 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.078843 28171 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:09.073046 28449 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.080051 28473 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.075731 28634 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:09.078092 28518 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.081589 28634 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:09.067631 28599 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:09.068750 28541 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:09.083235 28599 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:09.085958 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.085696 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:09.087548 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:09.086462 22753 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:09.084457 28395 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:09.064546 28643 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:09:09.144551 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.073906 28628 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:09.167788 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.254312 22748 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:09.075981 28632 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:09.087788 28644 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:09.087581 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:09.202654 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 115246us
I20260709 13:09:09.266743 28627 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:09.267053 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:09.210002 28646 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:09:09.216466 22829 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:09:09.170225 28529 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:09.086616 28262 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 13:09:09.216957 28231 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:09.224967 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.251287 28651 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.265858 28499 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:09.223026 28544 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:09.252506 28645 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:09:09.268147 22829 pending_rounds.cc:85] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 8
I20260709 13:09:09.307991 22829 pending_rounds.cc:107] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:09:09.308315 22829 raft_consensus.cc:2889] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:09.310886 28449 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:09.269196 28562 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:09.311759 28562 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.312130 28562 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:09.269464 28262 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:09.315245 28262 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:09.316042 28262 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.320387 22900 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:09.271245 28231 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:09.273757 28651 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.050540 28528 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.283234 28473 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:09.317202 28262 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:09.326645 28631 consensus_queue.cc:1175] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3b626c41600d4737a178ebab8a7ebfc2" responder_term: 13 status { last_received { term: 13 index: 12 } last_committed_idx: 10 last_received_current_leader { term: 13 index: 12 } }
I20260709 13:09:09.240056 28524 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:09.329530 28524 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:09.330410 28524 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.331337 28524 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:09.069111 28533 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:09.332005 28528 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:09.242460 28560 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:09.247776 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 4e4b3212e80646379ccf77b5952a4363: Illegal state: Replica 3b626c41600d4737a178ebab8a7ebfc2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:09.333446 28651 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:09.240770 28348 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.248582 28521 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:09.265612 28644 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:09.335081 28521 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:09.335191 28644 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:09.188845 22904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:09.335976 28521 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.336553 22904 raft_consensus.cc:606] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:09.340011 28560 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:09.340191 23446 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 15: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:09.280474 22897 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:09.288048 28563 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:09.345747 28563 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:09.346496 28563 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.347003 22904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:09.347697 28563 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:09.347754 22904 raft_consensus.cc:606] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:09.289077 28579 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:09.306941 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 52881.8us
I20260709 13:09:09.352294 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:09.303342 28518 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:09.353402 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:09:09.265889 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 178406us
I20260709 13:09:09.354035 28529 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:09.267920 28603 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:09.355036 28529 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:09.281878 28643 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:09.277352 28499 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:09.357129 28643 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.358309 28499 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.359354 28499 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:09.218664 28611 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:09.335985 28348 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:09.279744 28544 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:09.337209 28521 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:09.290639 28537 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:09.370605 22904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:09.371434 22904 raft_consensus.cc:606] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:09.371949 22904 raft_consensus.cc:614] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:09.374394 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:09.267796 28627 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:09:09.387674 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.408046 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:09:09.408295 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:09.408085 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.379720 28495 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.413486 28147 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.416340 28147 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:09.417590 22899 raft_consensus.cc:2368] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14.
I20260709 13:09:09.413508 28603 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:09.413851 28495 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.422401 28617 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:09.377038 28533 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:09:09.395542 28651 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.380070 28529 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:09:09.405571 28627 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:09.406440 28282 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.425794 28171 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.425971 28562 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.431249 28644 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:09.433436 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.433971 22654 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 12, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 12 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:09.434945 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:09.438477 28420 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:09.475984 22654 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 14, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 14 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:09:09.442914 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.403664 28544 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.511791 22654 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:09.445600 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 370546us
I20260709 13:09:09.513917 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:09.450838 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet ccab55924ea9473faf1d4d97313e9284: Illegal state: Replica 3b626c41600d4737a178ebab8a7ebfc2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:09.514271 22654 catalog_manager.cc:5697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.467151 28147 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.456761 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:09.457931 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:09.518064 22827 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:09.473726 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:09.518651 22825 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:09.476889 23444 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:09.519299 22905 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:09:09.455981 28603 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:09.520170 22823 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:09:09.519915 22654 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:09.480584 22828 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:09.455576 22829 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:09.523000 22654 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:09.524371 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.526129 22654 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 13, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 13 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:09.465243 28651 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.531173 22654 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:09:09.540172 22653 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 341) took 19152 ms (client timeout 15000 ms). Trace:
I20260709 13:09:09.542418 22654 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 14 to 17, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.535645 23416 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:09.472339 28282 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:09.551373 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 281138us
W20260709 13:09:09.559773 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.562284 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:09.540699 22653 rpcz_store.cc:269] 0709 13:08:50.387913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.405444 (+ 17531us) service_pool.cc:224] Handling call
0709 13:09:09.540072 (+19134628us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:07.117985 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:07.136698 (+ 18713us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:07.136763 (+    65us) write_op.cc:435] Acquired schema lock
0709 13:09:07.136789 (+    26us) tablet.cc:662] Decoding operations
0709 13:09:07.138306 (+  1517us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:07.138414 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:07.138452 (+    38us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:09:07.139941 (+  1489us) tablet.cc:701] Row locks acquired
0709 13:09:07.139995 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:09:07.149075 (+  9080us) write_op.cc:270] Start()
0709 13:09:07.149359 (+   284us) write_op.cc:276] Timestamp: P: 1783602547148987 usec, L: 0
0709 13:09:07.157260 (+  7901us) op_driver.cc:348] REPLICATION: starting
0709 13:09:07.245520 (+ 88260us) log.cc:844] Serialized 5099 byte log entry
0709 13:09:07.431744 (+186224us) spinlock_profiling.cc:243] Waited 9.9 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb71cf10ac 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b
0709 13:09:07.952215 (+520471us) op_driver.cc:464] REPLICATION: finished
0709 13:09:08.168262 (+216047us) write_op.cc:301] APPLY: starting
0709 13:09:08.184386 (+ 16124us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:08.500079 (+315693us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:08.500124 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:08.764238 (+264114us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:08.764526 (+   288us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:09:08.764576 (+    50us) write_op.cc:312] APPLY: finished
0709 13:09:08.764939 (+   363us) log.cc:844] Serialized 121 byte log entry
0709 13:09:08.765111 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:08.765253 (+   142us) write_op.cc:454] Released schema lock
0709 13:09:08.769035 (+  3782us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:08.843733 (+ 74698us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:08.846576 (+  2843us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":366364,"reported_tablets":46,"spinlock_wait_cycles":21248,"child_traces":[["op",{"apply.queue_time_us":137253,"apply.run_cpu_time_us":24711,"apply.run_wall_time_us":690437,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58467,"num_ops":13,"prepare.queue_time_us":177114,"prepare.run_cpu_time_us":38981,"prepare.run_wall_time_us":794245,"raft.queue_time_us":291388,"raft.run_cpu_time_us":43951,"raft.run_wall_time_us":83373,"replication_time_us":794764,"spinlock_wait_cycles":22761344,"thread_start_us":871529,"threads_started":4,"wal-append.queue_time_us":400960}]]}
I20260709 13:09:09.521853 28544 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:09.571259 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.533181 28650 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:09.549626 22654 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.568872 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:09.525444 28147 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.571305 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:09.580700 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:09.582026 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.582628 22754 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:09:09.583065 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.574180 22654 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:09.558354 28579 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:09.613894 28579 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.614826 28147 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:09.616341 28579 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:09.586238 28644 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:09.586756 28603 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.618532 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:09.619278 22754 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:09.609054 22827 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:09.587455 28638 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:09.588511 28171 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:09.573642 28262 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:09.591720 22824 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:09:09.591890 28544 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:09.592629 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:09:09.598032 28649 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.075s	user 0.000s	sys 0.002s
W20260709 13:09:09.598549 28652 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:09:09.599555 23453 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:09:09.600373 23450 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:09:09.601855 28495 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:09.602042 23444 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:09.603138 28282 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:09.645967 28644 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.646569 28644 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.605670 28634 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:09.607331 22752 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:09:09.608361 22748 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:09:09.609510 28605 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:09:09.619338 28541 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:09:09.633665 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 98708.7us
I20260709 13:09:09.638240 22755 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:09.636336 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:09.585093 28499 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:09.640779 28518 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:09.631086 28560 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:09.630654 22654 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:09.673019 22654 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:09.619130 28348 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.632575 28638 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:09.636447 28262 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:09.559825 28627 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.676424 22654 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:09.618920 28617 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:09:09.692317 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:09.681188 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:09.683493 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:09.702512 23450 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:09.702457 23447 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 9.
I20260709 13:09:09.710372 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:09.710093 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:09.729280 22754 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:09.733623 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:09.668048 28282 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.674649 28654 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:09.747933 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:09.753695 23450 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:09.679806 28634 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:09.689906 28262 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:09.675202 28560 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:09.752027 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:09.777252 22903 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:09:09.752434 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:09.749884 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:09.778452 22754 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:09.778812 22904 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:09.667439 28521 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:09:09.681618 28541 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:09.694496 22654 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 14, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 14 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:09.791599 22654 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:09.789469 22754 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:09.766796 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:09.724916 28568 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:09.725085 28495 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:09.795478 22901 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:09.736022 28171 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.802065 22905 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:09:09.795524 22897 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:09.696141 28561 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:09:09.727002 28650 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:09.680169 28348 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.777077 28562 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:09.705065 28624 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.098s
I20260709 13:09:09.791015 28560 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:09.696341 28605 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:09.775817 28634 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:09.696447 28528 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:09.820991 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 52044.7us
I20260709 13:09:09.823354 23416 leader_election.cc:400] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14.
I20260709 13:09:09.804715 28537 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:09.802386 28568 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.793720 22654 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.838485 22654 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 11 to 14, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 14 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:09.757349 28282 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:09.832300 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 210223us
I20260709 13:09:09.840503 22654 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.777359 28654 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.857791 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:09.863282 22654 catalog_manager.cc:5697] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:09.884407 22755 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:09:09.885648 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:09.705176 28628 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.133s
I20260709 13:09:09.826746 28634 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:09.891366 22654 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:09.892956 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:09.893654 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.836320 28537 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.894459 22748 raft_consensus.cc:2410] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:09:09.908476 22901 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:09.843598 28568 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:09.811741 28495 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.909569 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:09.927181 22827 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:09.895344 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:09.930295 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.931044 22755 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:09.909834 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:09.933547 22823 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:09.829248 28651 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:09.914783 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.935608 22748 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:09.916361 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:09.918705 22905 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:09:09.814888 28171 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.938405 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.939126 22748 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
W20260709 13:09:09.893819 28563 consensus_meta.cc:287] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.004s
I20260709 13:09:09.948820 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:09.949627 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 54360.8us
I20260709 13:09:09.969353 22829 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 15: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:09.971815 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:09.984534 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:10.047962 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.836546 28605 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.919739 28537 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:10.050745 22824 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
W20260709 13:09:10.055603 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 84511.6us
I20260709 13:09:09.818620 28650 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:09.917619 28534 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.915951 28598 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.838584 28528 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.917846 28659 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:10.081557 22901 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:09:10.120431 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:10.131935 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 247217us
W20260709 13:09:10.046396 28637 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
W20260709 13:09:10.154634 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 329946us
W20260709 13:09:10.132409 28658 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.429s	user 0.001s	sys 0.000s
I20260709 13:09:09.805584 28638 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:10.235710 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:09.981458 28167 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:10.191934 22897 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:09:09.945919 28643 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:10.093621 28653 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.197439 22823 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:10.200662 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:09.944682 28541 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:09.970832 28573 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.198s
I20260709 13:09:10.236449 22748 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 9.
I20260709 13:09:10.053617 28529 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:10.247849 28348 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:10.256099 22829 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:10.269551 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 299512us
W20260709 13:09:10.233271 28660 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.066s	user 0.003s	sys 0.000s
I20260709 13:09:10.194028 28624 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 13:09:10.254323 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:10.278395 22755 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:10.265336 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:10.215797 28659 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:10.284735 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.285506 22823 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:10.244834 28628 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:09:10.283344 28657 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.462s	user 0.001s	sys 0.000s
W20260709 13:09:10.265060 28661 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 13:09:10.268574 28576 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.261013 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:10.287868 22868 leader_election.cc:336] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.584s (SENT)
I20260709 13:09:10.313578 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:10.200135 28534 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:10.205914 28598 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.265066 28473 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.279462 22752 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:10.295022 28628 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:10.315300 22824 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:10.351977 28628 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:10.352732 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.353489 22824 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:10.354478 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.355415 22824 raft_consensus.cc:2410] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:09:10.356258 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.357055 22824 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:10.317564 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:10.357975 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:10.358435 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.359835 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.278540 28579 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:10.268864 28529 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:10.363379 28529 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.363660 28654 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.283833 28624 consensus_queue.cc:1243] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:09:10.332435 28420 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.366345 28420 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.369726 28420 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:10.340101 28675 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:09:10.370568 28600 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:10.327021 28473 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.370954 28600 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:10.371806 28600 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.372721 28600 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:10.372898 28473 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.373749 28282 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.376662 28650 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.343619 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:10.344262 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.377756 22831 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:10.344869 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.378466 22830 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:10.379915 28495 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:10.380520 28495 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:10.346024 22828 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:09:10.347302 28673 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:09:10.346566 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.350867 28598 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:10.351323 28677 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:10.320225 28576 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:10.342538 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.349236 28534 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.336345 28644 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.384394 22755 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
W20260709 13:09:10.309512 22719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.011s	sys 0.011s
I20260709 13:09:10.310246 28603 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.392642 22826 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:09:10.395061 22719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.011s	sys 0.011s
W20260709 13:09:10.395407 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 661824us
I20260709 13:09:10.278162 28653 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.400547 22829 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:09:10.381660 28231 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:10.404629 22827 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:10.404965 23450 pending_rounds.cc:85] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 14
I20260709 13:09:10.406491 23450 pending_rounds.cc:107] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:09:10.406827 23450 raft_consensus.cc:2889] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:10.399536 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 192011us
I20260709 13:09:10.391785 28684 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:09:10.409260 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:10.412812 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.516283 28628 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.527381 22830 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:09:10.527976 22830 pending_rounds.cc:85] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 9
I20260709 13:09:10.528201 22830 pending_rounds.cc:107] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:09:10.528416 22830 raft_consensus.cc:2889] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:10.529249 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:10.529896 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.532658 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:10.533720 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.538623 28576 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.539047 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:10.542129 28684 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:10.542729 28684 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.543141 22756 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:10.543609 22751 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:10.544890 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:10.546169 28659 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:10.546787 28659 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.547312 28348 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.398492 28603 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:10.549603 28528 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.415403 28231 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.416821 28624 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.544809 28684 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:10.483258 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 73276.6us
I20260709 13:09:10.650452 23419 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:10.399663 28643 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:10.396216 28568 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.653049 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:09:10.647941 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 409582us
I20260709 13:09:10.648844 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:10.654124 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:10.671294 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.527513 28653 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.634927 28524 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:10.696017 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.406674 28537 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.699926 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:10.704957 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:10.706045 22751 raft_consensus.cc:2410] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:09:10.645483 28528 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.672217 28282 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:10.647511 28611 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:09:10.711282 28231 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:10.711472 28617 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:09:10.714330 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60272.6us
I20260709 13:09:10.647727 28576 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
W20260709 13:09:10.702796 28560 consensus_meta.cc:287] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.867s	user 0.009s	sys 0.002s
I20260709 13:09:10.670668 28684 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:10.701146 28420 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.719898 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:10.716152 28282 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.726648 28282 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:10.672971 28521 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:09:10.717483 28644 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:10.626757 28659 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:10.647570 28167 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:10.636751 28348 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.701721 28526 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:09:10.719457 28573 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:10.770290 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 50456.9us
I20260709 13:09:10.719384 28617 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:10.705518 28524 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.652730 28624 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:10.769284 28695 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:09:10.728829 28473 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:10.727974 28628 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:10.786180 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:10.796173 22871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.019s	sys 0.015s
I20260709 13:09:10.769851 28644 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:10.789253 22899 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.007s	sys 0.029s
W20260709 13:09:10.798874 22899 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.008s	sys 0.029s
W20260709 13:09:10.782363 28544 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.031s	sys 0.089s
I20260709 13:09:10.776234 28563 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:09:10.783793 28634 consensus_meta.cc:287] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.854s	user 0.011s	sys 0.000s
W20260709 13:09:10.796720 22871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.019s	sys 0.015s
W20260709 13:09:10.807662 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 537391us
I20260709 13:09:10.784610 28535 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.728530 28560 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.784548 28617 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:10.768141 28420 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:10.811726 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2654) took 1242 ms. Trace:
I20260709 13:09:10.769675 28684 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:10.807109 28544 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.033s	sys 0.089s
I20260709 13:09:10.816476 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:10.772264 28528 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.806586 28644 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:10.842965 23447 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:10.844619 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:10.785567 28573 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.818521 22899 rpcz_store.cc:276] 0709 13:09:09.569275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.569448 (+   173us) service_pool.cc:224] Handling call
0709 13:09:09.569752 (+   304us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:09.574364 (+  4612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:09.574393 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:09.587915 (+ 13522us) log.cc:844] Serialized 52 byte log entry
0709 13:09:09.700087 (+112172us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b18000b8420. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:09:09.905358 (+205271us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:10.799370 (+894012us) log.cc:844] Serialized 14 byte log entry
0709 13:09:10.799700 (+   330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:10.799796 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:10.799846 (+    50us) raft_consensus.cc:1685] finished
0709 13:09:10.811515 (+ 11669us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:10.811675 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":510763,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":1880,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":10516,"spinlock_wait_cycles":72889856,"thread_start_us":1031395,"threads_started":2,"wal-append.queue_time_us":390157}
I20260709 13:09:10.771278 28643 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.817008 28518 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:10.824369 28696 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.871953 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3271) took 1028 ms. Trace:
W20260709 13:09:10.871569 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 332592us
I20260709 13:09:10.814713 28535 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.814128 28634 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.887701 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:10.889307 22750 rpcz_store.cc:276] 0709 13:09:09.843949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.844125 (+   176us) service_pool.cc:224] Handling call
0709 13:09:09.894583 (+ 50458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:10.214223 (+319640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.214256 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:10.239498 (+ 25242us) log.cc:844] Serialized 26 byte log entry
0709 13:09:10.412294 (+172796us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:10.412345 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:10.412406 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:10.871735 (+459329us) raft_consensus.cc:1685] finished
0709 13:09:10.871757 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:10.871882 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":373584,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":9572,"thread_start_us":432463,"threads_started":2,"wal-append.queue_time_us":445509}
I20260709 13:09:10.872468 28654 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.891494 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.869715 28167 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:10.893754 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.894097 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:10.895462 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:10.889060 28518 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:10.902313 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:10.893359 28701 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:10.896253 28544 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:09:10.903908 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.904429 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:10.898761 28692 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:10.906431 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.007s	sys 0.031s
W20260709 13:09:10.909912 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.007s	sys 0.031s
W20260709 13:09:10.904541 28262 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.102s	user 0.006s	sys 0.006s
W20260709 13:09:10.928696 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.891696 28696 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.905618 28701 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.899106 28654 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.904302 28537 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:10.906924 28700 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:10.908442 28544 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:10.955837 28696 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:10.946620 28526 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.927479 28603 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.987546 28348 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.988355 28420 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:10.988981 28171 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:10.991950 28600 consensus_meta.cc:287] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.000s
I20260709 13:09:10.992935 22831 raft_consensus.cc:2410] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:09:10.978439 28544 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:10.943377 28659 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:10.946709 28697 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:09:10.958921 23447 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:10.980381 28573 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.980731 28535 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:10.980851 28560 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:10.980933 28699 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:09:10.981922 28710 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:09:10.982388 28624 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:10.982733 28634 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:10.945297 28598 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.985605 28528 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:10.985741 28526 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:10.986634 28708 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:10.928320 28473 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:09:11.004236 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 95082.6us
I20260709 13:09:11.039670 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:10.950311 28701 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:10.949976 28534 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.941836 28147 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:10.942070 28262 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:10.929749 28692 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.040637 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.040859 22822 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
W20260709 13:09:11.041883 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 197365us
I20260709 13:09:11.042953 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:11.044884 23444 raft_consensus.cc:2410] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:09:11.046123 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:11.048208 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.033296 28563 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:11.049638 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.050284 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:11.050535 22828 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:11.052778 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.041548 28684 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:11.049891 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.062351 28717 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:11.055038 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.057237 28262 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.057734 22754 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:11.063338 23449 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:11.065718 28605 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:11.067260 28692 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:11.069223 28563 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.071354 28605 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:11.071964 28605 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.074559 28605 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.062513 22826 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:11.044451 28534 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.077260 28719 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.076928 28473 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.077754 28719 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.078810 28534 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.075784 28684 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.079847 28719 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.079591 28684 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.080709 28719 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:11.081184 28719 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.081619 28684 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.083089 28719 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.084360 28719 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.084818 28719 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.085976 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.086788 28719 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.086911 28696 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:11.087430 28696 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.087589 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.087932 28696 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:11.088338 22903 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:11.038121 28598 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:11.092514 28704 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:09:11.093580 28598 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.043536 28537 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:11.043777 28147 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:11.095078 28147 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:11.103500 28662 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.218s	user 0.011s	sys 0.000s
I20260709 13:09:11.108769 28651 raft_consensus.cc:993] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:11.110934 28524 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.076236 28526 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.113739 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.115429 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.114508 23445 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:11.122462 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.123193 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.124222 23445 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:11.125259 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.125671 28651 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.126093 23449 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:11.126333 28651 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.128775 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:11.129132 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 76746.9us
I20260709 13:09:11.134788 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.139200 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.134491 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.142203 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.146821 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:11.147037 22868 leader_election.cc:336] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.690s (SENT)
I20260709 13:09:11.147646 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:11.147981 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.150283 28598 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:11.154448 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.155907 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.156090 23444 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:11.135562 28526 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:11.133190 28537 raft_consensus.cc:1081] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:11.155356 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.158213 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.150827 28535 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.182536 23445 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:11.148238 28654 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.158627 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.187600 28147 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.154642 28719 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:11.193233 28654 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:11.193811 28654 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.195920 28654 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.198529 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:11.150957 28599 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:11.201663 28599 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.202474 28599 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:11.207628 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:11.208330 23450 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:11.209432 28147 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.209944 28651 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.210851 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.157450 28526 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.216490 22824 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:11.155812 28598 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.175661 28644 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.190809 28535 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.223872 28644 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.178725 28617 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.178128 28282 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.218564 28526 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:11.234295 23452 raft_consensus.cc:2393] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 10: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:11.224920 28598 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:11.237890 28701 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.238301 28617 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.239312 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2658) took 1379 ms. Trace:
I20260709 13:09:11.240571 28535 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.240836 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:11.240586 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:11.244948 28518 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.257244 23450 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:11.241065 28696 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.246803 28282 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.234671 28544 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.268059 22824 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:11.268455 28544 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:11.269639 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1498) took 1556 ms. Trace:
I20260709 13:09:11.270001 23447 rpcz_store.cc:276] 0709 13:09:09.709780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.709894 (+   114us) service_pool.cc:224] Handling call
0709 13:09:11.265843 (+1555949us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400750154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:11.265917 (+    74us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2560255744}
I20260709 13:09:11.256459 28537 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.270957 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:11.271152 28537 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.271592 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:11.272918 22719 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.536s (SENT) [suppressed 31 similar messages]
I20260709 13:09:11.249722 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3268) took 1441 ms. Trace:
I20260709 13:09:11.250766 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.273213 28537 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.279932 28628 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.280616 28611 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:11.280861 28499 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.281059 28611 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:11.281409 28499 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.281915 28611 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.282685 28701 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.283391 28499 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.274262 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.275245 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.287310 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.287506 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.288053 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:11.288374 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:11.288863 22904 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:11.274240 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.289949 23447 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:11.273275 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:11.256672 22902 rpcz_store.cc:276] 0709 13:09:09.859770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.978800 (+119030us) service_pool.cc:224] Handling call
0709 13:09:10.047635 (+ 68835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:10.080880 (+ 33245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.080932 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:10.081011 (+    79us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:10.081060 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.239095 (+1158035us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b5400107554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:11.239142 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.239257 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75877760}
I20260709 13:09:11.292595 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.293257 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.293823 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:11.278810 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 557264us
I20260709 13:09:11.298727 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.299397 22896 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:11.291124 22754 rpcz_store.cc:276] 0709 13:09:09.807912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.813652 (+  5740us) service_pool.cc:224] Handling call
0709 13:09:09.813971 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.063616 (+1249645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.249492 (+185876us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540016b054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:11.249639 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859950464}
I20260709 13:09:11.293537 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:11.301271 22903 raft_consensus.cc:2410] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:09:11.301352 28579 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:11.300096 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.287865 22899 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:11.301926 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:11.302244 22905 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:11.302727 23447 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:11.303138 22830 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:11.301939 28579 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.280591 28628 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.306214 28628 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.307186 22902 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 13:09:11.300760 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.300663 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.308629 22901 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:11.309609 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.310178 22822 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:11.312698 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.313443 22902 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:11.315719 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.316396 22902 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:11.318343 23452 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:11.318828 22822 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 13:09:11.320263 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:11.310927 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.322965 22830 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:11.279145 28541 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.324765 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:11.328584 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.324285 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.329382 22901 raft_consensus.cc:2410] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 21 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:09:11.329736 22822 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:11.324872 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.331126 22755 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:11.341055 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.341697 22896 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:11.346773 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.347353 22896 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:11.352861 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.354125 22823 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:11.322144 28634 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.356945 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.361860 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2652) took 1793 ms. Trace:
I20260709 13:09:11.370024 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:11.362349 22900 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 16. Ops: []
I20260709 13:09:11.365846 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2105) took 1023 ms. Trace:
I20260709 13:09:11.366461 28535 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.367914 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.376816 22831 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:11.368723 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.369884 22898 rpcz_store.cc:276] 0709 13:09:09.568750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.570295 (+  1545us) service_pool.cc:224] Handling call
0709 13:09:09.570481 (+   186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:09.626588 (+ 56107us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:09.641174 (+ 14586us) log.cc:844] Serialized 14 byte log entry
0709 13:09:09.822682 (+181508us) log.cc:844] Serialized 14 byte log entry
0709 13:09:09.828286 (+  5604us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:09.841764 (+ 13478us) log.cc:844] Serialized 26 byte log entry
0709 13:09:09.841952 (+   188us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:09.843421 (+  1469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:10.671995 (+828574us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b18000b8480. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:11.250348 (+578353us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b18000b8480. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb79d24aa1 00007fcb79d1bb37 00007fcb79d1b9ca 00007fcb79e1326f
0709 13:09:11.361551 (+111203us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b5400207954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:11.361601 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:11.361646 (+    45us) raft_consensus.cc:1685] finished
0709 13:09:11.361661 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.361808 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":923,"prepare.queue_time_us":908599,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":6402,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":17909,"spinlock_wait_cycles":195103616,"thread_start_us":121835,"threads_started":1,"wal-append.queue_time_us":612405}
I20260709 13:09:11.373713 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:11.374749 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:11.371007 22827 rpcz_store.cc:276] 0709 13:09:10.342103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.342239 (+   136us) service_pool.cc:224] Handling call
0709 13:09:11.361223 (+1018984us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b54002ca454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:11.365778 (+  4555us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27688192}
I20260709 13:09:11.322551 28671 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:11.389631 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.418706 22902 raft_consensus.cc:2393] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:11.352280 28611 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:11.329947 28624 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.422719 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:11.425467 22897 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:11.425272 22904 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:11.428757 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.430119 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:11.430663 22756 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:11.355726 28541 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.434463 28541 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.434893 28541 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.435298 28541 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:11.435676 28706 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.410s	user 0.000s	sys 0.002s
I20260709 13:09:11.414594 28524 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:11.438308 22756 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:09:11.414650 28529 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.443555 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.442174 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.443033 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.424726 28537 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.444730 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.444976 22756 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:11.445992 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.446180 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.446779 22748 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:11.447765 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.448287 22752 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:11.419886 28534 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:11.450629 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.440126 28628 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.451691 28628 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.453986 28628 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.454006 28599 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.456489 28541 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.428192 28624 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.458545 28624 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.424728 28535 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.460515 28644 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:11.460750 28679 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:11.450115 28473 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:09:11.457757 28599 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.439486 28524 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.462231 28524 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:11.436605 28632 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.460629 28535 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.464715 28529 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.465411 28632 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:11.466820 28696 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.377951 28634 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.467777 28632 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:11.460523 28518 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:11.460637 28262 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:11.468649 28262 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.469094 28634 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.470173 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 1049) took 1787 ms. Trace:
I20260709 13:09:11.471571 28262 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.470569 23453 rpcz_store.cc:276] 0709 13:09:09.682684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.682819 (+   135us) service_pool.cc:224] Handling call
0709 13:09:09.736594 (+ 53775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:09.934537 (+197943us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:09:10.296110 (+361573us) log.cc:844] Serialized 14 byte log entry
0709 13:09:10.645761 (+349651us) log.cc:844] Serialized 14 byte log entry
0709 13:09:10.645936 (+   175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:10.645996 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:10.646057 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:10.710558 (+ 64501us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b1800263f80. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:11.223450 (+512892us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b1800263f80. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:11.469904 (+246454us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b5400567cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:11.469955 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.470078 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":464835,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":62,"raft.queue_time_us":547,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":162,"spinlock_wait_cycles":470457344,"thread_start_us":587296,"threads_started":2,"wal-append.queue_time_us":513830}
I20260709 13:09:11.472873 28537 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.474198 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 1053) took 1075 ms. Trace:
I20260709 13:09:11.474570 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.474987 28537 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.474694 28524 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.418768 28671 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.476182 28568 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:11.475947 28534 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.523010 28671 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.517382 28579 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.477483 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:11.524080 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 251166us
W20260709 13:09:11.524268 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 46872.7us
W20260709 13:09:11.524652 22719 leader_election.cc:336] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.703s (SENT)
I20260709 13:09:11.525024 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:11.449374 28231 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:11.479486 22749 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
W20260709 13:09:11.479987 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.481285 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.474598 23446 rpcz_store.cc:276] 0709 13:09:10.398726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.398857 (+   131us) service_pool.cc:224] Handling call
0709 13:09:10.399181 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:10.645618 (+246437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.645649 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:10.646289 (+   640us) log.cc:844] Serialized 52 byte log entry
0709 13:09:10.845480 (+199191us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:10.845529 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.473814 (+628285us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54000b2cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:11.473857 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:11.473901 (+    44us) raft_consensus.cc:1685] finished
0709 13:09:11.473913 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.474143 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":526,"raft.queue_time_us":76030,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":6598,"spinlock_wait_cycles":366168320,"thread_start_us":136696,"threads_started":1,"wal-append.queue_time_us":201300}
I20260709 13:09:11.490212 22756 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:11.497666 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:11.527704 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 76626.9us
I20260709 13:09:11.499243 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:11.500087 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.528527 22747 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:11.500695 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:11.502717 28638 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:11.504525 28526 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.506583 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.519017 28529 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.527868 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.529515 23453 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:11.529776 22750 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:11.527081 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:11.536432 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.540658 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.547439 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.536777 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:11.551395 22896 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:09:11.552423 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.552935 22896 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:11.553180 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.553025 22898 raft_consensus.cc:1240] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:11.554430 22904 raft_consensus.cc:2410] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:09:11.559262 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:11.560187 22756 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:09:11.534118 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.560608 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.561738 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.564849 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.565502 22747 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:11.563760 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.569358 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.570273 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.568306 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:11.571692 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.569366 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.576488 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.602378 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.603381 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.570835 22750 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:11.605222 22897 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:09:11.528913 28395 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:11.576095 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:11.609195 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.575101 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.608151 22830 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:11.614701 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 1986) took 1723 ms. Trace:
I20260709 13:09:11.609426 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.604084 22896 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:11.616343 28473 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:11.616752 28473 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:11.617117 28473 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:11.617282 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.618331 28653 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:11.618754 28653 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.619112 28653 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:11.621716 28262 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:11.622670 28644 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:11.623111 28644 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.623467 28644 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:11.558147 28671 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:11.631074 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.558579 28579 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:11.632503 28579 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.634796 28579 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.557879 28651 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.635727 28651 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.558554 28624 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.615152 22900 rpcz_store.cc:276] 0709 13:09:09.891418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.164004 (+272586us) service_pool.cc:224] Handling call
0709 13:09:11.362190 (+1198186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.370697 (+  8507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.614501 (+243804us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400207950. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:11.614631 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2641761024}
I20260709 13:09:11.615486 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:11.632633 28653 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.644747 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:11.558133 28541 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.557847 28696 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:11.659086 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.682490 22753 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:11.683701 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:11.684332 22753 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:11.659687 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:11.562933 28700 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.588258 28524 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.533631 28231 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.604674 28713 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:11.661726 28624 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.604327 28167 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:11.696636 28167 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.697016 28167 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:11.604857 28562 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:11.563174 28147 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:11.706059 28147 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.707223 28231 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.708544 28147 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:11.567377 28719 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.567269 28518 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:11.709384 28518 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.709753 28518 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:11.705531 28562 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.710644 28562 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:11.710702 28713 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.711216 28713 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:11.568755 28282 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.095s
I20260709 13:09:11.662830 28535 consensus_queue.cc:1059] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:11.610572 28395 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.663893 28717 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:11.655004 28534 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.665588 28627 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:11.666002 28495 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:11.666702 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.667115 28526 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.669188 28644 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.637821 28651 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.673130 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:11.678761 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:11.791815 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 118745us
I20260709 13:09:11.559243 28731 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:11.608695 28528 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:11.672183 28653 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.643424 23453 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
W20260709 13:09:11.690304 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 78565.2us
I20260709 13:09:11.692005 28541 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.691104 28700 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.694432 28537 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:11.705521 28521 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.714259 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:11.682253 28696 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.681815 28677 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:11.709401 28719 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.609681 28730 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.120s
I20260709 13:09:11.693418 28628 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:11.737913 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:11.739027 22900 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:11.740025 22756 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:11.820710 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:11.728296 28395 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:11.802922 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:11.793370 23419 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.576543 28680 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.112s
I20260709 13:09:11.804780 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.811187 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:11.801426 28499 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.052s	sys 0.026s
I20260709 13:09:11.809789 22901 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:11.728653 28679 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 13:09:11.832944 28688 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:11.816504 22749 raft_consensus.cc:2368] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14.
I20260709 13:09:11.801541 28528 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.836279 28528 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:11.837234 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:11.838261 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:11.839154 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.842118 22749 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:09:11.842971 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.843569 22749 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:11.844668 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.846174 22749 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:11.823622 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.824735 22756 raft_consensus.cc:2393] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 10: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:11.814563 28696 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:11.847692 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:11.848142 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.848521 22749 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:09:11.825850 22898 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
W20260709 13:09:11.849136 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.848821 22756 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:11.829833 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:11.827081 22752 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:11.830953 22750 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:11.850297 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:11.850838 22756 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:11.851141 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:11.830668 28499 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.052s	sys 0.027s
I20260709 13:09:11.804683 28563 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:11.809039 28731 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.854221 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:11.854936 22748 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:11.816704 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 248858us
I20260709 13:09:11.812693 28537 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:11.899464 22904 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: []
W20260709 13:09:11.902180 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48031.8us
I20260709 13:09:11.903456 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.904713 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:11.855473 28650 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:11.863916 28534 raft_consensus.cc:3055] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:11.870532 28719 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.874125 28541 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:11.877444 28651 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:11.878093 28644 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:11.880460 28624 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.815104 28521 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.856456 23419 leader_election.cc:400] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14.
I20260709 13:09:11.893599 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:11.850052 28677 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:11.865162 22752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:09:11.860282 22756 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:09:11.921216 23446 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:11.921561 22756 pending_rounds.cc:85] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 7
I20260709 13:09:11.909634 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:11.921947 22756 pending_rounds.cc:107] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:09:11.908746 22747 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:11.921036 23419 leader_election.cc:403] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:09:11.922259 22901 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:11.922425 22756 raft_consensus.cc:2889] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:11.912819 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:11.921595 22752 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:11.923473 22752 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:11.907265 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:11.877156 28473 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:11.938725 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:11.975584 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 129676us
W20260709 13:09:11.981123 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 321718us
I20260709 13:09:11.982129 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:11.978212 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:11.985359 22825 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:11.904054 28628 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:09:11.987337 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48689.9us
I20260709 13:09:11.988477 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:11.989475 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 421311us
I20260709 13:09:11.991302 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:11.992434 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:11.983708 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:11.908450 28526 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:11.912209 28651 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:11.912009 28534 raft_consensus.cc:740] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:11.906909 28650 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:11.908262 28231 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.126s
I20260709 13:09:12.061730 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:12.062409 22747 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:09:11.919224 28730 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.123s
I20260709 13:09:12.017015 28518 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.036345 28708 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:12.033582 28688 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:12.026855 28529 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:11.987125 28654 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14.
I20260709 13:09:11.913669 28563 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:12.109377 28563 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.111795 28563 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:11.924253 28677 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:11.982503 28739 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:09:11.923766 28576 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:11.974588 28680 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:11.920177 28729 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:09:12.121623 23416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.024s	sys 0.021s
I20260709 13:09:12.036088 28701 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:12.125975 23453 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:12.047019 28147 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:12.076927 22753 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:12.164196 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:12.165748 22753 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:09:12.079507 22825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2107) took 1723 ms (client timeout 1967 ms). Trace:
I20260709 13:09:12.079643 23449 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
W20260709 13:09:12.079751 22797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.034s	sys 0.083s
W20260709 13:09:12.167143 22797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.034s	sys 0.083s
W20260709 13:09:12.167651 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 611587us
I20260709 13:09:12.080639 23446 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:12.083019 22901 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:12.083565 22749 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:12.169099 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.012027 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:12.085206 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.170145 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:12.106844 28518 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:12.069574 28598 consensus_meta.cc:287] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.813s	user 0.006s	sys 0.007s
I20260709 13:09:11.987344 28719 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:12.164191 22752 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:12.121961 23416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.024s	sys 0.021s
I20260709 13:09:12.063822 28650 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:12.060702 28534 consensus_queue.cc:260] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.059697 28526 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:12.170399 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 85274.2us
I20260709 13:09:12.121019 28677 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:12.234972 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:12.171152 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:12.236181 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 65095.9us
I20260709 13:09:12.236838 22797 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:12.169262 22747 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:12.175824 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:12.237984 23446 raft_consensus.cc:2368] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
I20260709 13:09:12.123543 28708 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:12.166946 22825 rpcz_store.cc:269] 0709 13:09:10.342746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.343391 (+   645us) service_pool.cc:224] Handling call
0709 13:09:12.066140 (+1722749us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54002314d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:12.066265 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3066206976}
W20260709 13:09:12.179163 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 561873us
I20260709 13:09:12.041872 28167 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:12.186506 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 174541us
I20260709 13:09:12.114833 28700 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:12.186107 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:12.172228 28679 raft_consensus.cc:2764] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:09:12.242794 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:12.282505 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:12.282594 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 96585.1us
I20260709 13:09:12.294121 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.187826 28495 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:12.294889 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:12.295076 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:12.288214 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:12.241601 22752 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:12.177359 28624 raft_consensus.cc:2764] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:09:12.213526 28147 raft_consensus.cc:697] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:12.213191 28701 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:12.234230 28617 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:12.294842 28729 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:12.305251 22871 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:12.305605 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:12.308889 22747 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:12.164824 28576 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.306572 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:12.184640 28598 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.315366 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:12.316071 23446 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:12.238596 28524 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:12.295006 28717 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:12.294553 28749 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:12.318401 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:12.375013 22747 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:12.297561 28541 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:09:12.339556 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.172689 28651 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.323381 23419 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:12.398344 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 75034.6us
W20260709 13:09:12.390693 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 72350us
W20260709 13:09:12.403484 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 464633us
I20260709 13:09:12.297438 28680 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:12.308344 28617 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:12.404145 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.404150 23446 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:12.407045 22824 raft_consensus.cc:2368] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
I20260709 13:09:12.312525 28700 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:12.313938 28729 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:12.384342 28682 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.021s	sys 0.036s
W20260709 13:09:12.397668 28499 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.023s	sys 0.049s
I20260709 13:09:12.384629 28717 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.325407 28537 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:12.389613 28749 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:12.383481 28495 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.323041 28751 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:12.420799 28524 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:12.442008 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:12.406896 28719 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:12.402202 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:09:12.433400 28682 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.022s	sys 0.036s
I20260709 13:09:12.400188 28731 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:12.419426 28603 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:12.436566 28499 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.023s	sys 0.049s
I20260709 13:09:12.434090 28617 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:12.382449 28568 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:09:12.419148 28521 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:12.452767 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 50641.2us
I20260709 13:09:12.433686 28729 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:12.440331 28717 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:12.449411 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 214478us
I20260709 13:09:12.448560 28751 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:12.461549 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:12.494557 23446 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:12.474426 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:12.475296 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:12.476131 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:12.476979 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:12.497844 22749 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 15: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:12.477813 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:12.478768 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 241980us
I20260709 13:09:12.443822 28749 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:12.438068 28680 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:12.490170 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:12.470508 28741 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.406s	user 0.001s	sys 0.001s
I20260709 13:09:12.455442 28651 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:12.523216 22900 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:09:12.458357 28563 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:12.448570 28495 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.526468 22904 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:12.527647 28737 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
W20260709 13:09:12.528393 28743 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:09:12.453816 28603 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:12.532624 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:12.492551 28719 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:12.524623 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:12.495230 28521 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.551930 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:12.451330 28701 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.444362 28526 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.506016 28751 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:12.568198 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 164076us
W20260709 13:09:12.539752 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 140800us
I20260709 13:09:12.584909 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.579593 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.588333 22900 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:12.558525 28534 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:12.563922 28537 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:12.553622 28745 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:12.469516 28147 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.554845 28563 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:12.585209 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:12.598737 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.600265 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.568343 28603 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:12.576980 28719 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:12.577639 28167 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:12.605983 23419 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:12.608085 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:12.611279 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1151) took 1295 ms. Trace:
I20260709 13:09:12.606029 22896 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
W20260709 13:09:12.600528 28713 consensus_meta.cc:287] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.889s	user 0.011s	sys 0.000s
I20260709 13:09:12.614511 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2744) took 1306 ms. Trace:
I20260709 13:09:12.619804 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:12.600885 28528 consensus_meta.cc:287] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.002s
I20260709 13:09:12.621640 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2746) took 1132 ms. Trace:
W20260709 13:09:12.628047 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.621909 22755 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:12.622169 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.630107 22900 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:12.618285 22754 rpcz_store.cc:276] 0709 13:09:11.307952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.317486 (+  9534us) service_pool.cc:224] Handling call
0709 13:09:11.317758 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:11.363804 (+ 46046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.363836 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:11.465668 (+101832us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b2c00027958. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:09:11.466010 (+   342us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.784214 (+318204us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:11.784260 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.784333 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.599993 (+815660us) raft_consensus.cc:1685] finished
0709 13:09:12.600016 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.614452 (+ 14436us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45697,"raft.queue_time_us":46101,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":11591552,"thread_start_us":317798,"threads_started":1,"wal-append.queue_time_us":443103}
I20260709 13:09:12.611764 22826 rpcz_store.cc:276] 0709 13:09:11.309484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.309836 (+   352us) service_pool.cc:224] Handling call
0709 13:09:11.310103 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:11.310430 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.310464 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:11.310892 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.738267 (+427375us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:11.738308 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.738371 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.604918 (+866547us) raft_consensus.cc:1685] finished
0709 13:09:12.604939 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.605055 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8025,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":1017,"thread_start_us":362035,"threads_started":1,"wal-append.queue_time_us":761734}
I20260709 13:09:12.627895 22751 rpcz_store.cc:276] 0709 13:09:11.488946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.489069 (+   123us) service_pool.cc:224] Handling call
0709 13:09:11.499652 (+ 10583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:11.518314 (+ 18662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.518348 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:11.518806 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.886372 (+367566us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:11.886412 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.886474 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.621450 (+734976us) raft_consensus.cc:1685] finished
0709 13:09:12.621472 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.621591 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18283,"raft.queue_time_us":19412,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":114,"thread_start_us":367204,"threads_started":1,"wal-append.queue_time_us":891364}
I20260709 13:09:12.637342 28598 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:12.620844 28518 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:12.616585 28680 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.639261 28495 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:12.612713 28568 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:09:12.642439 28696 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.782s	user 0.009s	sys 0.000s
I20260709 13:09:12.642838 28696 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.644789 28696 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:12.645879 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.647042 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.647570 22900 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 9.
I20260709 13:09:12.648345 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:12.648543 28395 consensus_meta.cc:287] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.816s	user 0.003s	sys 0.003s
I20260709 13:09:12.649595 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.650067 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.650098 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.649417 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.651479 22900 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:12.653755 28719 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:12.655900 28627 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.854s	user 0.003s	sys 0.006s
I20260709 13:09:12.656641 22828 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
W20260709 13:09:12.657943 28756 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:12.586376 28708 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:12.659873 28562 consensus_meta.cc:287] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.949s	user 0.006s	sys 0.003s
I20260709 13:09:12.607226 28473 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:12.660267 28562 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.661473 28665 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.661806 28744 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:12.661940 28665 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.662158 28611 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:12.618064 28576 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:12.576661 28521 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.663461 28713 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.664702 28521 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:12.666004 28765 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:09:12.666286 28713 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:12.666980 28748 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:09:12.667033 28713 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:12.593716 28745 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:12.667979 28745 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:12.667810 28713 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.669632 28563 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:12.674088 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48938.2us
I20260709 13:09:12.674963 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.678790 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:12.679152 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:12.679491 22754 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:12.679701 22755 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:09:12.648944 28395 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.700487 22755 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:12.707612 22904 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:12.681449 28518 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:12.693337 28282 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:12.710347 28282 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.710557 28518 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.711573 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:12.712350 28282 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:12.712677 23416 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:12.713493 28473 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:12.713889 28562 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:12.713953 28473 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.714615 28562 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:12.714406 28751 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.715251 28562 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:12.715965 28617 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.717695 28568 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:12.717967 28747 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:12.718678 28568 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.716029 28473 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:12.720000 28617 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:12.712822 28282 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.722028 28282 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.723788 28751 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:12.722483 22871 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:12.724601 28282 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:12.725292 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:12.731825 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:12.728636 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:12.729410 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2110) took 1458 ms. Trace:
I20260709 13:09:12.729852 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:12.730136 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:12.729705 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:12.735472 22754 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:12.735598 23447 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:12.733453 22824 rpcz_store.cc:276] 0709 13:09:11.270896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.271371 (+   475us) service_pool.cc:224] Handling call
0709 13:09:12.729209 (+1457838us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400091fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:12.729359 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2609384704}
I20260709 13:09:12.744614 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:12.751334 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:12.687494 28524 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:12.786150 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:12.786888 22899 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:12.787277 22899 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:12.779590 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 171570us
I20260709 13:09:12.673667 28528 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.784394 22754 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:12.789285 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:12.789726 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:12.791100 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:12.727063 28751 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:12.793236 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:12.805949 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.800081 23447 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:12.725494 28529 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.778952 28282 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:12.796674 22899 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.811491 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:12.812134 22900 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:09:12.783092 28495 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:12.808082 28751 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.816686 23445 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:12.820088 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:12.823834 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:12.813089 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 322994us
I20260709 13:09:12.826332 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:12.832942 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.846231 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.810065 28644 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:12.812991 28769 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:12.850327 28638 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:12.851197 28638 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:12.794405 28767 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:09:12.816694 28777 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:12.853745 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.824203 28751 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:12.794883 28768 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:09:12.843621 28535 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:12.813098 28529 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.845700 28526 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:12.861265 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.849423 28644 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:12.848044 28495 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:12.869844 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:12.858103 28544 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:12.854660 28282 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:12.848178 28771 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:12.856142 28708 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:12.866997 28167 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:12.855341 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:12.854578 28638 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:12.874238 22754 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:09:12.861486 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 326127us
I20260709 13:09:12.871791 28526 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:12.874962 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 146788us
I20260709 13:09:12.859612 28627 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:12.874073 28644 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:12.881132 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.913515 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.856964 28729 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.872099 22900 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:12.914740 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.915983 28535 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
W20260709 13:09:12.917729 22828 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2961) took 1865 ms (client timeout 1890 ms). Trace:
W20260709 13:09:12.918154 22828 rpcz_store.cc:269] 0709 13:09:11.052475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.052582 (+   107us) service_pool.cc:224] Handling call
0709 13:09:12.917569 (+1864987us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54001fa9d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:12.917674 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3687101184}
I20260709 13:09:12.918778 28638 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.919752 28638 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:12.914242 28627 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.921998 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:12.876958 28544 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:12.878854 28771 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:12.885409 28767 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:12.855822 28524 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.926929 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.929834 22900 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:12.863157 28777 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.879140 28713 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:12.956457 28777 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:12.910223 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 105914us
I20260709 13:09:12.888302 28769 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.916384 28773 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:12.931586 28719 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:12.933434 28495 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:12.959122 28757 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.514s	user 0.002s	sys 0.000s
I20260709 13:09:12.907625 28282 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:12.960767 28282 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:12.927003 28650 consensus_meta.cc:287] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.003s
I20260709 13:09:12.934291 28749 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:12.935211 28528 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:12.914202 28395 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:12.938521 28778 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:09:12.939270 28603 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.877945 28717 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:12.886061 28708 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:12.929366 28767 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:12.929215 28771 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:12.941720 28760 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:09:12.943796 28529 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.013914 22902 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:12.944254 28774 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:12.951355 22654 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 342) took 6233 ms. Trace:
I20260709 13:09:12.997126 28598 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.954365 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.912436 28526 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:12.888018 28700 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:12.963752 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:12.914749 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:12.964295 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:12.968631 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:12.969295 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.018034 22754 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:12.916724 28779 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.973461 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.022104 22830 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:12.972787 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.023123 22826 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:13.001683 28611 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:13.014858 22654 rpcz_store.cc:276] 0709 13:09:06.718407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:06.718594 (+   187us) service_pool.cc:224] Handling call
0709 13:09:12.951264 (+6232670us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:10.870063 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:10.947351 (+ 77288us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:10.950795 (+  3444us) write_op.cc:435] Acquired schema lock
0709 13:09:10.953809 (+  3014us) tablet.cc:662] Decoding operations
0709 13:09:10.984806 (+ 30997us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:10.984863 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:10.984897 (+    34us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:09:10.999095 (+ 14198us) tablet.cc:701] Row locks acquired
0709 13:09:10.999118 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:09:10.999272 (+   154us) write_op.cc:270] Start()
0709 13:09:10.999408 (+   136us) write_op.cc:276] Timestamp: P: 1783602550999221 usec, L: 0
0709 13:09:10.999432 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:09:11.000421 (+   989us) log.cc:844] Serialized 8226 byte log entry
0709 13:09:11.175635 (+175214us) op_driver.cc:464] REPLICATION: finished
0709 13:09:11.185502 (+  9867us) write_op.cc:301] APPLY: starting
0709 13:09:11.185583 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:11.250408 (+ 64825us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:11.250512 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:11.664076 (+413564us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:11.664447 (+   371us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:09:11.664494 (+    47us) write_op.cc:312] APPLY: finished
0709 13:09:11.664940 (+   446us) log.cc:844] Serialized 188 byte log entry
0709 13:09:11.665853 (+   913us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:11.666065 (+   212us) write_op.cc:454] Released schema lock
0709 13:09:11.666126 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:11.714692 (+ 48566us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:11.726598 (+ 11906us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":119003,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":9642,"apply.run_cpu_time_us":29302,"apply.run_wall_time_us":502963,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91907,"num_ops":21,"prepare.queue_time_us":533536,"prepare.run_cpu_time_us":15974,"prepare.run_wall_time_us":186430,"raft.queue_time_us":76153,"raft.run_cpu_time_us":2062,"raft.run_wall_time_us":9875,"replication_time_us":176036,"thread_start_us":479885,"threads_started":4,"wal-append.queue_time_us":96141,"wal-append.run_cpu_time_us":52085,"wal-append.run_wall_time_us":1569776}]]}
I20260709 13:09:12.973886 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:12.998607 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:12.986541 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 193348us
I20260709 13:09:13.029515 28750 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:13.030289 28717 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:13.030715 28770 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:09:13.030874 28717 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:13.031281 28717 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.031548 28775 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.032176 28775 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.033541 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.034266 22901 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:13.034219 28775 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.035040 28758 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:13.035254 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.035589 28758 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.037225 28749 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:13.037925 28708 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:13.038022 28767 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.039104 28767 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:13.039510 28747 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:13.039913 28747 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:13.039978 22794 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:13.043635 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:13.044109 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.044894 22753 raft_consensus.cc:2393] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:13.045459 28779 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.047619 28779 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.048123 28779 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:13.048568 28779 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:13.049983 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:13.051110 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.052129 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.052704 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.053745 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.053526 28745 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.054394 22749 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:13.055024 22825 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:13.055092 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.055490 28745 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.056020 28576 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:13.056384 28576 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.056439 22753 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:13.056783 28576 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:13.053102 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.058787 22827 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:13.002658 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.059613 22822 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:13.005010 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.060484 22755 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:13.006405 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:13.004202 28535 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.930617 28544 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:12.996881 28650 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.016026 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1750) took 1444 ms. Trace:
I20260709 13:09:13.086024 22748 rpcz_store.cc:276] 0709 13:09:11.571683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.898213 (+326530us) service_pool.cc:224] Handling call
0709 13:09:11.898472 (+   259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:11.902365 (+  3893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.902393 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:11.902799 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 13:09:12.172562 (+269763us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:12.172600 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.172665 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.958647 (+785982us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:13.008450 (+ 49803us) raft_consensus.cc:1685] finished
0709 13:09:13.008469 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.015965 (+  7496us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3574,"raft.queue_time_us":4870,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3603,"spinlock_wait_cycles":3958528,"thread_start_us":269495,"threads_started":1,"wal-append.queue_time_us":497663}
I20260709 13:09:12.997730 28537 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:12.952912 28677 consensus_meta.cc:287] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.666s	user 0.004s	sys 0.004s
I20260709 13:09:13.021112 28774 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:13.017774 22900 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 13: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:13.030014 28750 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.031631 28717 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.001178 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:12.927958 28627 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:13.057001 28751 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.065501 22754 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:13.126305 22747 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:13.126359 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.009531 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.091454 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.128890 22748 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:13.128770 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.091765 22797 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:13.129930 22900 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:13.026443 28231 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:13.107522 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.131371 28231 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:13.017795 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 103147us
I20260709 13:09:13.131870 28231 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:13.132449 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:13.133404 28680 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:13.117642 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.133765 28680 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.134156 28680 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:13.119216 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.137506 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:13.121273 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.122501 23444 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:09:13.139801 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 140542us
I20260709 13:09:13.140476 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.142156 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.142807 23444 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:13.133765 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:13.144423 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3294) took 1536 ms. Trace:
I20260709 13:09:13.144883 22756 rpcz_store.cc:276] 0709 13:09:11.608131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.923389 (+315258us) service_pool.cc:224] Handling call
0709 13:09:11.923682 (+   293us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:11.924021 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.924049 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:11.987820 (+ 63771us) log.cc:844] Serialized 52 byte log entry
0709 13:09:12.534192 (+546372us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:12.534230 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.534297 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.144149 (+609852us) raft_consensus.cc:1685] finished
0709 13:09:13.144179 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.144353 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":259,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":49876,"thread_start_us":467341,"threads_started":1,"wal-append.queue_time_us":660380}
I20260709 13:09:13.093415 28708 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:13.150992 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 59281.2us
I20260709 13:09:13.123809 22822 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:13.118257 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.000133 28769 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.026932 28598 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.152619 22900 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:09:13.148773 28526 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.062619 28611 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.117908 28774 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:13.159539 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:13.165840 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.040850 28747 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.173754 22824 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:13.153167 28562 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:13.156422 28524 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:13.083628 28535 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.063874 28779 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.156946 28147 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:13.117522 28537 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.180409 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:13.157548 28758 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.196048 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:13.166894 28730 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:13.124845 28677 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.199950 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.176414 28651 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:13.210117 22828 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.193742 28747 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:13.203318 28624 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:13.161875 28708 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.205071 28745 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:13.227380 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.229144 28650 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.181088 28731 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:09:13.232002 23453 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:13.204629 28730 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.202822 28534 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:13.212520 28763 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
I20260709 13:09:13.248970 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.255410 28526 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:13.256920 28794 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:09:13.257978 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.259155 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.260115 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.260552 28751 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:13.265981 28729 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.246639 28628 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 13:09:13.267460 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 1062) took 1090 ms. Trace:
I20260709 13:09:13.267922 23445 rpcz_store.cc:276] 0709 13:09:12.177107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.177246 (+   139us) service_pool.cc:224] Handling call
0709 13:09:13.267265 (+1090019us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54005550d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:13.267394 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195857792}
I20260709 13:09:13.269502 28535 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.270668 28771 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.250768 28730 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:13.274605 28537 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.279510 28624 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.280073 28624 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:13.286371 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.287019 28708 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.287379 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.287914 22827 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.289110 28677 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.287619 28680 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.291200 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.298475 22750 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:13.291808 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.292958 22754 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:13.299772 22754 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:13.293717 22902 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:09:13.294034 28771 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.295867 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.303906 22826 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:13.304991 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.306111 22828 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:13.309367 22902 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:09:13.310083 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.310122 22830 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:13.302861 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:13.311124 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.311846 22905 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
W20260709 13:09:13.313396 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.313351 22827 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:13.315092 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:13.316685 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 150940us
I20260709 13:09:13.300472 22754 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.323375 22748 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:13.344338 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.348410 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.349218 22751 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:13.394516 22754 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:13.395835 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:13.395792 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 51531.8us
I20260709 13:09:13.397053 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.324251 28534 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:13.322496 28651 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.309886 28598 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:13.434088 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 255238us
I20260709 13:09:13.435168 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.462911 28774 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.463728 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.464340 22824 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:13.464697 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.465030 28774 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.321956 28730 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:13.448947 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.308869 28746 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
I20260709 13:09:13.397806 28650 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:13.398209 28473 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.457156 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.465256 22825 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:13.465648 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.436739 28562 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.436079 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.456524 28534 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.449398 28700 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:13.438043 28808 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:13.473933 22748 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:13.474472 22824 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:13.466316 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.489596 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.436265 28627 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.468623 28651 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.527737 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.492039 22754 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:09:13.528524 23444 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:13.528026 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.533090 23452 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 15: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:09:13.515636 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 3583f8fda02d46af9ef0ff2ae6f1b443: Illegal state: Replica 3b626c41600d4737a178ebab8a7ebfc2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:13.533252 22754 pending_rounds.cc:85] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 12
I20260709 13:09:13.537590 22754 pending_rounds.cc:107] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:09:13.537818 22754 raft_consensus.cc:2889] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:13.536397 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.541738 22825 raft_consensus.cc:2368] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
W20260709 13:09:13.542793 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 76556.7us
I20260709 13:09:13.543481 22719 leader_election.cc:400] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
I20260709 13:09:13.543730 22719 leader_election.cc:403] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:13.545789 22748 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:13.547250 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.548246 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.548969 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:13.490378 28562 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.475009 28810 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:13.521533 28799 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:09:13.471240 28231 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:13.521772 28797 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:09:13.494055 28769 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:13.525833 28644 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.003s
I20260709 13:09:13.552261 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:13.549825 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.552699 23445 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:09:13.473645 28650 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:13.491753 28534 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:13.475751 28611 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.553747 22753 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:09:13.476392 28537 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:13.555214 23450 raft_consensus.cc:2410] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:09:13.555601 22829 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: [14.15-14.15]
I20260709 13:09:13.556982 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.527952 28771 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.560413 28812 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:13.586158 28627 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.595687 28231 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.584548 28650 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.594229 28651 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.596405 22749 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:13.596372 28650 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:13.567821 28805 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:09:13.569026 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 1064) took 1354 ms. Trace:
I20260709 13:09:13.597857 23453 rpcz_store.cc:276] 0709 13:09:12.214784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.217644 (+  2860us) service_pool.cc:224] Handling call
0709 13:09:13.568823 (+1351179us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400554454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:13.568969 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2151517568}
I20260709 13:09:13.549751 28731 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
I20260709 13:09:13.492635 28473 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:13.600689 28754 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.130s	user 0.000s	sys 0.002s
I20260709 13:09:13.600824 28473 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.570446 28624 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.602792 28624 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.571409 28562 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.550164 28526 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:09:13.603313 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.603960 28526 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:13.604343 28526 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:13.604863 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.605011 28717 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.605135 28526 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.580084 28167 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:13.606417 28526 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:13.607684 28717 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:13.607977 22899 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:13.582248 28544 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:13.515712 28753 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.983s	user 0.000s	sys 0.002s
I20260709 13:09:13.609869 22898 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:13.610770 28544 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.613677 22822 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:13.613996 23451 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:13.555833 28708 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:13.561832 28680 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.514634 28730 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.580737 22824 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:13.549959 28758 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:09:13.619539 22754 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:13.582569 23445 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:09:13.590548 28576 consensus_meta.cc:287] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.000s
I20260709 13:09:13.614528 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:13.587036 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.575258 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.615576 22898 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:13.551543 28700 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.624330 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:13.595077 28798 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:09:13.633787 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.633970 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.582932 28814 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:13.642294 23452 raft_consensus.cc:2368] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14.
I20260709 13:09:13.629202 28576 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.619578 28231 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:13.655069 28231 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:13.655460 28231 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:13.656169 28576 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.656709 28708 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:13.630795 28700 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.660027 22824 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:13.660591 28700 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:13.661271 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.642257 23453 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:13.625284 28562 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:09:13.628217 28808 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:09:13.620898 28745 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:13.664841 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:13.666651 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:13.667335 28562 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:13.667744 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2006) took 1019 ms. Trace:
I20260709 13:09:13.668166 22831 raft_consensus.cc:2368] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15.
I20260709 13:09:13.620397 28696 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:09:13.668126 22903 rpcz_store.cc:276] 0709 13:09:12.648011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.650345 (+  2334us) service_pool.cc:224] Handling call
0709 13:09:12.650607 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:12.691045 (+ 40438us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:09:12.691073 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:12.709620 (+ 18547us) log.cc:844] Serialized 26 byte log entry
0709 13:09:12.961406 (+251786us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b18002aca00. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:09:13.154046 (+192640us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:13.154081 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.154180 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.667360 (+513180us) raft_consensus.cc:1685] finished
0709 13:09:13.667388 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.667677 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32629,"raft.queue_time_us":40316,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":14366,"spinlock_wait_cycles":46987136,"thread_start_us":192497,"threads_started":1,"wal-append.queue_time_us":544284}
I20260709 13:09:13.669728 28617 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:13.671015 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.646354 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.672834 22827 raft_consensus.cc:2368] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14.
I20260709 13:09:13.621241 28771 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.676842 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.677445 22755 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:13.680907 22868 leader_election.cc:400] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14.
I20260709 13:09:13.681771 22868 leader_election.cc:403] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:09:13.582226 28644 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.682730 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.684098 22903 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:13.680485 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:13.683625 22753 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:13.688807 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.690414 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.691012 28771 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.691504 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.686506 28808 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14.
I20260709 13:09:13.692178 22755 raft_consensus.cc:2368] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20.
I20260709 13:09:13.691547 28816 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:13.694736 22752 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:09:13.695631 28644 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.696007 22752 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:13.693014 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.698414 28603 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.574512 28810 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.700722 28810 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.700927 28747 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.701550 22756 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:09:13.701467 28747 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.699366 22755 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:09:13.702601 28665 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.683354 28713 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:13.702966 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:13.703374 28747 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.705657 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:13.706231 22897 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:13.689255 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.697136 22752 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:13.685894 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:13.604360 28534 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.713290 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:13.713917 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.717223 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.727334 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.726286 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.717446 28759 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.734849 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.734776 28759 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.735229 22898 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:13.735548 23450 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:09:13.735846 22898 raft_consensus.cc:606] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:13.713469 28700 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:13.736505 28700 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.736845 28700 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.736936 28759 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:13.722347 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.725574 22871 leader_election.cc:400] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20.
I20260709 13:09:13.764768 22871 leader_election.cc:403] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:09:13.726691 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.721938 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:13.726984 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.716583 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.765800 23445 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:13.765985 23448 raft_consensus.cc:2410] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:09:13.737767 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:13.766002 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44127.6us
I20260709 13:09:13.737844 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.766993 23446 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:13.767107 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:13.738857 22905 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:09:13.738801 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:13.739535 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:13.768105 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.740304 22896 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:13.721057 28696 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:13.686771 28746 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:09:13.730535 22899 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:13.775452 22897 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:13.775596 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.741792 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.776192 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.776458 22900 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:13.776924 23447 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:13.777413 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.742348 22828 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:13.777995 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.743183 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:13.744693 23450 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:13.744612 28535 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:13.685007 28562 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:13.726001 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:13.720495 28791 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:09:13.722077 28747 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:09:13.727790 28231 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.752636 28700 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.754352 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.822508 22829 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:13.755683 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.756115 28713 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:13.757085 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.828691 23445 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:13.771631 23416 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:13.774544 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 49100.8us
I20260709 13:09:13.779842 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:13.833686 22896 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:13.832960 22830 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:13.780472 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.781154 22897 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
W20260709 13:09:13.782330 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 56390.8us
I20260709 13:09:13.783309 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.834857 22897 raft_consensus.cc:606] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.835096 22828 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:13.783896 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.835951 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.783826 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.788236 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3296) took 1317 ms. Trace:
I20260709 13:09:13.836537 22747 rpcz_store.cc:276] 0709 13:09:12.470979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.471094 (+   115us) service_pool.cc:224] Handling call
0709 13:09:12.471364 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.126811 (+655447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.788035 (+661224us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54002684d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:13.788184 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505835776}
I20260709 13:09:13.786505 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.775398 28677 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:13.806783 28719 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:13.839110 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.782827 28777 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:09:13.807569 22826 raft_consensus.cc:2368] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16.
I20260709 13:09:13.846254 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.846449 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.812517 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.814663 23447 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:13.817209 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:13.824131 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.774012 28696 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.770833 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:13.776201 22899 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:13.835214 22897 raft_consensus.cc:614] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:13.783140 28792 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:13.855185 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.790287 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.843564 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.858174 22828 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:13.781685 28638 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 13:09:13.844815 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73273.2us
I20260709 13:09:13.838254 28603 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:13.788048 28816 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.858292 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:13.867439 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1390) took 1307 ms. Trace:
I20260709 13:09:13.821513 22822 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:13.752861 28524 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:09:13.861227 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.871951 22896 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:13.865932 22829 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:13.825426 28282 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:13.807129 28654 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:09:13.842264 28719 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:13.871268 22828 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:13.867818 22830 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:13.881749 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:13.867832 22904 rpcz_store.cc:276] 0709 13:09:12.551502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.551640 (+   138us) service_pool.cc:224] Handling call
0709 13:09:13.857917 (+1306277us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400002dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:13.858067 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304077056}
I20260709 13:09:13.856878 28747 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:13.857187 28763 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:13.886346 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.859064 28696 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:13.853394 28677 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:13.860463 28745 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:13.866864 28611 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:13.867100 28774 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:13.806529 28562 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.894096 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:13.867077 28792 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:13.867317 28603 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:13.895305 22828 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:13.880686 28282 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:13.915091 22825 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:09:13.902042 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 352725us
I20260709 13:09:13.883424 28529 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:13.898202 28779 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:13.896664 22830 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:13.940006 22830 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
W20260709 13:09:13.898999 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.885062 28713 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.887527 28719 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.868058 28651 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:09:13.906697 23416 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:13.879760 28717 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:09:13.893707 28771 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:13.917128 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:13.917579 28677 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.895251 28535 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:09:13.918692 28231 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:13.918781 28603 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.920238 28282 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.901772 28544 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:09:13.897434 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.900493 28747 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:13.883838 28816 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:13.886754 28563 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:09:13.944432 28713 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:13.912451 28767 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:13.941285 22830 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:13.938827 28167 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:13.956024 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49395.3us
W20260709 13:09:13.967484 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50437.5us
I20260709 13:09:13.934396 28534 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:13.978864 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:13.979707 22830 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:13.956885 28806 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:13.970417 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 73050us
I20260709 13:09:13.952633 28771 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.982496 22831 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:09:13.991489 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.969411 28823 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:13.985920 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:13.986616 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.988078 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.967957 28562 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:13.989539 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:13.955713 28700 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:13.995919 22719 leader_election.cc:400] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
I20260709 13:09:14.002144 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.002485 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:14.005259 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:13.971200 28779 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:14.006942 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:14.007611 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.010886 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:14.010993 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:14.011864 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:09:14.008108 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.017565 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:13.975930 28282 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:14.021620 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:14.022965 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:14.023952 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.026248 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:13.983517 28603 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:14.000841 28771 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:14.025280 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.034303 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:14.028199 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.036015 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:14.037456 22719 leader_election.cc:400] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14.
I20260709 13:09:14.037978 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:14.038594 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:14.038969 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.002781 28767 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:14.039266 22752 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:14.021871 28534 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:14.022969 28745 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:14.030797 28231 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:14.026338 28822 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:14.009754 28769 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:14.026180 28823 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:14.030248 28777 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:14.048062 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:14.021142 28562 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:14.026082 28611 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:14.025735 28814 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.025914 28791 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.028780 28774 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:14.048768 28819 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:14.049692 28830 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:14.000929 28806 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.043411 22899 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:14.076282 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:14.077351 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:14.078181 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:14.078817 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:14.079210 22899 raft_consensus.cc:2368] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15.
I20260709 13:09:14.033967 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.030012 28650 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:13.992709 28167 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.059934 22830 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:14.044900 28810 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:13.974768 28792 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.042078 28529 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:14.068327 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:14.068357 22752 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:14.059284 28812 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:14.082209 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:14.083577 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 49673.4us
I20260709 13:09:14.126601 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.128612 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:14.075158 28819 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:14.074045 28769 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:14.093719 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:14.131637 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:14.132769 22830 raft_consensus.cc:2368] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15.
W20260709 13:09:14.133056 22719 leader_election.cc:336] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.536s (SENT)
I20260709 13:09:14.133627 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:14.135742 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:14.048873 28747 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:14.121500 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.074061 28814 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.083175 28774 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:14.139369 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:14.214650 22830 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:14.124017 22902 raft_consensus.cc:2393] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:14.150815 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:14.084923 28779 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:14.074110 28791 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.217367 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:14.220822 22902 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:09:14.219636 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69229.5us
I20260709 13:09:14.138266 28819 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:14.172551 28759 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:14.202863 28611 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:14.212491 28231 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:14.246469 22830 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:14.222752 28518 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:14.251845 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:14.252590 22896 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:14.129294 28696 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.256608 22830 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:09:14.257339 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 2129) took 1167 ms. Trace:
I20260709 13:09:14.142508 28529 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 13:09:14.123976 28767 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.258589 22830 rpcz_store.cc:276] 0709 13:09:13.089533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.255943 (+1166410us) service_pool.cc:224] Handling call
0709 13:09:14.256319 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:14.257065 (+   746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.257268 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:14.274008 23453 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:14.275379 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:14.245527 28713 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:14.278369 22830 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:14.282496 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2985) took 1103 ms. Trace:
I20260709 13:09:14.282969 22830 rpcz_store.cc:276] 0709 13:09:13.179141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.281621 (+1102480us) service_pool.cc:224] Handling call
0709 13:09:14.281938 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.282132 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:14.282168 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:14.282219 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:14.282269 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.282333 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.282441 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:14.228786 28719 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.294523 28791 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:14.255494 28231 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.297207 28231 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:14.298424 28806 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:14.258100 28167 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:14.307060 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.315332 28819 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.316294 28231 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:14.251792 28518 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:09:14.323284 22753 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:14.227031 28562 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.233290 28823 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:14.338737 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 260965us
I20260709 13:09:14.339387 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:14.341336 23453 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:14.344751 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:14.345597 22752 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:14.351140 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:14.351904 22752 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:14.355767 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:14.356364 22897 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:14.354705 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:14.357486 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:14.358076 22897 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:14.363884 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.265735 28769 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.341425 28680 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:14.367139 22902 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:14.351816 28624 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:14.346611 28816 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:14.368817 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:14.369273 22897 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:14.375284 28773 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:14.375933 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:14.379757 28771 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.390341 22719 leader_election.cc:400] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15.
I20260709 13:09:14.394358 23453 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:14.393689 22719 leader_election.cc:403] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:09:14.371857 28231 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:14.328318 28774 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.393582 28713 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:14.421595 28791 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:14.421656 28713 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.422437 28719 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:14.393676 28759 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:14.393435 28841 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.399195 28806 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:14.429960 28806 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.407619 28729 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:14.430361 28806 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:14.430624 28729 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:09:14.431003 28763 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.002s
I20260709 13:09:14.431131 28759 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.431510 28763 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20.
I20260709 13:09:14.431617 28729 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.432659 28729 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:14.393522 28812 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:14.401826 28650 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15.
I20260709 13:09:14.413681 28819 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:14.420622 28696 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:14.422148 28791 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:14.438727 22899 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:14.374536 28624 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:14.433185 28759 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:14.434767 22830 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:14.416702 28651 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:14.428354 22831 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:14.404120 28773 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:14.450977 28651 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:14.450858 28773 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.451401 28651 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:14.441083 28791 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.452244 28651 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.452987 28814 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:14.430361 28841 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.452621 28713 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:14.459702 28810 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:09:14.473716 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 250263us
I20260709 13:09:14.460610 28651 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:14.465806 28844 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:09:14.439862 28529 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:09:14.493016 22898 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:14.507565 22755 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:14.493635 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:14.494333 28624 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:14.502045 28786 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.476s	user 0.003s	sys 0.000s
I20260709 13:09:14.484447 28848 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:14.505072 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:14.506672 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:14.511726 28847 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:14.512405 28562 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:14.517446 28231 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:14.519838 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1172) took 1512 ms. Trace:
I20260709 13:09:14.521032 22823 rpcz_store.cc:276] 0709 13:09:13.007841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.008234 (+   393us) service_pool.cc:224] Handling call
0709 13:09:13.008495 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.037528 (+ 29033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:13.037558 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:13.037597 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:13.037622 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.519616 (+1481994us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b540008b454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:14.519676 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.519790 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66362624}
I20260709 13:09:14.520772 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:14.523375 22747 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:14.522910 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:14.530303 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:14.534677 28841 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:14.520928 28231 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:14.538044 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 1177) took 1516 ms. Trace:
I20260709 13:09:14.538435 22827 rpcz_store.cc:276] 0709 13:09:13.021793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.604622 (+582829us) service_pool.cc:224] Handling call
0709 13:09:14.537883 (+933261us) spinlock_profiling.cc:243] Waited 67.2 ms on lock 0x7b5400081e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:14.537996 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154558208}
I20260709 13:09:14.539461 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:14.539609 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:14.540884 22827 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:14.541951 23450 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
W20260709 13:09:14.542526 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.543205 22827 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:09:14.544484 22827 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:09:14.545615 22827 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:14.519328 28847 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.546628 22827 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:14.546941 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:14.539641 28793 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.502s	user 0.004s	sys 0.002s
I20260709 13:09:14.532080 28624 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:14.549302 23416 leader_election.cc:400] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14.
I20260709 13:09:14.538077 28231 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:14.552233 23416 leader_election.cc:403] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:14.555310 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:14.557552 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.529990 28836 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 13:09:14.564224 28832 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.534s	user 0.000s	sys 0.001s
I20260709 13:09:14.541864 28563 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:09:14.557488 28792 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:14.565408 28849 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:09:14.555794 28848 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 13. Current term is 14.
I20260709 13:09:14.570333 28700 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:14.570756 28700 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:14.571530 28700 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.572582 28700 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:14.567884 28830 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:09:14.577653 28771 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:14.578608 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:14.581333 28847 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:14.583032 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:14.610105 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.614132 28806 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.616880 28806 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:14.617712 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:14.618449 22755 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:14.627718 23444 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:14.602202 28473 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 13:09:14.645814 28859 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:09:14.646517 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2976) took 1631 ms. Trace:
I20260709 13:09:14.646899 22828 rpcz_store.cc:276] 0709 13:09:13.015021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.885928 (+870907us) service_pool.cc:224] Handling call
0709 13:09:13.895150 (+  9222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.912618 (+ 17468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.646349 (+733731us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b5400230ad0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:14.646465 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20622592}
I20260709 13:09:14.648430 22828 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:14.651609 22828 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:14.652848 22755 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:14.655733 22828 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:09:14.661494 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.661564 28767 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:09:14.678648 28767 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:14.679092 28767 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:14.680336 28767 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.681893 28767 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:14.684371 22826 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2973) took 1674 ms (client timeout 2096 ms). Trace:
W20260709 13:09:14.685185 28282 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.658s	user 0.005s	sys 0.004s
I20260709 13:09:14.686594 28576 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:14.687141 28576 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:14.686937 28747 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:14.688293 28576 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:14.684923 22826 rpcz_store.cc:269] 0709 13:09:13.009566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.603082 (+593516us) service_pool.cc:224] Handling call
0709 13:09:14.684144 (+1081062us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b540003fcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:14.684303 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467701504}
I20260709 13:09:14.691674 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:14.698486 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:14.698387 28700 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:09:14.709384 28857 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:09:14.702600 28806 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:14.712989 22755 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:09:14.766971 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.702519 28624 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:14.705654 28576 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:14.885784 28806 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:14.891831 28806 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.888820 28836 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.893803 28836 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.895823 28836 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:14.718321 28729 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:14.897853 28729 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.899835 28729 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:14.900712 28231 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.902911 28231 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:14.888935 28830 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.906085 28830 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:14.910310 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:14.711129 28700 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.911407 22750 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:14.914181 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:14.916523 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2987) took 1412 ms. Trace:
I20260709 13:09:14.807339 28810 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:14.916904 22823 rpcz_store.cc:276] 0709 13:09:13.504294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.540348 (+1036054us) service_pool.cc:224] Handling call
0709 13:09:14.540606 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.540785 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:14.541032 (+   247us) log.cc:844] Serialized 14 byte log entry
0709 13:09:14.916247 (+375215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:14.916288 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:14.916331 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.916384 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.916474 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":807,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":30,"raft.queue_time_us":69,"thread_start_us":374861,"threads_started":1}
W20260709 13:09:14.927103 22822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2982) took 1908 ms (client timeout 2187 ms). Trace:
I20260709 13:09:14.927129 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:14.778146 28708 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.139s
I20260709 13:09:14.914875 28700 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:14.936470 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.888159 28624 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:14.927668 22822 rpcz_store.cc:269] 0709 13:09:13.018851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.625929 (+607078us) service_pool.cc:224] Handling call
0709 13:09:14.926973 (+1301044us) spinlock_profiling.cc:243] Waited 58.6 ms on lock 0x7b54001edcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:14.927054 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134777216}
I20260709 13:09:14.918612 28836 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:09:14.920073 28867 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 13:09:14.946646 28870 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:09:14.939580 28729 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.920943 28810 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.936880 28231 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:14.956398 28231 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:14.943120 22650 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 10 to 11, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 11 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.944368 28836 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.957665 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:14.960218 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:14.961539 22749 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:09:14.962924 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.963100 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:14.964993 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:14.966140 22749 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:14.970218 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:14.970985 22754 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:14.960196 28836 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:14.965628 28823 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:14.962718 28814 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:14.965811 28599 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:14.957346 28729 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:14.984902 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.984817 22754 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:14.986598 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:14.976513 28779 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:09:15.004217 28830 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:14.991798 28847 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:14.981676 28526 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:09:15.020129 28810 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:14.987777 28599 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:14.990155 28231 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.021005 28847 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:15.038290 28599 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.039119 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.043555 22747 raft_consensus.cc:2368] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:15.047791 22756 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:09:15.049837 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 89674.7us
I20260709 13:09:15.051898 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:15.032711 28624 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.055135 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:15.055876 22902 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
W20260709 13:09:15.056005 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.056646 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:15.056609 22797 leader_election.cc:400] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:15.058082 22797 leader_election.cc:403] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:15.059792 22902 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:15.052868 28823 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:15.060248 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:15.061848 22719 leader_election.cc:400] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15.
I20260709 13:09:15.060879 22747 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 9. Current term is 10. Ops: []
W20260709 13:09:15.066105 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.069272 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:15.071460 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:15.061769 28599 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:09:15.067934 28624 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:15.071663 28768 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:15.061276 28651 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:09:15.082868 28877 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:09:15.073701 28810 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:15.074857 28823 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:15.091022 28810 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:15.091475 28810 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:15.097000 28599 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:15.104760 28653 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:15.071920 28526 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:09:15.109392 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.115115 28830 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.114959 28653 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:15.116858 22900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:15.084612 28768 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.119907 22747 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 15: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:15.120330 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.121901 22900 raft_consensus.cc:606] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:09:15.123172 22900 raft_consensus.cc:614] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:15.115619 28823 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:15.106177 28680 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:15.115236 28878 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:15.081221 28847 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.129576 28768 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:15.155597 28791 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:15.162098 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1173) took 2151 ms. Trace:
I20260709 13:09:15.162590 22825 rpcz_store.cc:276] 0709 13:09:13.010541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.883321 (+872780us) service_pool.cc:224] Handling call
0709 13:09:13.883580 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.915702 (+ 32122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.161794 (+1246092us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b54001f9fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:15.162025 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71538560}
I20260709 13:09:15.171303 22900 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:15.173652 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:15.174402 22896 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:15.174571 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:15.173828 22900 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:15.178670 28526 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.179348 28526 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.182384 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:15.185520 28599 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:15.190327 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 1075) took 1204 ms. Trace:
I20260709 13:09:15.190774 23444 rpcz_store.cc:276] 0709 13:09:13.986183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.987202 (+  1019us) service_pool.cc:224] Handling call
0709 13:09:13.993583 (+  6381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.628158 (+634575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.190025 (+561867us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b5400730354. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:15.190266 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1454062848}
I20260709 13:09:15.193162 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1174) took 2180 ms. Trace:
I20260709 13:09:15.193666 22829 rpcz_store.cc:276] 0709 13:09:13.013146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.882921 (+869775us) service_pool.cc:224] Handling call
0709 13:09:13.883196 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.912878 (+ 29682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:13.912927 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:13.912984 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:13.913039 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.192965 (+1279926us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b54002c9054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:15.193006 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.193115 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67626240}
I20260709 13:09:15.195454 28747 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:09:15.199792 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.200433 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:15.215518 28877 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:15.201287 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.216744 22756 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:15.218384 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 1076) took 1230 ms. Trace:
I20260709 13:09:15.202924 28713 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:15.218740 23449 rpcz_store.cc:276] 0709 13:09:13.987633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.987799 (+   166us) service_pool.cc:224] Handling call
0709 13:09:15.218221 (+1230422us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b54005550d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:15.218334 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1471517952}
I20260709 13:09:15.205279 28526 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.206736 28729 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:15.210768 28833 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.161s	user 0.005s	sys 0.006s
I20260709 13:09:15.212810 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:15.224378 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2986) took 1720 ms. Trace:
I20260709 13:09:15.224812 28524 raft_consensus.cc:2764] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:09:15.227327 28524 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:15.227797 28561 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:15.228319 28561 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:15.228595 28729 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:15.224732 22830 rpcz_store.cc:276] 0709 13:09:13.503941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.284854 (+780913us) service_pool.cc:224] Handling call
0709 13:09:14.285164 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:14.443376 (+158212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.224178 (+780802us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400431f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:15.224325 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343649024}
I20260709 13:09:15.229802 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:15.130012 28830 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.241819 22899 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:15.242506 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:15.244726 22899 raft_consensus.cc:606] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:15.247830 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:15.255312 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:15.255605 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.256207 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.256773 22829 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:15.257808 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:15.147727 28750 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.246295 28524 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:15.261473 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.256207 22823 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:15.262053 23444 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:15.254060 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.262629 22830 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:15.248540 28603 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:15.263582 28749 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:09:15.267503 28823 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.267733 28651 raft_consensus.cc:2955] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:09:15.268942 28808 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:15.269526 28808 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.271530 28808 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:15.143872 28680 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.152109 28700 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.229136 28526 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:15.274741 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:15.301497 22903 raft_consensus.cc:606] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:15.301947 22903 raft_consensus.cc:614] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:15.280622 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:15.281899 22824 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:09:15.282662 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.284121 22831 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:15.306279 22828 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:15.306659 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:15.145282 28878 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:15.308902 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.310400 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:15.259384 28653 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.312526 28653 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:15.313606 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.314513 22756 raft_consensus.cc:2393] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 10: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:15.229090 28561 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 14.15, Last appended by leader: 15, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:15.315763 28879 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:09:15.316028 28561 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:15.260386 28599 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 13:09:15.322100 28599 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:15.322479 28599 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:15.323294 28599 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.324532 28599 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:15.263507 28779 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:15.333627 28779 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:15.334067 28779 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:15.263479 28627 raft_consensus.cc:3055] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:15.330572 28700 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:15.298796 28526 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:15.293136 28624 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.330747 28231 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:15.326540 28814 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:15.229094 28729 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.338402 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:15.345870 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.347288 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:15.338660 28767 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.656s	user 0.008s	sys 0.004s
I20260709 13:09:15.357885 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:15.360764 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:15.343356 28880 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:15.336740 28750 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.365515 22827 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:15.367085 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:15.342706 28627 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:15.391086 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.395918 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.397841 22752 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:15.399330 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.400715 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:15.403759 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.406422 22752 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:15.371870 28746 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.371443 28806 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:15.373107 28767 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:15.418239 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 244544us
I20260709 13:09:15.369736 28814 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:15.421412 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.420957 22752 raft_consensus.cc:2955] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:09:15.358337 28624 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.434485 22899 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:15.363956 28231 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:15.446034 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:15.456496 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 205340us
I20260709 13:09:15.463006 22824 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
W20260709 13:09:15.465117 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.376238 28880 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.378180 28750 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.483083 22755 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:09:15.485121 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:15.485905 22896 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:15.486222 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.486828 23445 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:15.464309 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.487869 22824 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:15.488535 23445 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:15.488864 22824 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: [14.15-14.15]
W20260709 13:09:15.494115 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 55907.7us
I20260709 13:09:15.494280 22826 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:15.505587 22822 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:15.508059 22826 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:15.520948 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:15.532369 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:15.427616 28746 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.437296 28767 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.437417 28814 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.538053 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:15.538895 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45917.9us
W20260709 13:09:15.550988 28883 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:09:15.551098 28888 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:15.551867 28888 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.554335 28888 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.555713 28807 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:15.548521 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:15.555887 28875 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:09:15.447916 28627 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.438930 28526 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.542068 28806 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:09:15.597183 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48744.2us
I20260709 13:09:15.565865 22900 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:15.530982 28808 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:15.536731 28700 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:15.570389 28534 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.570699 28231 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.571450 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1539) took 1686 ms. Trace:
W20260709 13:09:15.571897 28885 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:09:15.573158 28880 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.556257 28895 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:15.563174 28680 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.576444 28544 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:15.557013 28562 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:15.581616 28708 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:15.563838 28839 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:15.584856 22749 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:15.634914 22749 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:15.585095 28768 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.540644 28830 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.593595 28526 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:15.555748 28891 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:09:15.595824 28836 consensus_meta.cc:287] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.616s	user 0.008s	sys 0.000s
I20260709 13:09:15.598459 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:15.606442 28876 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:09:15.618594 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.622391 23447 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:15.632905 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:15.603411 28644 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:15.608433 23448 rpcz_store.cc:276] 0709 13:09:13.885470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.885593 (+   123us) service_pool.cc:224] Handling call
0709 13:09:13.898599 (+ 13006us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:13.904451 (+  5852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:13.904478 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:13.906088 (+  1610us) log.cc:844] Serialized 26 byte log entry
0709 13:09:15.225575 (+1319487us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b1800036640. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:09:15.456217 (+230642us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:15.456254 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:15.456332 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:15.571126 (+114794us) raft_consensus.cc:1685] finished
0709 13:09:15.571165 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:15.571380 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16013,"raft.queue_time_us":7098,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5682,"spinlock_wait_cycles":12208512,"thread_start_us":230005,"threads_started":1,"wal-append.queue_time_us":1641634}
I20260709 13:09:15.640144 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.606040 28894 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.609373 28717 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:09:15.607802 28808 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.643513 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:15.645001 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.599541 28810 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:15.646576 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48223.9us
W20260709 13:09:15.650225 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 154378us
I20260709 13:09:15.679549 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:15.650650 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.647786 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.617898 28729 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.640533 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:15.652721 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54932.6us
I20260709 13:09:15.683813 22824 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:15.641212 22749 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:15.685204 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.610548 28534 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.651700 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.643308 28544 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:15.681627 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.682979 22794 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:15.688457 22868 leader_election.cc:400] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16.
I20260709 13:09:15.689216 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:15.691594 22868 leader_election.cc:403] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:15.692101 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.651520 28830 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:15.690054 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.693344 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.696349 22826 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:15.697088 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.697678 22826 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:15.698316 28624 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.698640 28848 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:15.699095 28848 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:15.701625 22754 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:15.704181 28167 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16.
I20260709 13:09:15.700979 28810 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:15.710342 28899 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:09:15.712138 28746 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.717525 28814 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.644114 28708 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.727939 22754 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:15.724036 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.642179 28839 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.732990 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.735057 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.735821 22826 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:15.642339 28562 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.739105 28889 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:09:15.739492 28908 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:15.701606 28696 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:15.648587 28836 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.733963 22824 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:15.738519 28893 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:15.738773 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.759102 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:15.633203 28895 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.695326 28806 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:09:15.691491 28561 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:15.739929 28908 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.769817 28767 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.759392 22822 raft_consensus.cc:2393] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:15.770237 28908 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:15.739233 28907 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:09:15.771440 28839 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:15.772498 28562 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.679787 28894 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.773002 28836 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:15.737653 28665 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.785948 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.768935 28808 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.773562 22905 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:09:15.776607 23446 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:15.777709 28908 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.778400 28534 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.778808 28779 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.800890 28779 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.780061 28708 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.780324 28806 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:15.756740 28830 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:09:15.801280 28908 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.801782 28806 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:15.781234 28895 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.781939 28878 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.782020 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.802977 28806 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:15.781777 28544 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.804543 28806 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:15.760848 28890 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
W20260709 13:09:15.782619 28651 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.002s
I20260709 13:09:15.782720 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.809201 22827 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:15.809450 28651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:42486 (request call id 21) took 2130 ms. Trace:
I20260709 13:09:15.810024 28651 rpcz_store.cc:276] 0709 13:09:13.679144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.679331 (+   187us) service_pool.cc:224] Handling call
0709 13:09:13.767446 (+ 88115us) log.cc:844] Serialized 499 byte log entry
0709 13:09:14.416107 (+648661us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b5400439520. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d7bb26 00007fcb74d7549a 00007fcb74d74714 00007fcb74d54eaa 00007fcb74d5bd53 00007fcb74d5bcd9 00007fcb74d5bc69 00007fcb74d5bc31 00007fcb74d5adfd 000000000038f354
0709 13:09:14.494824 (+ 78717us) spinlock_profiling.cc:243] Waited 6.68 ms on lock 0x7b5400439520. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d74142 00007fcb74d54eaa 00007fcb74d5bd53 00007fcb74d5bcd9 00007fcb74d5bc69 00007fcb74d5bc31 00007fcb74d5adfd 000000000038f354 000000000038f199 00007fcb71d0e475
0709 13:09:15.208083 (+713259us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b180045e6e0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db
0709 13:09:15.210022 (+  1939us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:15.809097 (+599075us) log.cc:844] Serialized 14 byte log entry
0709 13:09:15.809350 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36871,"raft.queue_time_us":225550,"raft.run_cpu_time_us":5036,"raft.run_wall_time_us":712208,"spinlock_wait_cycles":109781248,"thread_start_us":483182,"threads_started":3,"wal-append.queue_time_us":242123}
I20260709 13:09:15.753921 28696 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:15.783411 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:15.757440 28909 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:09:15.784107 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.818281 22824 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:15.725189 28750 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.784617 23453 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 13:09:15.785262 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.768697 28807 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:15.820796 28807 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:15.789743 22826 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:09:15.761662 28791 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:09:15.739507 28768 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.797245 28665 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:15.835364 28665 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:15.833235 28708 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.836400 28708 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.838428 28708 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:15.839332 22822 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:15.841360 28914 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 13:09:15.842216 28814 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:15.842605 28814 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:15.843418 28814 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.842222 28892 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:09:15.848999 28894 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:15.791522 28769 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:15.853991 28769 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.856014 28769 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:15.858807 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:15.860327 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 106914us
I20260709 13:09:15.867322 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:15.832105 28534 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:15.879432 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 186256us
I20260709 13:09:15.870605 28814 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:15.880309 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:15.889479 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 149052us
W20260709 13:09:15.903618 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 161978us
I20260709 13:09:15.904685 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:15.905526 28717 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:15.892198 28807 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.906494 28806 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:15.906857 28806 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:15.907198 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:15.907661 28806 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.909096 28768 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:15.909255 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:15.909525 28768 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:15.909905 28768 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:15.912732 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:15.914216 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:15.916805 28473 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.920226 22901 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:15.907979 28807 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:15.930883 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.942137 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:15.943728 23449 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:15.942931 22903 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:15.870466 28880 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:09:15.887769 28700 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:15.870805 28282 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:09:15.955262 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.955089 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.955264 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:15.960848 22901 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:15.961297 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:15.962457 23448 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:15.919188 28708 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:15.964200 22748 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:15.941123 28873 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:09:15.967322 22753 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:15.958287 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:15.889976 28534 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:15.973569 22900 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:15.975521 22747 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:09:15.990549 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.978751 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:15.996435 22822 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:15.979769 23444 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:15.997762 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.980737 22826 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:09:15.981436 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:16.001257 22824 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 14: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:09:15.963927 22868 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 18a127a20cfa4496bad51cb634bd0d0f. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:16.001816 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 80074.7us
I20260709 13:09:16.013602 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:16.024802 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:16.029752 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.029831 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:15.946041 28792 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:16.002669 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:16.031279 22794 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:16.064591 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:16.084636 23448 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:16.071311 22901 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:16.090698 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:16.088791 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 86248.8us
I20260709 13:09:16.096911 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:16.107995 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:16.109282 22822 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:15.950064 28473 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.110540 22896 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 13:09:16.111311 22903 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:16.057660 28758 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:16.063745 28792 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.142814 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.341933 28792 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:16.144219 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.344704 22903 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:09:16.228722 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 121880us
I20260709 13:09:16.146267 28769 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:16.144289 28779 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.348733 28769 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:16.349436 28769 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:16.349867 28779 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.227412 28877 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:09:16.207964 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:16.110324 28903 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:09:16.199167 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:16.207077 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 240823us
I20260709 13:09:16.086069 28909 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.257911 28473 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.087532 28599 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:16.374308 28599 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:16.375891 28599 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:15.964493 28700 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.380134 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.059034 28750 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:16.263300 28806 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:16.093839 28808 raft_consensus.cc:2764] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:09:16.284070 28920 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:09:16.399647 22903 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:09:16.294098 28918 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:16.086787 28928 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:09:16.253791 28526 consensus_meta.cc:287] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.003s
I20260709 13:09:16.288499 28534 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.260237 28696 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:16.403689 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.404019 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2785) took 1445 ms. Trace:
I20260709 13:09:16.093204 28708 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:16.408500 28847 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.286106 28768 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.401072 28928 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.227608 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.356712 22747 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
W20260709 13:09:16.362360 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.147149 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:16.358940 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.363337 22897 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:09:16.380339 28700 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.112859 28746 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:16.418078 22896 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:16.375943 28848 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.420219 28848 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:16.388096 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.373032 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:16.368829 28779 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:16.142257 28921 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:09:16.408419 28708 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:16.413280 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2999) took 1158 ms. Trace:
I20260709 13:09:16.429276 22828 rpcz_store.cc:276] 0709 13:09:15.254605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.254713 (+   108us) service_pool.cc:224] Handling call
0709 13:09:15.267184 (+ 12471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.306651 (+ 39467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:16.412675 (+1106024us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b540043b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:16.413025 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71788416}
I20260709 13:09:16.415254 22871 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:16.415103 28526 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.432153 22905 raft_consensus.cc:2368] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17.
I20260709 13:09:16.432353 28526 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.434933 28526 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.381613 28750 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:16.437230 28750 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.412168 28768 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.414326 28534 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.438329 28534 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.404558 22750 rpcz_store.cc:276] 0709 13:09:14.958322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:14.958493 (+   171us) service_pool.cc:224] Handling call
0709 13:09:14.958794 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:15.000286 (+ 41492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.000321 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:15.001727 (+  1406us) log.cc:844] Serialized 26 byte log entry
0709 13:09:15.130473 (+128746us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:15.130525 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:16.403361 (+1272836us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b5400267ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:16.403458 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:16.403530 (+    72us) raft_consensus.cc:1685] finished
0709 13:09:16.403543 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:16.403826 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":852,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"spinlock_wait_cycles":94708096,"thread_start_us":124248,"threads_started":1,"wal-append.queue_time_us":248057}
I20260709 13:09:16.441205 28534 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.443059 28909 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:16.413192 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 348537us
I20260709 13:09:16.412855 28928 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.457793 28700 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:16.458636 28700 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:16.460049 28926 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.252s	user 0.004s	sys 0.000s
I20260709 13:09:16.462014 28808 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:16.463392 28627 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:16.471127 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 75611.8us
W20260709 13:09:16.412741 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 304668us
I20260709 13:09:16.476771 28750 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.477217 28895 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:09:16.477511 28750 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:16.480325 28750 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.481603 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:16.483688 28895 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:16.484220 28895 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:16.484249 23446 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:16.485688 28895 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.506132 28893 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.506947 28893 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.460249 28700 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:16.511070 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 45736.8us
I20260709 13:09:16.409307 28847 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:16.447145 28792 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.517746 28893 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:16.519163 28923 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:09:16.517171 28909 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.525081 28928 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.525738 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.525748 28928 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.526259 28808 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.527069 28808 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.528254 28928 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.499060 28769 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:16.531240 28808 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.577584 28906 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.577695 28708 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:16.578666 28708 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.581158 28708 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.581390 28895 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:16.582259 28895 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:16.366816 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:16.587834 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.588304 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.588558 23446 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:16.590710 28563 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.591513 28563 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.578554 28906 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.598253 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.600637 23450 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:16.601431 28906 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.602600 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:16.603060 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.603539 22748 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 15: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:16.603770 23446 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:16.604357 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:16.605122 22750 raft_consensus.cc:3060] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:16.605988 28563 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.609047 28892 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:16.611654 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2719) took 1123 ms. Trace:
W20260709 13:09:16.612421 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 68802.6us
I20260709 13:09:16.612051 22898 rpcz_store.cc:276] 0709 13:09:15.487822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.487998 (+   176us) service_pool.cc:224] Handling call
0709 13:09:15.488323 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:15.764035 (+275712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.764078 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:15.764132 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:15.764180 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:16.611223 (+847043us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400107554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:16.611283 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:16.611491 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633551232}
I20260709 13:09:16.614656 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:16.617779 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.618554 28792 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.626262 28847 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.573942 28808 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:16.585310 28895 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.635116 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.634836 28808 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:16.639036 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62440.8us
I20260709 13:09:16.639685 28808 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.645776 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:16.647527 28769 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:16.649430 28934 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.394s	user 0.000s	sys 0.002s
I20260709 13:09:16.649441 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1547) took 1321 ms. Trace:
I20260709 13:09:16.650177 23444 rpcz_store.cc:276] 0709 13:09:15.327612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.327751 (+   139us) service_pool.cc:224] Handling call
0709 13:09:15.331237 (+  3486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:16.035904 (+704667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:16.648829 (+612925us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b5400672d54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:16.649198 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491069824}
I20260709 13:09:16.651014 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1809) took 1212 ms. Trace:
W20260709 13:09:16.641163 28941 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:09:16.651391 22752 rpcz_store.cc:276] 0709 13:09:15.438824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.619612 (+180788us) service_pool.cc:224] Handling call
0709 13:09:15.619900 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:15.620219 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:15.620247 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:15.918310 (+298063us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b2c00027958. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:09:15.943344 (+ 25034us) log.cc:844] Serialized 26 byte log entry
0709 13:09:16.337834 (+394490us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:16.338668 (+   834us) log.cc:844] Serialized 14 byte log entry
0709 13:09:16.340325 (+  1657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:16.340454 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:16.648686 (+308232us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:16.650652 (+  1966us) raft_consensus.cc:1685] finished
0709 13:09:16.650676 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:16.650845 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2800,"prepare.queue_time_us":17206,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":12850,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":6123,"spinlock_wait_cycles":65190528,"thread_start_us":363262,"threads_started":1,"wal-append.queue_time_us":304868}
I20260709 13:09:16.653215 22900 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
W20260709 13:09:16.647905 28933 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.380s	user 0.004s	sys 0.000s
I20260709 13:09:16.662792 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.663640 23450 raft_consensus.cc:2368] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16.
I20260709 13:09:16.673034 22750 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:16.678447 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.683074 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.685695 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.685091 23444 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:16.684108 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:16.688585 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.689288 23453 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:16.687732 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.691057 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.696020 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.697268 22750 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:16.696810 22896 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:16.696753 22905 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:16.699219 28769 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.702229 28769 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.704650 28769 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:16.706249 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.694141 28717 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:16.708284 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.711413 23444 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 23: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:16.714493 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.710089 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.717067 23450 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:16.717779 23445 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:09:16.708541 28717 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:16.719141 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:16.719442 28717 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:16.719774 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.720440 22752 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:09:16.721148 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.721345 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.721139 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:16.722246 28792 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:16.724606 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:16.721197 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:16.709692 28906 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.718473 28895 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:16.734474 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.724478 28518 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 13:09:16.732990 28847 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:16.738731 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.739243 28847 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.739727 22756 raft_consensus.cc:2393] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:16.739665 28847 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:16.736153 28895 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:16.725178 28769 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:09:16.745963 28792 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:16.753793 28696 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:16.750628 28895 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:16.757021 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.761056 22748 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:16.753953 28792 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.747251 28906 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:16.770351 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.764376 28563 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:16.777046 28563 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.775717 28923 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:16.779512 28923 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.780001 28563 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.776333 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:16.785152 28923 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:16.786067 28949 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:09:16.787387 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:16.788619 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:16.788757 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:16.789384 22897 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:16.789748 28823 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:16.790251 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:16.792505 28849 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:16.792870 28849 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.793263 28849 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:16.793728 28644 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:16.794234 28644 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.794644 28644 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:16.789762 22897 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:16.796794 22897 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.797992 22897 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:16.776089 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:16.825894 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 49917.8us
W20260709 13:09:16.834995 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46559.8us
I20260709 13:09:16.836910 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:16.834937 28951 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:09:16.845605 28953 ts_tablet_manager.cc:933] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.848618 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:16.843715 28769 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:16.849642 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.869177 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.869699 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.870226 28769 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.870654 28769 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:16.870875 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.871935 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.872429 22747 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:09:16.875509 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:16.876554 28814 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:16.877207 28814 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:16.877617 28814 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:16.877959 28814 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:16.854116 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.864840 28906 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:16.881604 22751 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:16.856513 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.842787 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.857594 28745 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:16.861964 28952 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:09:16.867137 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 110112us
I20260709 13:09:16.869838 22754 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:16.850539 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.839370 28563 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.857489 28878 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:16.893465 22898 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:16.900741 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:16.906437 28603 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:16.917917 22748 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:09:16.918165 28653 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:16.918344 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:16.919175 28745 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.923084 28839 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:16.925077 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.926644 28524 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.928229 22904 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:09:16.916353 28644 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.932232 28653 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:16.930446 28603 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.933784 28563 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:16.957562 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.939332 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:16.942883 22896 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:16.937846 22756 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:09:16.938196 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:16.963092 22747 raft_consensus.cc:2393] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:16.957657 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.960021 22756 pending_rounds.cc:85] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 10
I20260709 13:09:16.964875 22756 pending_rounds.cc:107] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:09:17.006362 22756 raft_consensus.cc:2889] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:16.958745 28603 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:16.954758 28524 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:17.015833 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.016556 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:17.019186 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:17.012750 28823 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:17.047742 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.059073 22827 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:09:17.047904 28773 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:17.074352 28925 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:17.075201 28925 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.077468 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.078011 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.078816 22822 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:17.058178 28717 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.080160 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:17.081615 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:17.082441 22830 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:17.083702 28653 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.083930 28847 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.085697 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:17.086922 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.087637 22823 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 15: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:17.089452 28906 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.089641 28823 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:17.090030 28906 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:17.091991 28906 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.093175 22826 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:17.080277 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:17.093871 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 74755.4us
I20260709 13:09:17.081037 22822 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:17.092607 22905 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:17.095985 28814 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:17.098088 28814 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.099442 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.100083 22827 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:17.100973 28717 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.088284 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.101284 22827 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:09:17.102986 28878 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:17.118525 28918 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.155s
I20260709 13:09:17.119004 28878 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:17.125821 28955 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:09:17.126120 22651 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 7 to 12, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 12 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.127888 22651 catalog_manager.cc:5697] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:17.130048 28849 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.131356 28745 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:17.136392 22650 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 339) took 25143 ms (client timeout 15000 ms). Trace:
I20260709 13:09:17.137552 22651 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 13, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68), now has a pending config: VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65), VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66), VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67), VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 13 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } } pending_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.092026 28823 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.162495 28745 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:17.189154 28745 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:17.191411 28745 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:17.182757 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.601s (SENT) [suppressed 1 similar messages]
W20260709 13:09:17.142460 22650 rpcz_store.cc:269] 0709 13:08:52.041722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.055456 (+ 13734us) service_pool.cc:224] Handling call
0709 13:09:17.136088 (+25080632us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:15.122398 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:15.311048 (+188650us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:15.326776 (+ 15728us) write_op.cc:435] Acquired schema lock
0709 13:09:15.337851 (+ 11075us) tablet.cc:662] Decoding operations
0709 13:09:15.384805 (+ 46954us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:15.411555 (+ 26750us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:15.423180 (+ 11625us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:09:15.436289 (+ 13109us) tablet.cc:701] Row locks acquired
0709 13:09:15.439417 (+  3128us) write_op.cc:260] PREPARE: finished
0709 13:09:15.491575 (+ 52158us) write_op.cc:270] Start()
0709 13:09:15.506017 (+ 14442us) write_op.cc:276] Timestamp: P: 1783602555483030 usec, L: 0
0709 13:09:15.514740 (+  8723us) op_driver.cc:348] REPLICATION: starting
0709 13:09:15.610646 (+ 95906us) log.cc:844] Serialized 533 byte log entry
0709 13:09:16.297656 (+687010us) op_driver.cc:464] REPLICATION: finished
0709 13:09:16.299378 (+  1722us) write_op.cc:301] APPLY: starting
0709 13:09:16.299479 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:16.300686 (+  1207us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:16.300797 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:16.301521 (+   724us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:16.301727 (+   206us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:09:16.301786 (+    59us) write_op.cc:312] APPLY: finished
0709 13:09:16.301985 (+   199us) log.cc:844] Serialized 25 byte log entry
0709 13:09:16.302152 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:16.302243 (+    91us) write_op.cc:454] Released schema lock
0709 13:09:16.302281 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:16.302671 (+   390us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:16.302701 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":537,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":1419,"apply.run_cpu_time_us":3293,"apply.run_wall_time_us":3293,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8307,"num_ops":1,"prepare.queue_time_us":105808,"prepare.run_cpu_time_us":55837,"prepare.run_wall_time_us":726690,"raft.queue_time_us":235114,"raft.run_cpu_time_us":1679,"raft.run_wall_time_us":1739,"replication_time_us":782763,"thread_start_us":434303,"threads_started":4,"wal-append.queue_time_us":302194}]]}
I20260709 13:09:17.193446 22831 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:09:17.188163 28878 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.200938 28644 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:17.183810 22651 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.206319 28918 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:17.216185 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:17.219924 22904 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:17.220656 28953 tablet_copy_client.cc:323] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.67:35797
I20260709 13:09:17.227516 22754 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:17.224644 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:17.234476 22747 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:17.176144 28906 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:17.230393 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 354980us
I20260709 13:09:17.199841 28823 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:17.257604 22651 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 11, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 11 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.267112 28925 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.267545 22651 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 14 to 16, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 16 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.268998 28849 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.271622 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:17.273082 28878 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.273905 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:17.276335 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 69133.6us
W20260709 13:09:17.278352 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 95602.8us
I20260709 13:09:17.269001 22651 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 16 to 18, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 18 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:17.283447 22747 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:17.284821 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:17.290063 22751 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:17.277326 28627 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:17.216805 28895 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.297255 22868 leader_election.cc:400] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16.
I20260709 13:09:17.297814 22868 leader_election.cc:403] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:17.298333 22905 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:17.289043 28814 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:17.340310 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:17.343019 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69294.6us
I20260709 13:09:17.341905 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:17.281713 28906 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:17.339844 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.345496 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:17.342000 22651 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 15 to 17, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 17 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.346148 22905 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:17.298851 28524 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.350529 28745 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:17.350072 22651 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 12, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 12 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.351758 22651 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 13 to 15, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 15 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.352628 22830 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:17.353502 22827 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:17.355026 22651 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 14, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 14 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:17.292929 28823 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.359524 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:17.340358 28773 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16.
I20260709 13:09:17.362948 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:17.349418 28893 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:17.369043 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:17.372454 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44495.7us
I20260709 13:09:17.345439 28627 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.376801 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:17.377425 22897 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:09:17.379132 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:17.380126 22830 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:17.381248 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:17.382506 28906 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.382613 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:17.383152 28906 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:17.383569 28906 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:17.383949 28906 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:17.384380 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 226105us
I20260709 13:09:17.385452 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:17.385771 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:17.386018 22797 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:17.343714 28923 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:09:17.386471 23447 raft_consensus.cc:2368] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 23. Current term is 24.
I20260709 13:09:17.386868 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:17.386960 28759 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:17.387447 28759 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:17.388482 28759 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.389298 28524 raft_consensus.cc:2804] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:17.389758 28759 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:17.389853 28524 raft_consensus.cc:697] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:17.391043 28524 consensus_queue.cc:237] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:17.392592 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 67449ba22ccb4938be8a5e28664cc566: Remote error: Service unavailable: leader transfer in progress
I20260709 13:09:17.394011 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:17.394816 22900 raft_consensus.cc:2410] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 16 because replica has last-logged OpId of term: 14 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 14.
I20260709 13:09:17.395668 28665 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:17.396067 28665 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.396431 28665 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:17.401533 22904 raft_consensus.cc:2393] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:17.402720 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 71505.3us
I20260709 13:09:17.403681 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:17.404997 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:17.407196 22651 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 9 to 12, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 12 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.377713 28893 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:17.407392 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:17.410946 22897 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:17.416710 22822 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:17.416898 22868 leader_election.cc:400] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 23. Current term is 24.
I20260709 13:09:17.417347 22868 leader_election.cc:403] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:09:17.419719 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:17.402719 28878 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:17.422475 23446 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:09:17.424044 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.376309 28769 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.425064 22651 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 9 to 12, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 12 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:17.416575 28893 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:17.436046 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 47666.9us
I20260709 13:09:17.436594 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:17.423117 28839 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:17.437016 22797 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:17.424814 28878 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:17.456225 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:17.460929 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:17.409485 28975 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:09:17.461493 22822 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:17.473115 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:17.471976 22651 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 15, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 15 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:09:17.534742 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.539942 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:17.457796 28878 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:17.543586 22830 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
W20260709 13:09:17.547981 28603 consensus_meta.cc:287] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.002s
I20260709 13:09:17.548369 28603 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.549479 22831 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:17.556806 22822 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:17.558037 28627 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.559370 28792 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:17.473726 28653 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:17.539228 28895 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.563416 22651 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 13, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 13 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.565500 28974 raft_consensus.cc:2764] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:09:17.566136 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:17.581717 23416 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:17.586992 28986 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:17.598869 28980 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.599452 28980 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.604019 28524 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:17.605481 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 168504us
I20260709 13:09:17.605690 28982 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:09:17.606832 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 138709us
I20260709 13:09:17.609584 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:17.608318 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:17.611567 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:17.565403 28823 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.562512 28653 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.559938 28792 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:17.604700 28524 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.624661 22751 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:09:17.615371 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:17.628568 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:17.631065 22651 catalog_manager.cc:5697] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 6 to 9, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 9 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:17.608390 28978 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:17.584090 28644 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:17.635087 22651 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 15 to 18, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 18 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:17.636245 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:17.607527 28925 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:17.607826 28895 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:17.609112 28982 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:17.626870 28653 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:17.623769 28980 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.653077 22651 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 13, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 13 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:17.640034 28823 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.639411 28978 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:17.639703 28963 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.644150 28974 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:17.661216 28895 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:17.675077 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 59779.2us
I20260709 13:09:17.710670 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:17.723739 22751 raft_consensus.cc:2368] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 23. Current term is 24.
I20260709 13:09:17.675179 22651 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 14 to 18, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 18 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:17.674860 28977 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:09:17.724977 22651 catalog_manager.cc:5697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 14, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 14 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:17.708699 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:17.740918 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.710301 28814 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.031s
I20260709 13:09:17.743261 28745 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:17.743886 28895 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.744149 28563 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.731860 28984 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:09:17.745012 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:17.667284 28906 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.745841 23449 raft_consensus.cc:2410] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 16 because replica has last-logged OpId of term: 14 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 14.
I20260709 13:09:17.747480 22754 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:17.665573 28627 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:17.668102 28978 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:17.753732 28973 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.337s	user 0.020s	sys 0.048s
I20260709 13:09:17.751201 28906 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.674456 28823 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:17.751477 28847 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.694248 28603 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.673069 28923 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:17.758258 28923 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:17.758677 28923 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:17.759059 28968 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 13:09:17.692911 28982 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:17.762007 28982 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:17.695679 28893 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.763438 28717 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:17.763837 28717 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:17.764381 28893 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:17.768607 28979 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:09:17.770980 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:17.771885 28970 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.422s	user 0.008s	sys 0.016s
W20260709 13:09:17.771997 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 61061.5us
I20260709 13:09:17.772776 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:17.773062 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1823) took 1059 ms. Trace:
I20260709 13:09:17.773499 28991 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:17.774382 28991 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:17.773442 22750 rpcz_store.cc:276] 0709 13:09:16.714018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.714141 (+   123us) service_pool.cc:224] Handling call
0709 13:09:16.730470 (+ 16329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:16.756165 (+ 25695us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:16.756188 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:16.757262 (+  1074us) log.cc:844] Serialized 26 byte log entry
0709 13:09:17.236444 (+479182us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:17.236536 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:17.236791 (+   255us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:17.771238 (+534447us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:17.772815 (+  1577us) raft_consensus.cc:1685] finished
0709 13:09:17.772834 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:17.773011 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1790,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":7610624,"thread_start_us":470578,"threads_started":1,"wal-append.queue_time_us":528481}
I20260709 13:09:17.775492 28991 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:17.776633 28967 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 13:09:17.777716 28975 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:17.778467 28792 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:17.780550 28981 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.176s	user 0.002s	sys 0.000s
I20260709 13:09:17.784134 28823 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:17.784685 22651 catalog_manager.cc:5697] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 9 to 13, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 13 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
W20260709 13:09:17.794448 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 173603us
I20260709 13:09:17.795320 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:17.796911 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 70341.1us
I20260709 13:09:17.690845 28769 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:17.804466 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:17.699535 28983 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:17.764613 28717 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.784929 28878 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:17.817235 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44533.6us
I20260709 13:09:17.840296 22751 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:17.696264 28665 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:09:17.861622 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 47959.9us
W20260709 13:09:17.861542 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 57178.1us
I20260709 13:09:17.872126 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:17.872900 22756 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:09:17.880200 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.880656 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:17.880795 22829 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:17.881367 22748 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:17.827263 28983 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.875427 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:17.871767 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:17.882869 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:17.874526 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 227361us
I20260709 13:09:17.883795 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:17.938002 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 135e2f7520a34b53aaf4ddb360a8a029: Illegal state: Replica 6a5191aaff76469bbf6c4735dc9e6509 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } } pending_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:09:17.946171 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 63387.2us
I20260709 13:09:17.710659 28974 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:17.710913 28963 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:17.993952 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:17.877059 28849 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:17.991853 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:17.993032 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:18.003389 22827 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:09:17.865453 28992 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.992460 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:17.878273 28563 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:09:18.002038 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 159860us
I20260709 13:09:17.995606 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:18.009848 22828 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:17.865794 28918 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.001907 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:17.945974 28717 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:18.005241 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:18.005302 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:17.980154 28983 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:18.020763 23449 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:18.022850 22755 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:17.994302 28792 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:18.037755 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:18.002816 28895 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:18.056069 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 64230.8us
I20260709 13:09:18.012972 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.008731 28769 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:09:18.069507 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 73405.4us
I20260709 13:09:18.029258 28524 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:18.095152 22751 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:09:18.097640 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 59979.4us
I20260709 13:09:18.064064 28563 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:18.070345 28990 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:18.069866 28769 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.098410 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:18.099520 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.036947 28847 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:09:18.112612 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:18.098626 28839 consensus_meta.cc:287] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.000s
I20260709 13:09:18.081918 28773 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:18.030895 28918 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.015451 28992 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:18.131985 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 256719us
I20260709 13:09:18.108105 28769 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:18.142624 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 59723.4us
I20260709 13:09:18.142926 28665 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:18.140476 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:18.136714 28849 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:18.072575 28895 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.133462 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:18.179164 22825 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:18.123139 28839 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 23. Current term is 24.
I20260709 13:09:18.126672 28990 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:09:18.191649 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48062.7us
I20260709 13:09:18.191933 28563 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.194417 28978 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.187786 28696 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:18.196576 28992 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.199939 28925 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:18.128650 28773 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:18.127839 28823 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:18.224725 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:18.207648 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:18.229815 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:18.256088 22748 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:18.256377 22825 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
W20260709 13:09:18.256474 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48892.2us
I20260709 13:09:18.185495 28653 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.257266 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:18.257653 28759 consensus_meta.cc:287] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.867s	user 0.010s	sys 0.001s
I20260709 13:09:18.258836 28653 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.259920 28759 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:18.260344 28759 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:18.244273 28878 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:18.264398 22748 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:18.266901 28892 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:18.267428 28892 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:18.267819 28892 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:18.233191 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 220327us
I20260709 13:09:18.234090 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:18.271130 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.184515 29000 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.185704 29005 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:18.245251 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 104912us
I20260709 13:09:18.248121 28975 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:09:18.224817 28990 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.291332 28990 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:18.292662 29001 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 13:09:18.270449 28773 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:18.250794 28839 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:18.254505 28923 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.255435 28918 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:18.271310 22822 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:18.129990 28984 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.068s
W20260709 13:09:18.276806 23447 consensus_meta.cc:287] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.660s	user 0.005s	sys 0.004s
I20260709 13:09:18.332713 23447 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:18.332830 22831 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:18.243853 28696 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.282055 22824 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:18.282866 28895 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:18.283527 28993 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
W20260709 13:09:18.283793 28988 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.399s	user 0.001s	sys 0.000s
W20260709 13:09:18.288023 28996 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.153s	user 0.002s	sys 0.000s
I20260709 13:09:18.271642 28925 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.249197 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:18.301240 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:18.355432 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 62715.6us
I20260709 13:09:18.298312 29003 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:18.304152 28978 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.304443 28974 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.304587 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:18.304876 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.242769 28983 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.305809 22827 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:18.306330 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:18.307389 29000 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:18.308804 29005 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.269762 28963 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:18.312108 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53153.2us
W20260709 13:09:18.366149 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 65808.6us
I20260709 13:09:18.367116 22871 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.314430 29004 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Committing config change with OpId 11.11: config changed from index 7 to 11, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:18.318588 28717 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:18.293377 28849 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:18.319494 28982 consensus_meta.cc:287] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.557s	user 0.005s	sys 0.005s
I20260709 13:09:18.261189 28759 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:18.323148 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 83648.2us
I20260709 13:09:18.246315 28792 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 24 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:18.339828 22826 raft_consensus.cc:2955] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:09:18.376137 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.322495 23446 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:18.354861 22828 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:18.382226 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:18.382673 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.368876 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:18.384485 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 79989.2us
I20260709 13:09:18.386662 22829 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
W20260709 13:09:18.386934 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 80673.4us
I20260709 13:09:18.390838 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.360169 28986 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:18.396744 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:18.392520 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:18.378512 28839 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:18.402185 29003 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:18.400890 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:18.382654 28918 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:18.386504 28974 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:18.403558 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 330412us
I20260709 13:09:18.415005 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:18.412575 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:18.418470 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:18.387605 28963 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:18.420737 22828 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:18.423414 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:18.399540 28524 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:18.390863 29015 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:18.402874 28717 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:18.406389 28986 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:18.432585 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:18.436187 22719 leader_election.cc:400] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17.
I20260709 13:09:18.436508 23450 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:18.437183 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.437757 28839 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:18.438328 28839 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.439102 28975 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:18.400465 28769 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.441255 29003 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:18.442914 29003 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.443252 29003 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:18.444149 29010 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:18.444597 29010 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.444955 29010 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:18.445408 28987 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:18.445479 29012 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.445976 28987 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.446466 28987 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:18.446241 29012 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.449801 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:18.451056 28994 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:18.451682 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:18.455497 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.456453 28995 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:18.457007 28995 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.459537 28745 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:18.459743 23450 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:18.438512 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:18.462886 28603 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:18.463692 28745 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:18.432895 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:18.467213 23449 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:18.408646 28992 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:18.430047 29002 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:18.403018 28895 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:09:18.470875 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1226) took 1264 ms. Trace:
W20260709 13:09:18.475075 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 72796.9us
I20260709 13:09:18.462558 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:18.474458 22830 rpcz_store.cc:276] 0709 13:09:17.206890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.545661 (+338771us) service_pool.cc:224] Handling call
0709 13:09:17.546013 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:17.642274 (+ 96261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:17.642315 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:17.665432 (+ 23117us) log.cc:844] Serialized 52 byte log entry
0709 13:09:17.994782 (+329350us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:17.994851 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:17.994926 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:18.287239 (+292313us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:18.466268 (+179029us) raft_consensus.cc:1685] finished
0709 13:09:18.466288 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:18.470777 (+  4489us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13937,"raft.queue_time_us":58712,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":182,"spinlock_wait_cycles":18115968,"thread_start_us":301875,"threads_started":2,"wal-append.queue_time_us":359427}
I20260709 13:09:18.488796 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.489172 22871 leader_election.cc:400] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 23. Current term is 24.
I20260709 13:09:18.471495 28974 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:18.431077 29014 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:09:18.473161 29002 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:18.498728 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 86213.2us
W20260709 13:09:18.501012 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 87472us
I20260709 13:09:18.491639 28975 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:09:18.507551 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 45083.3us
I20260709 13:09:18.468108 28963 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.499168 28980 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:18.474612 28992 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.515027 22747 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
W20260709 13:09:18.515205 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 127658us
I20260709 13:09:18.513545 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:18.504791 29002 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:18.524299 28995 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:18.531461 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:18.531616 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.532531 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:18.533056 22754 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:18.533296 22901 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:18.533454 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.533843 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.534226 22902 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:18.533237 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.536823 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:18.538825 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.540124 28769 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.541755 29000 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:18.542292 29000 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:18.545682 28918 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:18.546363 28918 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.547624 28983 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.548705 28918 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:18.540124 28839 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:18.551820 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:18.554186 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:18.488059 28892 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:18.569983 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:18.569415 28986 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:18.571141 28892 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.471647 28792 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.506291 28923 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.495599 28717 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:09:18.586577 22749 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:18.510162 28975 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:18.596385 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76717.9us
I20260709 13:09:18.600739 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.601424 22898 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:18.604235 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.605347 22897 raft_consensus.cc:2410] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:09:18.607287 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:18.588137 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.610535 22905 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:09:18.610883 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.589291 22754 raft_consensus.cc:2468] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:18.614701 22749 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:18.617303 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.611443 22902 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:09:18.618566 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:18.619997 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:18.621054 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:18.585772 28925 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.600603 28975 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:18.589002 28839 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:18.634793 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:18.631273 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:18.588052 29019 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:09:18.644866 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:18.647346 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46334.8us
I20260709 13:09:18.649618 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:18.613160 28918 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:18.589126 29000 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:09:18.623785 28974 raft_consensus.cc:2764] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:09:18.665530 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.639464 28878 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:18.634699 28906 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:18.671634 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.673800 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.674332 22901 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:18.676662 28923 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:18.677661 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:18.681236 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:18.681735 22914 tablet_copy_service.cc:140] P 3b626c41600d4737a178ebab8a7ebfc2: Received BeginTabletCopySession request for tablet 18a127a20cfa4496bad51cb634bd0d0f from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:42490)
I20260709 13:09:18.682415 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:18.682663 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:18.683518 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:18.683660 29032 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:18.684070 23452 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:18.684240 29032 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.685176 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:18.682420 22914 tablet_copy_service.cc:161] P 3b626c41600d4737a178ebab8a7ebfc2: Beginning new tablet copy session on tablet 18a127a20cfa4496bad51cb634bd0d0f from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:42490: session id = c45e1805f049467697a7358fb566de2f-18a127a20cfa4496bad51cb634bd0d0f
I20260709 13:09:18.673797 22899 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:18.686671 22901 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:18.688354 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.688898 22901 raft_consensus.cc:2376] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16. Re-sending same reply.
I20260709 13:09:18.689004 23449 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:18.591187 29002 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:18.690732 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 168604us
I20260709 13:09:18.646155 22748 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 7 to 11, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:18.654327 28823 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:18.696610 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.697387 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.698029 22897 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:18.699370 28963 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:18.699688 29021 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:09:18.699959 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.700290 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:18.700536 22901 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:18.700825 29008 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:18.701333 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.701890 22901 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:18.622026 28990 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.672173 29005 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:18.724202 23444 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:18.672472 28918 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.716706 23448 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:18.723798 28627 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:18.699906 28963 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:09:18.767355 22831 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:09:18.703677 28823 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:18.752130 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 105621us
I20260709 13:09:18.656203 28839 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.656391 29019 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.702170 22829 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 7 to 11, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:18.764272 22901 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:18.761137 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:18.758275 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.777720 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:18.768873 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:18.774646 28823 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:18.780854 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:18.780925 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.781705 22901 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:18.706110 28717 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:09:18.758350 28878 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:18.764935 28986 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:18.792090 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:18.782950 29023 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.786902 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:18.793140 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 138141us
I20260709 13:09:18.793550 22822 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:18.809930 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:18.810170 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.798915 28665 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:18.777216 28627 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.816134 22822 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:09:18.822661 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:18.816839 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.829378 22827 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:18.838618 22897 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:18.822540 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:18.821233 22903 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:09:18.839073 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:18.839757 22754 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:18.842417 22898 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:09:18.844831 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:18.830047 22797 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:18.847443 22751 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:18.849661 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:18.799033 28986 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.850909 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:18.852263 29049 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:18.819038 29023 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.832924 28983 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:18.853186 29032 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:18.891431 29002 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.891582 29032 consensus_queue.cc:1059] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:09:18.891957 29035 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:09:18.892021 28987 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.894258 28987 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:18.854485 29041 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:09:18.855137 22914 tablet_copy_source_session.cc:215] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:09:18.856729 29027 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:09:18.874650 28563 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:18.856995 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:18.829391 28980 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:18.814978 29047 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:18.862453 29003 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.827407 29012 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:18.833755 28773 consensus_meta.cc:287] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 13:09:18.875416 22901 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:18.877455 22828 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:18.879361 22754 raft_consensus.cc:1240] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:09:18.829984 28717 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.977025 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:18.862566 29010 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.915578 29047 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:18.986446 29047 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:18.989391 28717 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.991417 28717 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:18.885536 22831 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:18.837329 28665 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.895699 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:18.998433 28665 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:19.000331 28975 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.896052 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:19.001339 22750 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:19.001959 28839 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.002107 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:19.002811 28839 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:19.003326 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.901358 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:19.003978 22898 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:19.004782 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.932153 28878 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.901888 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:18.878549 29049 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:18.879705 28982 raft_consensus.cc:2804] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:09:18.913427 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56516.5us
I20260709 13:09:18.882910 28759 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:18.894430 29039 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:09:18.920871 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:19.033776 23448 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:18.974109 22899 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:18.918777 22905 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:18.917728 28563 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.919346 28974 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:18.893283 29023 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.043329 28974 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.043902 28974 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:18.979032 22902 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:18.880788 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:18.932176 29032 raft_consensus.cc:3055] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:19.046033 29032 raft_consensus.cc:740] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:18.981606 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:18.897823 28906 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:18.971702 28823 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.972070 28773 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:18.970099 29008 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:18.985457 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:18.985777 29010 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:18.996938 22824 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:09:19.005261 22898 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
W20260709 13:09:19.009039 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 112445us
I20260709 13:09:18.907562 29002 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:09:19.014046 22900 raft_consensus.cc:2368] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15.
I20260709 13:09:19.021224 22822 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:18.906455 28983 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:18.932286 28980 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.046126 22903 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
W20260709 13:09:19.523947 22829 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.000s	sys 0.013s
W20260709 13:09:19.049775 29048 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:09:19.524423 22829 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.000s	sys 0.013s
I20260709 13:09:19.525395 28983 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:18.980633 28991 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:19.526067 28991 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.351575 28963 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:19.438143 29051 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:09:19.529040 29031 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:19.530481 23450 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:19.143309 22831 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
W20260709 13:09:19.533525 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 512474us
I20260709 13:09:19.535327 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:19.536518 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:19.333041 29043 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:19.528079 28963 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.528277 28994 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:19.169304 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:19.241457 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:19.540019 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:18.993950 28849 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:19.525437 29032 consensus_queue.cc:260] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:19.526086 22901 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:19.449009 22830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.011s	sys 0.051s
I20260709 13:09:19.236790 28696 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:19.460456 23448 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:19.529943 29031 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.564348 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.555881 29053 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:19.567662 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.532711 28563 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:19.570533 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:19.049023 29044 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:09:19.698523 28696 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:19.699174 28696 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.577010 22904 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:09:19.701784 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.701817 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.702575 22830 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:19.702970 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.703464 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.703791 28975 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.704048 22830 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:19.704378 28975 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:19.709899 29019 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:19.579413 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.710852 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.572839 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:19.711462 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:19.711997 22822 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:19.712216 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:19.712387 28990 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:19.714354 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.714722 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.714860 29030 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:19.714998 23444 raft_consensus.cc:2368] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16.
I20260709 13:09:19.715245 29030 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:09:19.716099 29047 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.729s	user 0.006s	sys 0.000s
I20260709 13:09:19.716313 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.716667 28925 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:19.717111 28925 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:19.717979 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.717864 28925 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.718206 29031 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.718820 28925 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:19.577340 22903 pending_rounds.cc:85] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 12
I20260709 13:09:19.581403 22899 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:19.579447 29046 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:19.582245 22828 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:19.684841 29053 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:19.721868 29053 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:19.722702 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2041) took 1190 ms. Trace:
I20260709 13:09:19.723052 22900 rpcz_store.cc:276] 0709 13:09:18.532343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.538575 (+  6232us) service_pool.cc:224] Handling call
0709 13:09:19.722510 (+1183935us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b540034c1d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:19.722632 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091409792}
I20260709 13:09:19.724628 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1464) took 1044 ms. Trace:
W20260709 13:09:19.725454 29056 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.480s	user 0.011s	sys 0.021s
I20260709 13:09:19.649923 28994 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.726425 28918 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.727228 28994 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.716332 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1240) took 1021 ms. Trace:
I20260709 13:09:19.727680 22828 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:19.580713 22829 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:19.528159 28991 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.728561 28986 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.729244 28991 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:19.729660 28991 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:19.730520 28991 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:19.046574 29026 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:19.731596 29026 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:19.625737 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.727895 22826 rpcz_store.cc:276] 0709 13:09:18.694461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.704259 (+  9798us) service_pool.cc:224] Handling call
0709 13:09:18.763750 (+ 59491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:18.785785 (+ 22035us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:18.785818 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:18.790350 (+  4532us) log.cc:844] Serialized 52 byte log entry
0709 13:09:19.029669 (+239319us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:19.029807 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.030257 (+   450us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:19.716150 (+685893us) raft_consensus.cc:1685] finished
0709 13:09:19.716168 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.716285 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31852,"raft.queue_time_us":14734,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7370,"thread_start_us":162873,"threads_started":1,"wal-append.queue_time_us":757897}
I20260709 13:09:19.697911 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.054414 28982 raft_consensus.cc:697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:19.704902 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.703502 22825 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:19.519730 29003 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:19.515782 28974 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:19.705662 28849 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:19.736285 28974 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:19.716086 29030 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.720328 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1462) took 1040 ms. Trace:
I20260709 13:09:19.736959 29030 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:19.702540 28563 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.737118 22899 rpcz_store.cc:276] 0709 13:09:18.680073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.719618 (+1039545us) service_pool.cc:224] Handling call
0709 13:09:19.719890 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.720043 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.720065 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.720105 (+    40us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:19.720136 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.720186 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.720280 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:19.738929 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1465) took 1058 ms. Trace:
I20260709 13:09:19.581939 22747 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 17: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:19.739650 28563 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:19.035989 29036 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:19.740249 29036 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:19.740514 22747 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:19.741122 29036 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:19.741855 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.742115 29036 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:19.743820 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:19.744534 28892 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:19.701226 28696 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.736660 28849 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.745911 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.577998 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.716416 22901 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:19.720979 22903 pending_rounds.cc:107] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:09:19.732486 29026 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.19, Last appended by leader: 19, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.734110 28982 consensus_queue.cc:237] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:19.050870 29049 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.561528 29008 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.866532 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.716516 29047 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:19.767455 22822 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
W20260709 13:09:19.774757 22869 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.014s	sys 0.035s
I20260709 13:09:19.962185 23447 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:19.891736 22903 raft_consensus.cc:2889] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:19.752175 29062 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:09:19.598269 29043 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.862974 28963 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.883390 28974 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.744803 22747 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:19.745941 28892 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:09:19.785379 28878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.006s	sys 0.032s
I20260709 13:09:19.739230 22899 rpcz_store.cc:276] 0709 13:09:18.680877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.738228 (+1057351us) service_pool.cc:224] Handling call
0709 13:09:19.738489 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.738654 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.738678 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.738717 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:19.738747 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.738795 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.738882 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:19.911732 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:19.912858 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.747679 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:19.967185 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 219557us
I20260709 13:09:19.968376 29007 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.968822 29018 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:19.968875 29007 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:19.969497 29018 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:19.969604 28892 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:19.970467 29018 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 14.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.970574 28892 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.971133 29007 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:19.724957 22900 rpcz_store.cc:276] 0709 13:09:18.680601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.723927 (+1043326us) service_pool.cc:224] Handling call
0709 13:09:19.724181 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.724340 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.724365 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.724402 (+    37us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:09:19.724439 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.724487 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.724580 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:19.975034 28878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.006s	sys 0.045s
I20260709 13:09:19.917145 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.918277 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:19.976294 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:19.976760 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1469) took 1266 ms. Trace:
W20260709 13:09:19.937822 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 363005us
I20260709 13:09:19.977073 22901 rpcz_store.cc:276] 0709 13:09:18.710418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.976079 (+1265661us) service_pool.cc:224] Handling call
0709 13:09:19.976335 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.976491 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.976515 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.976550 (+    35us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:19.976584 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.976630 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.976715 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:19.978108 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:19.978231 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:19.978449 29043 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.978837 28878 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:19.979202 28878 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:19.979543 28878 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:19.979749 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1470) took 1233 ms. Trace:
I20260709 13:09:19.980114 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:19.980070 22901 rpcz_store.cc:276] 0709 13:09:18.746617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.979081 (+1232464us) service_pool.cc:224] Handling call
0709 13:09:19.979306 (+   225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.979477 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:19.979501 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.979536 (+    35us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:19.979569 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.979614 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.979705 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:19.938019 22868 leader_election.cc:400] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16.
I20260709 13:09:19.941275 28773 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:19.981961 28963 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:19.983202 28963 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:19.983582 28992 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:19.861616 28906 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.984476 28963 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:19.984838 28963 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:19.985558 28974 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.981583 22901 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:19.986690 28627 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:19.986974 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1471) took 1221 ms. Trace:
I20260709 13:09:19.987342 22901 rpcz_store.cc:276] 0709 13:09:18.765144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.981051 (+1215907us) service_pool.cc:224] Handling call
0709 13:09:19.981300 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.986762 (+  5462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.986927 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:19.859596 28986 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.981549 22868 leader_election.cc:403] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:09:19.988461 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 50568.8us
I20260709 13:09:19.989045 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1467) took 1307 ms. Trace:
I20260709 13:09:19.987555 23451 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:19.990595 22901 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 21. Current term is 22. Ops: [21.19-21.19]
I20260709 13:09:19.990855 22747 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:09:19.991196 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2045) took 1160 ms. Trace:
I20260709 13:09:19.991518 22901 rpcz_store.cc:276] 0709 13:09:18.831246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.990146 (+1158900us) service_pool.cc:224] Handling call
0709 13:09:19.990415 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:19.990953 (+   538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.991142 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:19.989753 22899 rpcz_store.cc:276] 0709 13:09:18.689158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.988388 (+1299230us) service_pool.cc:224] Handling call
0709 13:09:19.988633 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.988793 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.988814 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.988849 (+    35us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:19.988876 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.988921 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.989000 (+    79us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:19.992794 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2049) took 1160 ms. Trace:
I20260709 13:09:19.993127 22899 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: [15.8-16.9]
W20260709 13:09:19.961624 29057 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.440s	user 0.000s	sys 0.001s
I20260709 13:09:19.990110 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:19.999475 22823 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:19.993119 22901 rpcz_store.cc:276] 0709 13:09:18.832079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.992142 (+1160063us) service_pool.cc:224] Handling call
0709 13:09:19.992376 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.992527 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.992550 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.992587 (+    37us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:09:19.992621 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.992668 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.992754 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:19.997033 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:19.998347 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:19.985179 28963 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:20.005851 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2051) took 1162 ms. Trace:
I20260709 13:09:20.000578 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:20.007114 22822 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:20.006551 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:20.008697 22901 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:09:20.008741 22747 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:09:20.009757 22899 rpcz_store.cc:276] 0709 13:09:18.833022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.992641 (+1159619us) service_pool.cc:224] Handling call
0709 13:09:19.992914 (+   273us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:19.993955 (+  1041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.994902 (+   947us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:20.022380 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2052) took 1189 ms. Trace:
I20260709 13:09:20.022923 22899 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:09:20.010558 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.022696 22901 rpcz_store.cc:276] 0709 13:09:18.833351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.008202 (+1174851us) service_pool.cc:224] Handling call
0709 13:09:20.008443 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:20.022146 (+ 13703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.022336 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:20.024531 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2053) took 1170 ms. Trace:
I20260709 13:09:20.023809 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:20.024840 22899 rpcz_store.cc:276] 0709 13:09:18.853681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.022459 (+1168778us) service_pool.cc:224] Handling call
0709 13:09:20.022729 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:20.024301 (+  1572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.024485 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:19.961664 22869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.014s	sys 0.042s
I20260709 13:09:20.027498 29032 raft_consensus.cc:3060] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:20.036455 28994 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:20.037156 28994 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.037575 28994 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:20.037598 29053 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.039876 29053 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.036461 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2838) took 1220 ms. Trace:
I20260709 13:09:19.939102 29049 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:20.041172 28974 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:20.042346 29069 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:09:20.044065 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:20.045044 29031 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:20.045545 29031 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:20.040940 22753 rpcz_store.cc:276] 0709 13:09:18.815592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.817301 (+  1709us) service_pool.cc:224] Handling call
0709 13:09:18.817597 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:18.840401 (+ 22804us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:18.894458 (+ 54057us) log.cc:844] Serialized 14 byte log entry
0709 13:09:19.026887 (+132429us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:19.027415 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:09:19.027569 (+   154us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:19.027612 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.027704 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:20.036259 (+1008555us) raft_consensus.cc:1685] finished
0709 13:09:20.036284 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.036406 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":276420,"prepare.queue_time_us":49103,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":4616,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":41945,"thread_start_us":131968,"threads_started":1,"wal-append.queue_time_us":626646}
I20260709 13:09:20.046351 29031 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:19.980085 29047 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:19.999933 29036 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:09:20.010622 29052 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.638s	user 0.009s	sys 0.015s
I20260709 13:09:20.072238 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2840) took 1256 ms. Trace:
I20260709 13:09:20.075572 22753 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:20.052379 22829 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 10.
I20260709 13:09:19.940836 29046 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:20.088464 22899 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:09:20.123576 22750 rpcz_store.cc:276] 0709 13:09:18.816095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.816212 (+   117us) service_pool.cc:224] Handling call
0709 13:09:18.826503 (+ 10291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.001667 (+175164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.071967 (+1070300us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540011c1d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:20.072172 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401671552}
W20260709 13:09:20.088923 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88473.3us
I20260709 13:09:20.125025 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:20.125133 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 157138us
I20260709 13:09:20.007903 28978 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:20.139488 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:20.120371 29064 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.131s	user 0.000s	sys 0.003s
I20260709 13:09:20.122756 28918 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:20.145823 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.009810 28986 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.152357 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.153100 22753 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:20.145910 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.071257 28849 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:09:20.050487 29030 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.092s
I20260709 13:09:20.152446 22747 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:20.142791 28978 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:20.149551 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:20.151805 22750 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:20.159667 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2755) took 1468 ms. Trace:
I20260709 13:09:20.201622 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:20.172415 28963 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.173651 29008 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.174482 29024 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.174763 28773 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:20.175900 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:20.176595 28980 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:20.176822 22831 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2166) took 1415 ms (client timeout 1781 ms). Trace:
I20260709 13:09:20.227298 22747 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:20.183120 22902 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:09:20.227597 22831 rpcz_store.cc:269] 0709 13:09:18.761490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.783342 (+ 21852us) service_pool.cc:224] Handling call
0709 13:09:20.176624 (+1393282us) spinlock_profiling.cc:243] Waited 92.8 ms on lock 0x7b54002c9054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:20.176747 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213548416}
I20260709 13:09:20.186892 28823 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.189476 29046 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.190265 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:20.192896 28665 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:20.195734 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:20.199625 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.202636 22750 raft_consensus.cc:2368] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11.
W20260709 13:09:20.206346 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56888.6us
W20260709 13:09:20.227097 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51286.9us
I20260709 13:09:20.184901 22825 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:20.182981 22905 raft_consensus.cc:1275] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:20.238883 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:20.239382 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2754) took 1569 ms. Trace:
I20260709 13:09:20.201062 22900 rpcz_store.cc:276] 0709 13:09:18.691187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.973674 (+1282487us) service_pool.cc:224] Handling call
0709 13:09:19.973990 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:19.974335 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.974367 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:19.974786 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 13:09:20.142188 (+167402us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:20.142234 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.142310 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:20.159483 (+ 17173us) raft_consensus.cc:1685] finished
0709 13:09:20.159501 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.159612 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2099,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"thread_start_us":167047,"threads_started":1,"wal-append.queue_time_us":166363}
I20260709 13:09:20.208455 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.205699 29000 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:20.239806 22905 rpcz_store.cc:276] 0709 13:09:18.669651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.182385 (+1512734us) service_pool.cc:224] Handling call
0709 13:09:20.182667 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:20.239177 (+ 56510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.239337 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:20.241612 22748 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:20.244793 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.246076 22905 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:20.246485 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.249450 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.250495 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.250932 22896 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:20.197396 28918 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:20.255813 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:20.256356 22904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2040) took 1725 ms (client timeout 1934 ms). Trace:
I20260709 13:09:20.256822 22900 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:20.244498 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1645) took 1331 ms. Trace:
I20260709 13:09:20.245615 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.258595 22830 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:09:20.258955 22822 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:20.196372 29061 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:20.253827 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2844) took 1245 ms. Trace:
I20260709 13:09:20.263455 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.257498 23453 rpcz_store.cc:276] 0709 13:09:18.912814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.912937 (+   123us) service_pool.cc:224] Handling call
0709 13:09:18.967335 (+ 54398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.912474 (+945139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.912507 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.912545 (+    38us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:19.912574 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.244299 (+331725us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b54006b08d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:20.244343 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.244445 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113884032}
I20260709 13:09:20.265003 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:20.265208 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:20.256750 22904 rpcz_store.cc:269] 0709 13:09:18.531292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.531391 (+    99us) service_pool.cc:224] Handling call
0709 13:09:20.256224 (+1724833us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400184a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:20.256294 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2402858240}
I20260709 13:09:20.265434 22748 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:09:20.265549 22822 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:20.253412 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 129521us
I20260709 13:09:20.255004 28906 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.267563 28906 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.228374 28773 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.264201 22750 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:09:20.272652 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:20.268447 22830 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:20.268993 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:20.269181 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:20.274173 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:20.276840 22828 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:20.278291 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:20.279017 22831 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:20.205828 29015 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:20.226824 28992 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:20.263839 22756 rpcz_store.cc:276] 0709 13:09:19.008597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.208618 (+1200021us) service_pool.cc:224] Handling call
0709 13:09:20.253377 (+ 44759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:20.253527 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.253563 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:20.253599 (+    36us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:20.253627 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.253674 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.253790 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:20.242654 28839 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:20.284384 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:20.284668 28963 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.284889 22829 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:20.255201 29014 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.286886 29014 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.288271 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:20.288849 22826 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
W20260709 13:09:20.262040 22719 connection.cc:256] RPC call timeout handler was delayed by -1.0968076919999703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
I20260709 13:09:20.273032 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1861) took 1295 ms. Trace:
I20260709 13:09:20.271864 28918 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.293632 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.283298 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:20.280304 28773 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:20.237576 29007 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:09:20.252245 29008 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:20.301514 29008 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.226608 29024 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.303654 29008 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.304150 29008 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:20.290482 22752 rpcz_store.cc:276] 0709 13:09:18.978042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.007698 (+ 29656us) service_pool.cc:224] Handling call
0709 13:09:19.452113 (+444415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.706601 (+254488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.706646 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.706703 (+    57us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:19.706748 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.272851 (+566103us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54002689d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:20.272889 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.272989 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584692608}
I20260709 13:09:20.305073 29024 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.305790 29024 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:20.305907 22904 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:09:20.309286 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:20.312420 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:20.314702 29072 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:09:20.315081 28953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a127a20cfa4496bad51cb634bd0d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:20.316087 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:20.291366 22822 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:09:20.318153 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.318493 22896 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:20.248335 29034 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:20.295032 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.283547 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.252501 28696 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:20.282121 28653 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:20.314105 28978 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.319293 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.328755 22828 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:20.292510 22829 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:20.333426 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.334040 29014 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:20.334132 22900 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:20.312976 28994 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.336412 28986 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.330952 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:20.338793 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.284682 28839 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.339879 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.255331 29000 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.304610 29008 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.299664 29022 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:20.255563 29067 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:09:20.317365 29031 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:20.300335 28992 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:20.321738 28823 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.330022 28653 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:20.329741 28627 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:20.337016 28847 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:20.281543 28878 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.360190 28974 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:20.337170 28990 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.329514 28963 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:20.340022 29047 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:20.352617 22896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:20.351912 28839 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:20.364442 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 120890us
I20260709 13:09:20.396345 22896 raft_consensus.cc:606] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:20.295657 29015 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.406179 22753 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:20.340183 28696 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.385564 28653 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:20.397679 28990 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:20.423887 29054 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 1.174s	user 0.022s	sys 0.011s
I20260709 13:09:20.424070 28918 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:20.385874 28627 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.392974 28974 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.362591 29053 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.044s
W20260709 13:09:20.482143 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 235728us
I20260709 13:09:20.385191 28994 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.434864 29083 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:20.438521 29036 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16.
I20260709 13:09:20.338500 29034 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.442353 29008 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:20.446511 29000 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.450407 28878 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.451035 28992 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.457026 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1460) took 1777 ms. Trace:
W20260709 13:09:20.457231 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 183130us
I20260709 13:09:20.458520 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1862) took 1213 ms. Trace:
I20260709 13:09:20.467422 28906 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.461094 28745 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:09:20.510639 22754 rpcz_store.cc:276] 0709 13:09:19.245139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.713141 (+468002us) service_pool.cc:224] Handling call
0709 13:09:19.713410 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.977679 (+264269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.977708 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.977748 (+    40us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:19.977777 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.458314 (+480537us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b5400045e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:20.458358 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.458467 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607376128}
I20260709 13:09:20.472227 28696 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.474450 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1456) took 1826 ms. Trace:
I20260709 13:09:20.482846 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.488444 22896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:20.392735 28823 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.376741 29031 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.480150 28953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38064 (request call id 1556) took 3771 ms. Trace:
W20260709 13:09:20.493343 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 257178us
W20260709 13:09:20.523128 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 171977us
I20260709 13:09:20.482651 28974 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:20.480767 28627 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:20.498577 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:20.534238 23444 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:20.500087 23448 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:20.511009 22871 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.516203 22902 rpcz_store.cc:276] 0709 13:09:18.647536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.572105 (+924569us) service_pool.cc:224] Handling call
0709 13:09:19.572459 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:20.227861 (+655402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.474176 (+246315us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b54003bf6d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:20.474385 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1403406720}
I20260709 13:09:20.519737 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.520550 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.510187 22903 rpcz_store.cc:276] 0709 13:09:18.679575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.572727 (+893152us) service_pool.cc:224] Handling call
0709 13:09:19.577103 (+  4376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:19.990318 (+413215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:19.990347 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:20.008515 (+ 18168us) log.cc:844] Serialized 26 byte log entry
0709 13:09:20.203024 (+194509us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:20.203066 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.203156 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:20.456841 (+253685us) raft_consensus.cc:1685] finished
0709 13:09:20.456862 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.456974 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":52577,"threads_started":1,"wal-append.queue_time_us":281856}
I20260709 13:09:20.521942 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.523011 23453 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:20.524458 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.521064 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:20.493865 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 177015us
I20260709 13:09:20.495069 28994 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:20.533686 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.538851 22896 raft_consensus.cc:606] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:20.539223 22896 raft_consensus.cc:614] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:20.539378 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.540902 22899 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:20.546490 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:20.546561 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 207833us
I20260709 13:09:20.550489 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.530160 28696 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:20.552336 29046 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:09:20.576283 22871 leader_election.cc:400] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11.
I20260709 13:09:20.576444 29047 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:20.576524 28918 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:20.576617 22871 leader_election.cc:403] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:09:20.585667 28745 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:20.587517 28745 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.587586 28990 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.588207 28745 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.577162 29047 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:20.590335 29047 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.591665 29047 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:20.596029 28975 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:20.596562 28975 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.597503 28839 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.598484 28975 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.599434 28839 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.566216 29049 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:20.601377 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:20.602702 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56269.8us
I20260709 13:09:20.621356 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.551093 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:20.551349 29032 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:20.567051 28906 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:20.630439 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.627864 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.630571 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.569106 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:20.629062 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:20.633909 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.634734 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.581656 28847 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:20.640242 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.642212 22905 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [19.14-20.15]
I20260709 13:09:20.643445 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.554519 28878 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:09:20.650308 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.643466 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:20.646875 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 77852.8us
I20260709 13:09:20.573906 29022 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:09:20.626799 29049 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.621851 28696 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:20.651451 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.647224 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.651965 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.659515 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.660152 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:20.652436 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 104586us
I20260709 13:09:20.660786 22904 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:20.655432 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.661012 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.660243 22905 raft_consensus.cc:2376] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24. Re-sending same reply.
I20260709 13:09:20.661873 22903 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
W20260709 13:09:20.663550 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 111112us
I20260709 13:09:20.662910 22905 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:09:20.651717 29018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.009s	sys 0.005s
I20260709 13:09:20.668839 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:20.671574 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46574.8us
I20260709 13:09:20.672757 29085 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:20.673454 29034 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.657598 29049 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:20.657856 29007 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:20.653090 28839 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:09:20.674206 29104 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:20.715479 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.529975 28953 rpcz_store.cc:276] 0709 13:09:16.713670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.720222 (+  6552us) service_pool.cc:224] Handling call
0709 13:09:16.890976 (+170754us) ts_tablet_manager.cc:934] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
0709 13:09:20.252213 (+3361237us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:20.452036 (+199823us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:20.457902 (+  5866us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:20.479396 (+ 21494us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:20.479699 (+   303us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:20.480000 (+   301us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:20.480057 (+    57us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":369,"tablet-copy.queue_time_us":46902,"thread_start_us":313213,"threads_started":2}
I20260709 13:09:20.678067 28847 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:20.679375 29032 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:20.680560 28992 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:20.680984 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.682332 29053 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:20.655977 28975 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:20.667200 28878 consensus_queue.cc:1243] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:09:20.573994 29067 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:09:20.686609 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:20.687587 22827 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:09:20.687529 28978 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:20.688508 28773 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.743446 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1459) took 2064 ms. Trace:
I20260709 13:09:20.744354 28773 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.689797 29031 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.689699 29100 consensus_queue.cc:1048] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:20.690928 22797 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:20.695348 22901 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:20.743829 22898 rpcz_store.cc:276] 0709 13:09:18.679334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.197912 (+1518578us) service_pool.cc:224] Handling call
0709 13:09:20.229525 (+ 31613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:20.387149 (+157624us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:09:20.533298 (+146149us) log.cc:844] Serialized 14 byte log entry
0709 13:09:20.648475 (+115177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:20.648521 (+    46us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:20.648563 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.743240 (+ 94677us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54001b6054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:20.743282 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.743394 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32593,"prepare.queue_time_us":186927,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":47443,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":10812,"spinlock_wait_cycles":361794432,"thread_start_us":213558,"threads_started":2}
I20260709 13:09:20.698302 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:20.698925 23444 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
W20260709 13:09:20.749028 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50804.1us
I20260709 13:09:20.701582 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:20.697508 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:20.656284 28906 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:09:20.697940 29018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.009s	sys 0.005s
I20260709 13:09:20.697575 22903 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 7 to 11, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:20.719950 29104 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:20.738384 29022 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:09:20.759004 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 68147.9us
I20260709 13:09:20.759312 22822 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:09:20.734272 28975 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:20.725103 29032 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:20.734151 28992 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.740648 28953 tablet_copy_client.cc:806] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:20.742161 22904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:20.756824 28878 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:09:20.833359 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.818490 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.719374 28994 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:20.749400 28563 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.756168 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:20.758455 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.759780 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.792315 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:20.714164 29047 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:20.725776 29007 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:20.730190 29085 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:20.797432 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 99976.5us
I20260709 13:09:20.800701 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.869861 22896 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:20.816308 29024 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:20.873816 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.834023 28839 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.878198 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.823056 28986 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:20.813553 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.816150 22904 raft_consensus.cc:606] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Received request to transfer leadership to TS c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.885061 22904 raft_consensus.cc:614] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:20.885710 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:20.886085 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.886648 22904 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:20.742374 29014 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:20.759119 29022 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:20.888162 29014 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.888590 29022 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.889734 29022 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:20.891142 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:20.893887 29014 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.834399 22897 raft_consensus.cc:2376] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17. Re-sending same reply.
I20260709 13:09:20.895087 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.833667 28878 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:20.815881 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.758981 29104 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:20.799295 28975 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:20.865154 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 105432us
I20260709 13:09:20.797278 28990 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:20.864459 22827 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:20.815557 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.796888 28745 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.827586 28992 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:20.801671 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.819139 29061 raft_consensus.cc:993] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:20.803725 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.827997 28953 tablet_copy_client.cc:670] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:20.822642 29068 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:20.806674 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:20.881584 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.912572 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:20.912642 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 117290us
I20260709 13:09:20.818213 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:20.887676 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.814277 29053 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 20.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.060s
I20260709 13:09:20.897190 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.883368 28823 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.826004 29046 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:09:20.902827 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:20.873733 29024 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:20.866775 28974 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.909371 22901 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:20.909720 22900 raft_consensus.cc:2376] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17. Re-sending same reply.
I20260709 13:09:20.925406 22900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84ea43361e534fbebad704327c9f6367"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:20.925710 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:20.925922 22900 raft_consensus.cc:606] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:20.926256 22900 raft_consensus.cc:614] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:20.884711 28986 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:20.820711 29087 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:20.866855 29047 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:20.912669 22905 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 13: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:20.915735 28773 raft_consensus.cc:1081] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:20.904233 28878 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:20.916322 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.821161 29032 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.869166 28653 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.871622 28563 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.896163 28839 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.970291 22898 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:20.971518 28839 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.926267 22830 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:20.933928 22826 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:20.935971 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:20.936115 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 53859.3us
I20260709 13:09:20.973878 22822 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:20.938200 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.939479 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:20.979041 29103 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:20.983191 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.983963 22749 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:20.986133 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.988052 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:20.990864 22797 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:20.992345 22756 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:21.001863 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:21.003217 29018 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:21.003506 22822 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:21.005892 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:20.941642 22901 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:20.942674 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:21.007866 22905 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:09:20.932320 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:20.947145 29024 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.822664 28906 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.951479 29087 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:21.009727 29087 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:21.009907 29019 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:20.949656 28974 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.950070 28986 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:20.910058 29022 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.953022 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:20.960494 22827 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:09:20.908878 29091 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:21.013793 29031 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:21.014186 29031 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.014082 29022 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.014604 29031 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:21.016160 29022 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:20.960170 29047 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:21.017630 22905 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:21.018698 22905 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:20.912241 29068 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:21.022235 29068 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.023583 29068 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:20.964916 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:21.026363 29112 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:21.027084 29112 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:21.028198 29112 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.029598 28992 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:21.030031 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:21.030808 28892 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:21.031169 28892 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:21.031561 28892 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:21.031889 23416 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:21.031992 29019 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:21.030200 28992 consensus_queue.cc:1243] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:09:21.033339 29019 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:09:21.036005 29097 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:09:21.039809 28892 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.040028 29104 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.046190 28892 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:21.044451 29104 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:21.049360 29117 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:21.049610 29031 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:20.929050 29067 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:09:21.053061 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 120916us
I20260709 13:09:21.061420 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:20.928681 28990 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:21.063234 22825 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:21.074117 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:21.074314 23445 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:21.061589 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:21.074698 22905 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:21.075114 22756 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:21.076216 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:21.097940 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 158745us
I20260709 13:09:20.929713 29049 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:21.103538 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:21.066058 22754 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:09:21.109958 22754 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:21.110477 22754 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:21.114740 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:21.065017 29000 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:21.134699 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:21.126586 22748 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:21.136204 22748 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:21.127468 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:21.128038 29121 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:21.095705 28892 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:09:21.129259 29087 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.075700 28991 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:21.136924 22748 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 21.19, Last appended by leader: 19, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.137972 22905 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:21.095711 29032 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:21.103319 28849 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:09:21.154150 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 163379us
I20260709 13:09:21.147135 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:21.141857 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:21.111196 29024 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:21.149434 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:21.156836 22905 raft_consensus.cc:2368] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16.
I20260709 13:09:21.159749 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:20.951792 28994 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.164549 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:21.073598 28923 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 13:09:21.169224 22748 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:21.133630 29067 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:21.101635 29014 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 13:09:21.172570 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:21.173477 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:21.148447 29100 raft_consensus.cc:2764] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:09:21.095696 28990 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.050863 29007 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:21.174316 29067 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.170212 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:21.150488 29121 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:21.207819 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:21.210726 29067 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:21.177559 28918 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.219010 22748 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 21 index: 19. Preceding OpId from leader: term: 22 index: 20. (index mismatch)
I20260709 13:09:21.163523 29110 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:21.212931 22754 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:21.222952 22903 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 20. (index mismatch)
W20260709 13:09:21.498013 28627 consensus_meta.cc:287] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.945s	user 0.011s	sys 0.000s
I20260709 13:09:21.254233 28918 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.519796 28918 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:21.329550 23446 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:09:21.254959 29126 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:21.306634 29121 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:21.259935 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:21.524302 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 264406us
W20260709 13:09:21.524552 28878 consensus_meta.cc:287] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.517s	user 0.004s	sys 0.004s
I20260709 13:09:21.525184 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:21.525956 29127 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:09:21.526235 23416 leader_election.cc:400] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16.
I20260709 13:09:21.258167 29087 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:21.338064 28696 consensus_meta.cc:287] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.000s
W20260709 13:09:21.458181 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:21.542011 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 381406us
I20260709 13:09:21.262310 29084 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:21.256381 22754 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:21.333046 29131 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:21.328336 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:21.308861 22905 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
W20260709 13:09:21.346244 29129 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.221s	user 0.002s	sys 0.000s
I20260709 13:09:21.261406 28759 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:21.317718 29047 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.527331 29110 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.324085 29031 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:21.319137 28653 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:21.265987 29122 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:09:21.209472 29022 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.095115 29085 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:21.331583 29015 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:21.322651 28990 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:21.492455 22825 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 13:09:21.563138 29030 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.495137 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:21.254843 29130 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:09:21.394661 29091 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:21.495082 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:21.321877 28839 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:21.510426 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:21.253998 28963 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 13:09:21.329674 23452 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:21.339450 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1477) took 1094 ms. Trace:
W20260709 13:09:21.316334 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.330308 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:21.482860 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 274074us
W20260709 13:09:21.554247 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 346482us
I20260709 13:09:21.557693 22903 pending_rounds.cc:85] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 13
I20260709 13:09:21.525027 28878 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:21.298483 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.573671 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:21.552034 28696 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11.
I20260709 13:09:21.581986 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:21.493798 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:21.583379 22755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.002s	sys 0.012s
I20260709 13:09:21.562439 28759 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:21.539556 28627 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:21.667120 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.584455 29112 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:21.668107 22903 pending_rounds.cc:107] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:09:21.726262 22903 raft_consensus.cc:2889] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:21.698819 22755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.002s	sys 0.015s
W20260709 13:09:21.677378 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 150308us
W20260709 13:09:21.600472 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 105445us
I20260709 13:09:21.727885 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:21.714973 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:21.670797 22747 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 20: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:21.717347 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:21.717247 22752 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:21.667012 22899 rpcz_store.cc:276] 0709 13:09:20.245044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.245403 (+   359us) service_pool.cc:224] Handling call
0709 13:09:21.339243 (+1093840us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540031f954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:21.339383 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582863616}
I20260709 13:09:21.722601 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:21.732237 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:21.734009 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:21.671248 29122 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:09:21.735024 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:21.593295 29049 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:21.598734 28653 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:21.741729 22756 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:21.584303 29022 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.742012 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:21.742623 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:21.741747 23451 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:09:21.745190 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:21.575409 29126 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.141s
W20260709 13:09:21.668082 29125 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:09:21.764315 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:21.761621 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:21.765952 22899 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 24.
I20260709 13:09:21.779639 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:21.724457 28839 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:21.598587 29030 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:21.852947 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 88758.7us
I20260709 13:09:21.744879 28696 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.745149 29061 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:21.882717 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2769) took 1067 ms. Trace:
I20260709 13:09:21.759210 29026 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.870663 23452 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:21.883241 22904 rpcz_store.cc:276] 0709 13:09:20.815043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.963215 (+148172us) service_pool.cc:224] Handling call
0709 13:09:21.012654 (+ 49439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:21.012978 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:21.013003 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:21.031402 (+ 18399us) log.cc:844] Serialized 26 byte log entry
0709 13:09:21.100725 (+ 69323us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:21.447992 (+347267us) log.cc:844] Serialized 14 byte log entry
0709 13:09:21.448209 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:21.448281 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:21.882458 (+434177us) raft_consensus.cc:1685] finished
0709 13:09:21.882486 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:21.882647 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29931,"prepare.queue_time_us":206991,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":59554,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":13901,"thread_start_us":385928,"threads_started":2,"wal-append.queue_time_us":93607}
I20260709 13:09:21.779006 28982 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:21.858636 28990 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:21.963354 22902 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:09:21.855877 29049 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:21.855540 29126 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:09:21.985100 29152 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:09:21.985266 29102 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:09:21.952867 29122 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:09:22.015213 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:22.015826 22904 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:21.974956 29024 consensus_meta.cc:287] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.805s	user 0.009s	sys 0.000s
I20260709 13:09:22.031598 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 1111) took 1534 ms. Trace:
I20260709 13:09:21.880208 28696 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.032922 22897 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:21.974601 28990 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:22.031984 23445 rpcz_store.cc:276] 0709 13:09:20.496812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.496943 (+   131us) service_pool.cc:224] Handling call
0709 13:09:20.520106 (+ 23163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:21.094985 (+574879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.031198 (+936213us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54000b2cd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:22.031527 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274100992}
W20260709 13:09:22.041779 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 183171us
I20260709 13:09:22.043356 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:22.005401 28653 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:09:22.049094 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.944715 29026 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.045256 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:22.050457 29126 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:22.063038 22899 raft_consensus.cc:2410] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:09:22.040513 28848 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.065585 22904 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:22.046471 29015 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.076436 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:22.058480 29139 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:22.067761 28982 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:22.068746 29126 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:22.082970 29034 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:22.050431 28839 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.096796 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:22.078042 28990 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.099924 29166 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:22.105724 22749 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
W20260709 13:09:22.101698 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.077997 29146 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:22.102233 29022 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:22.102020 28759 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:22.113497 29134 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 13:09:22.103313 28696 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:22.071554 28848 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.126199 29120 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:22.126570 29120 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:22.127030 28848 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:22.086056 28878 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:22.091529 29032 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.928s	user 0.005s	sys 0.004s
I20260709 13:09:22.128273 28849 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:22.128650 28849 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:22.129405 28849 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.130462 28849 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:22.132624 22752 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:22.086506 29139 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.088416 28982 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:22.110162 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:22.074293 29015 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.090111 28627 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:22.122944 29068 consensus_meta.cc:287] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.099s	user 0.010s	sys 0.002s
I20260709 13:09:22.124181 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:22.216313 22749 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:22.142282 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:22.084761 29102 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:22.113646 28759 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:22.109596 29137 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:22.221695 22753 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:09:22.222213 22753 pending_rounds.cc:85] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 11
I20260709 13:09:22.222532 22753 pending_rounds.cc:107] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:09:22.177014 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2770) took 1361 ms. Trace:
I20260709 13:09:22.222826 22753 raft_consensus.cc:2889] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:22.183240 29026 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:22.185125 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 470197us
I20260709 13:09:22.224545 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:22.224730 22905 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:22.226682 22905 raft_consensus.cc:606] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:22.227048 22905 raft_consensus.cc:614] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:22.227622 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:22.222962 22897 rpcz_store.cc:276] 0709 13:09:20.815315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.907262 (+ 91947us) service_pool.cc:224] Handling call
0709 13:09:22.176756 (+1269494us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000d0a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:22.176933 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2462487296}
W20260709 13:09:22.189862 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 416971us
I20260709 13:09:21.942268 29061 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:21.558531 29131 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.235844 29131 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:22.114010 29146 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.236337 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:22.236908 23451 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:22.237542 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:22.205688 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.238339 23446 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:22.206156 29094 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:22.239251 29094 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:22.207231 29159 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:09:22.210119 29087 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:21.575701 29110 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.229624 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:22.174455 29032 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:22.192667 28982 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:22.233783 29154 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:09:22.237752 29146 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:22.242447 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:22.209929 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:22.193367 29067 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.902s	user 0.011s	sys 0.007s
I20260709 13:09:22.247028 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:22.231812 29171 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:22.247848 22897 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:09:22.188485 29139 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:22.242821 28994 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:22.251936 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:22.224679 28990 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:22.252395 29087 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.249854 29131 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:22.127317 29120 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.124697 29126 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.271274 29015 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:22.271377 29146 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:22.256762 22749 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
W20260709 13:09:22.273307 29151 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:09:22.249792 29137 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.274375 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2859) took 1292 ms. Trace:
I20260709 13:09:22.276059 29137 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:22.275038 22751 rpcz_store.cc:276] 0709 13:09:20.982406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.985001 (+  2595us) service_pool.cc:224] Handling call
0709 13:09:20.985413 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:21.401831 (+416418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:21.401877 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:21.402430 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:09:21.665831 (+263401us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:21.665869 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:21.665955 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.274117 (+608162us) raft_consensus.cc:1685] finished
0709 13:09:22.274142 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.274311 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65078,"raft.queue_time_us":142300,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":595953,"threads_started":2,"wal-append.queue_time_us":371934}
W20260709 13:09:22.280500 28991 consensus_meta.cc:287] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.126s	user 0.006s	sys 0.002s
I20260709 13:09:22.281582 28895 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.282034 28895 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.285022 28895 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:22.286315 29163 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:09:22.287961 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1497) took 1225 ms. Trace:
I20260709 13:09:22.288146 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1888) took 1622 ms. Trace:
I20260709 13:09:22.288570 22748 rpcz_store.cc:276] 0709 13:09:20.665793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.665891 (+    98us) service_pool.cc:224] Handling call
0709 13:09:20.666113 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:21.126482 (+460369us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b540003b220. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d72dc0 00007fcb74dfd1c0 00007fcb74dfe231 00007fcb74dfad8f 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c
0709 13:09:21.222068 (+ 95586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.287917 (+1065849us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400406ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:22.288085 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595976832}
I20260709 13:09:22.289824 29110 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:22.288434 22903 rpcz_store.cc:276] 0709 13:09:21.062347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.543202 (+480855us) service_pool.cc:224] Handling call
0709 13:09:21.557438 (+ 14236us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:21.779540 (+222102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:21.779574 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:21.780204 (+   630us) log.cc:844] Serialized 52 byte log entry
0709 13:09:22.030483 (+250279us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:22.055627 (+ 25144us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.209211 (+153584us) spinlock_profiling.cc:243] Waited 7.46 ms on lock 0x7b1800296380. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb787f33c7 00007fcb74e059a7 00007fcb74e150bf 00007fcb74e15038
0709 13:09:22.209519 (+   308us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.209762 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.209870 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.287667 (+ 77797us) raft_consensus.cc:1685] finished
0709 13:09:22.287694 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.287878 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":154964,"prepare.queue_time_us":196288,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":2204,"raft.queue_time_us":236878,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":22047,"spinlock_wait_cycles":19477760,"thread_start_us":87834,"threads_started":1,"wal-append.queue_time_us":236574}
I20260709 13:09:22.294358 29061 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:22.297178 29128 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.171s	user 0.000s	sys 0.002s
I20260709 13:09:22.256212 29032 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.299122 29032 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:22.299788 29120 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:22.300598 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2775) took 1444 ms. Trace:
I20260709 13:09:22.300945 22900 rpcz_store.cc:276] 0709 13:09:20.856178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.021692 (+165514us) service_pool.cc:224] Handling call
0709 13:09:21.022106 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:21.022549 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:21.022589 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:21.023226 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 13:09:21.100390 (+ 77164us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:21.100430 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:21.100491 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.296567 (+1196076us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:22.300312 (+  3745us) raft_consensus.cc:1685] finished
0709 13:09:22.300331 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.300522 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3451,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":138,"spinlock_wait_cycles":45911936,"thread_start_us":76719,"threads_started":1,"wal-append.queue_time_us":102342}
I20260709 13:09:22.303360 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1487) took 1428 ms. Trace:
I20260709 13:09:22.303702 22901 rpcz_store.cc:276] 0709 13:09:20.875047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.008903 (+133856us) service_pool.cc:224] Handling call
0709 13:09:21.341582 (+332679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:21.342074 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:21.342118 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:21.342172 (+    54us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:21.342215 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:21.342294 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.303209 (+960915us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b540031f950. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:22.303310 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5667,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":86612,"spinlock_wait_cycles":764384000}
I20260709 13:09:22.305462 29143 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:22.311039 28975 consensus_meta.cc:287] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 1.401s	user 0.009s	sys 0.000s
I20260709 13:09:22.311441 28975 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.315778 22750 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:22.315352 29102 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.319329 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:22.258140 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:22.317139 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1893) took 1261 ms. Trace:
I20260709 13:09:22.258905 22904 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:22.245042 29022 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.266134 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3039) took 1333 ms. Trace:
I20260709 13:09:22.112344 28653 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.268043 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:22.249436 29094 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:22.321216 22750 rpcz_store.cc:276] 0709 13:09:21.055702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.056771 (+  1069us) service_pool.cc:224] Handling call
0709 13:09:21.057004 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.316154 (+1259150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.316944 (+   790us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002aa154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:22.317088 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2891403136}
I20260709 13:09:22.325785 22828 rpcz_store.cc:276] 0709 13:09:20.932671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.932794 (+   123us) service_pool.cc:224] Handling call
0709 13:09:20.950717 (+ 17923us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:20.951258 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.951287 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:21.006870 (+ 55583us) log.cc:844] Serialized 26 byte log entry
0709 13:09:21.360050 (+353180us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:21.360104 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:21.360192 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.265943 (+905751us) raft_consensus.cc:1685] finished
0709 13:09:22.265961 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.266082 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2588,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":44255,"thread_start_us":251287,"threads_started":1,"wal-append.queue_time_us":505325}
I20260709 13:09:22.328948 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:22.329800 22748 raft_consensus.cc:2410] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 13:09:22.326864 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 58866.8us
W20260709 13:09:22.327730 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 69692.5us
I20260709 13:09:22.333825 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:22.334877 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:22.335714 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:22.337332 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:22.333930 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:22.341166 22750 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 23 because replica has last-logged OpId of term: 21 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 18.
I20260709 13:09:22.341080 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:22.268566 28665 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.341771 23453 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:22.345044 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:22.377022 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 45386.1us
I20260709 13:09:22.373639 28665 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.380669 28665 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:22.263790 29067 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.381541 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:22.383257 29067 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:22.383380 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:22.352205 29137 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:22.384702 29137 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:22.385151 29137 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:22.386080 29087 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:22.346149 29068 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 13:09:22.331974 29143 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:22.390609 22898 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:09:22.393565 22750 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:22.377772 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:22.333431 29131 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:22.354877 28991 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:22.270078 28994 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.346293 28745 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:22.351783 22904 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:22.354292 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:22.351761 28975 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:22.349963 28895 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:22.392686 29104 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.400157 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:22.432904 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 80084.6us
I20260709 13:09:22.429744 28773 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:22.470502 28773 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:22.471647 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 71578.8us
I20260709 13:09:22.475318 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:22.475582 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:22.474731 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:22.476532 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:22.477589 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:22.478296 22750 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:22.479089 22899 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:09:22.479936 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:22.403184 28665 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:22.332314 29022 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.390713 29121 raft_consensus.cc:2764] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
W20260709 13:09:22.447234 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.345978 29146 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 13:09:22.481918 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:22.482592 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:22.429587 29067 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:22.498816 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:22.501757 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:22.501873 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:22.506332 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:22.439703 29137 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.605687 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:22.604616 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2865) took 1620 ms. Trace:
I20260709 13:09:22.609836 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:22.610198 22903 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:22.609243 22753 rpcz_store.cc:276] 0709 13:09:20.984260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.987196 (+  2936us) service_pool.cc:224] Handling call
0709 13:09:20.987540 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.223488 (+1235948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.604154 (+380666us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54003ce6d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:22.604516 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2837074304}
I20260709 13:09:22.403515 29024 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 13:09:22.480608 29032 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:22.469295 28878 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:22.650012 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:22.472767 28773 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:22.534431 23451 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:22.502456 29061 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:22.655357 22905 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:22.666735 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:22.667678 22900 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [19.14-20.15]
W20260709 13:09:22.666749 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 61187.9us
I20260709 13:09:22.654338 23445 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:22.487215 28991 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:22.485960 29143 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:22.528246 29067 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.721746 22752 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:09:22.722194 22752 pending_rounds.cc:85] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 14
I20260709 13:09:22.722455 22752 pending_rounds.cc:107] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:09:22.722718 22752 raft_consensus.cc:2889] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:22.487023 28665 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:22.493513 29068 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:22.603132 29110 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:22.665889 29015 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.486603 29104 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.714712 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:22.760612 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51937.8us
W20260709 13:09:22.772290 22795 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.009s	sys 0.017s
I20260709 13:09:22.787505 22903 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:22.807199 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:22.706998 29061 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:22.865706 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1256) took 2108 ms. Trace:
I20260709 13:09:22.735061 29143 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.706914 29083 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:22.861704 29102 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.024s	sys 0.020s
I20260709 13:09:22.806376 29156 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:22.877123 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:22.865585 29007 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:22.878382 28839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.014s	sys 0.025s
I20260709 13:09:22.807521 28696 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:22.714466 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1496) took 1652 ms. Trace:
W20260709 13:09:22.889629 29102 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.024s	sys 0.020s
I20260709 13:09:22.704748 29032 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.901340 22898 rpcz_store.cc:276] 0709 13:09:21.062064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.314245 (+252181us) service_pool.cc:224] Handling call
0709 13:09:21.314572 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:22.393467 (+1078895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.714147 (+320680us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540034bcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:22.714393 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2474099456}
I20260709 13:09:22.862136 28627 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:22.734102 29067 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:22.868769 29061 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:22.907074 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2866) took 1751 ms. Trace:
I20260709 13:09:22.927529 22756 rpcz_store.cc:276] 0709 13:09:21.155469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.155612 (+   143us) service_pool.cc:224] Handling call
0709 13:09:21.155846 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:21.758274 (+602428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.906869 (+1148595us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b540011a8d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:22.907023 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346894720}
I20260709 13:09:22.889822 29049 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.772380 29110 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.736738 29121 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.921734 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:22.896267 28839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.014s	sys 0.025s
W20260709 13:09:22.938724 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.022s	sys 0.040s
I20260709 13:09:22.881758 29143 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:22.742563 28665 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:22.906080 29094 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:22.942783 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:22.939081 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.022s	sys 0.040s
I20260709 13:09:22.719012 28878 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:22.882651 28759 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.042s
W20260709 13:09:22.946342 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 562975us
I20260709 13:09:22.947505 22756 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:22.897367 29087 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:22.892899 29156 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.939046 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:22.960268 22898 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:22.892436 29083 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.947957 22756 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:22.766759 29167 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.178s
I20260709 13:09:22.774235 29015 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.773720 29068 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:22.974373 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.899366 29007 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.914893 28696 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.964861 22756 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.946820 29120 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.985416 28878 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:22.997391 29168 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:09:22.997960 29172 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:22.998493 29172 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.000525 29172 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.001412 28975 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:23.001956 28975 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.003363 28994 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.004464 28975 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.006014 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.013489 29094 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:23.018138 29019 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:23.018546 29019 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:23.019439 29019 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.020689 29019 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:23.024823 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1494) took 1963 ms. Trace:
I20260709 13:09:23.025431 22896 rpcz_store.cc:276] 0709 13:09:21.061615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.303818 (+242203us) service_pool.cc:224] Handling call
0709 13:09:21.345039 (+ 41221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:21.345429 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:21.345467 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:21.346031 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:09:21.580879 (+234848us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:21.580915 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:21.580973 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:21.985968 (+404995us) raft_consensus.cc:1685] finished
0709 13:09:21.985989 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:23.024333 (+1038344us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b5400206a50. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:23.024718 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":144972,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":3214,"spinlock_wait_cycles":93655424,"thread_start_us":234484,"threads_started":1,"wal-append.queue_time_us":324541,"wal-append.run_cpu_time_us":2386,"wal-append.run_wall_time_us":1273078}
W20260709 13:09:23.029876 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.033175 22902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2055) took 2393 ms (client timeout 2054 ms). Trace:
W20260709 13:09:23.033680 22902 rpcz_store.cc:269] 0709 13:09:20.639511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.639735 (+   224us) service_pool.cc:224] Handling call
0709 13:09:23.032935 (+2393200us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400184a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:23.033071 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2491500928}
I20260709 13:09:23.038573 29126 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.039073 29126 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.871502 22823 rpcz_store.cc:276] 0709 13:09:20.700006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.715734 (+ 15728us) service_pool.cc:224] Handling call
0709 13:09:20.715968 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:20.716131 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:09:20.871526 (+155395us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.307910 (+1436384us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b1800467f80. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb787f33c7 00007fcb74e059a7 00007fcb74e150bf 00007fcb74e15038
0709 13:09:22.308075 (+   165us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.807883 (+499808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:22.807935 (+    52us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:22.807978 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.808057 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.808159 (+   102us) inbound_call.cc:177] Queueing success response
0709 13:09:22.860579 (+ 52420us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b1800467f80. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
Metrics: {"mutex_wait_us":41134,"prepare.queue_time_us":248321,"raft.queue_time_us":41675,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":24471,"spinlock_wait_cycles":9094656,"thread_start_us":727140,"threads_started":3,"wal-append.queue_time_us":338453,"wal-append.run_cpu_time_us":1143,"wal-append.run_wall_time_us":1043568}
I20260709 13:09:23.040675 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:23.041075 29126 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:23.041991 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.042495 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.042907 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:23.044844 22902 raft_consensus.cc:1275] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:09:23.045167 22902 pending_rounds.cc:85] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 10
I20260709 13:09:23.045392 22902 pending_rounds.cc:107] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:09:23.045606 22902 raft_consensus.cc:2889] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:23.047276 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:23.048295 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:23.048761 22902 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:23.049757 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:23.050314 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.050431 22753 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:09:23.050881 22828 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:23.052385 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:23.052626 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.052888 22822 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:23.053750 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:23.053903 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.055060 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.056102 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.056360 28839 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.056903 28839 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.057588 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.058390 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.059351 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.059535 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:23.060027 22748 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 20: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:23.060134 22896 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:23.060777 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.061334 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.062561 28986 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:23.059602 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.066105 28745 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:23.067461 22822 raft_consensus.cc:2410] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:09:22.965525 29156 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:22.941129 29049 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:22.948315 28627 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:22.970146 28849 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:23.086894 22903 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:22.941833 29121 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.081287 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:23.093537 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.099892 22896 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:22.975895 28773 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.113612 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.114442 22748 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 22 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:09:23.115254 22902 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:23.116451 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.101504 22822 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:22.965674 29087 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.118614 22902 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:23.105957 22903 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:23.069177 29120 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.082538 28745 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:23.068416 29137 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.082916 29171 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:22.987430 22756 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:23.118650 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.126299 22751 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
I20260709 13:09:23.182962 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.126099 22829 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:23.126382 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:23.139971 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1506) took 1150 ms. Trace:
I20260709 13:09:23.188380 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.224476 22826 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:23.126005 22827 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:09:23.223443 23447 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:23.143782 23445 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:23.140074 22897 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:23.137730 22899 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:23.092665 28849 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.126241 22825 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:23.136291 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.125187 22797 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:23.084678 29015 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:23.243336 22755 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:23.244644 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.136938 23448 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:23.243472 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:23.126753 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:23.125473 22828 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:23.233031 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.118911 29087 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.109690 28892 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:09:23.128623 29169 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.642s	user 0.001s	sys 0.000s
W20260709 13:09:23.129076 29139 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.809s	user 0.007s	sys 0.002s
I20260709 13:09:23.245244 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.131238 29110 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.135943 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.261672 22829 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 13: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:23.224573 22904 rpcz_store.cc:276] 0709 13:09:21.990199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.077117 (+ 86918us) service_pool.cc:224] Handling call
0709 13:09:23.139777 (+1062660us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b54003ea654. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:23.139907 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160419584}
I20260709 13:09:23.235625 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:23.140986 29157 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.036s	user 0.012s	sys 0.012s
I20260709 13:09:23.266995 22749 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:23.269129 28847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.014s	sys 0.014s
W20260709 13:09:23.151495 28990 consensus_meta.cc:287] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.883s	user 0.008s	sys 0.001s
I20260709 13:09:23.125272 29104 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:23.288558 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:23.144079 28982 consensus_meta.cc:287] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.886s	user 0.005s	sys 0.004s
I20260709 13:09:23.134935 28563 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:23.274912 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 149778us
I20260709 13:09:23.120869 29121 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.226545 22899 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:23.153961 29007 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.279127 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.141618 29047 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.282912 22748 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:23.283623 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:23.435227 22904 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:09:23.152477 29032 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:23.125085 29083 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:23.162673 29049 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:23.095327 29030 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:23.155490 28696 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:23.144376 29166 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.292182 22826 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:23.126585 28773 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.268846 28849 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:23.283335 29184 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.280301 29172 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:09:23.306173 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.457692 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.392266 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.458969 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.428632 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.435757 28895 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.023s	sys 0.036s
I20260709 13:09:23.275206 28986 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.281572 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:23.273479 29091 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:23.485791 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 204270us
I20260709 13:09:23.279726 28892 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:23.226408 29171 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:23.460731 28847 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:09:23.481093 29156 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.467610 29184 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.490677 28994 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:23.490586 28986 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:23.492378 28994 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.492802 29184 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:23.451884 22869 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.014s	sys 0.051s
I20260709 13:09:23.282729 29137 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.245663 22902 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:09:23.456208 22753 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:23.443058 29146 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.469013 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:23.439862 28982 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.284051 29139 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.473451 29172 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:23.484187 28849 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.474583 29179 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:23.517684 22824 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:23.518664 28982 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.476348 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:23.518517 29172 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:23.519768 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 43499.4us
I20260709 13:09:23.519945 29000 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:23.520340 29000 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.473459 29089 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.520682 29000 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:23.520972 29089 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.522516 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:23.523080 29089 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.523587 22831 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:09:23.497675 29061 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.001s
I20260709 13:09:23.286559 29015 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.174s
I20260709 13:09:23.483268 29120 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.484916 28773 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.485450 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.487111 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:23.538908 22751 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
W20260709 13:09:23.539324 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 52295.5us
I20260709 13:09:23.495195 29171 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:23.492813 28994 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:23.467904 28990 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:23.506016 22869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.015s	sys 0.052s
I20260709 13:09:23.431304 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3409) took 1443 ms. Trace:
W20260709 13:09:23.485962 28895 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.023s	sys 0.036s
I20260709 13:09:23.506624 22749 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:09:23.544171 22749 pending_rounds.cc:85] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 10
I20260709 13:09:23.544464 22749 pending_rounds.cc:107] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:09:23.544752 22749 raft_consensus.cc:2889] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:23.496321 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.510088 22904 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:23.507575 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.474351 29132 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:23.525285 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:23.478511 29143 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.000s
I20260709 13:09:23.526242 22826 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:09:23.527997 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.528914 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.548904 22830 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:23.529531 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.529587 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.534746 22902 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:09:23.535647 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.540236 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:23.461108 29166 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.552311 22829 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:23.443225 29110 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:23.538403 29175 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.575s	user 0.000s	sys 0.016s
I20260709 13:09:23.494740 28986 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:23.535475 29032 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:23.539743 29178 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:23.547226 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1272) took 1466 ms. Trace:
I20260709 13:09:23.599637 22831 rpcz_store.cc:276] 0709 13:09:22.081276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.081396 (+   120us) service_pool.cc:224] Handling call
0709 13:09:22.088976 (+  7580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:23.524014 (+1435038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.547005 (+ 22991us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54003818d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:23.547173 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3298733952}
I20260709 13:09:23.600871 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.601416 22831 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:23.602362 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.548082 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:23.543711 28773 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.549619 22825 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
W20260709 13:09:23.603848 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 55854us
I20260709 13:09:23.490902 29091 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.542786 22754 rpcz_store.cc:276] 0709 13:09:21.988068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.989573 (+  1505us) service_pool.cc:224] Handling call
0709 13:09:21.989881 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.242262 (+252381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:22.242291 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:22.242326 (+    35us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:22.242354 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.431016 (+1188662us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b540032a854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:23.431071 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:23.431240 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":579993472}
I20260709 13:09:23.507790 29126 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:09:23.543396 29179 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:23.553989 28978 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:23.557783 29015 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:09:23.558046 29067 consensus_meta.cc:287] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.000s
I20260709 13:09:23.619594 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1271) took 1538 ms. Trace:
I20260709 13:09:23.619941 29067 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.620401 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2069) took 1834 ms. Trace:
I20260709 13:09:23.620754 22905 rpcz_store.cc:276] 0709 13:09:21.786147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.248689 (+462542us) service_pool.cc:224] Handling call
0709 13:09:22.249017 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.707227 (+458210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.620209 (+912982us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b540034c1d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:23.620349 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":933864832}
I20260709 13:09:23.621939 29067 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.555011 28892 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.558503 29068 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.531800 29146 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.622920 28892 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:23.619951 22824 rpcz_store.cc:276] 0709 13:09:22.080976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.081103 (+   127us) service_pool.cc:224] Handling call
0709 13:09:22.081497 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.518055 (+1436558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.619391 (+101336us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400089b54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:23.619545 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3302508416}
I20260709 13:09:23.561825 29110 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:23.563174 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2071) took 1576 ms. Trace:
I20260709 13:09:23.624131 29146 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.624006 22897 rpcz_store.cc:276] 0709 13:09:21.986566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.262931 (+276365us) service_pool.cc:224] Handling call
0709 13:09:23.139896 (+876965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.258953 (+119057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.563002 (+304049us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54003ea650. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:23.563120 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016220544}
I20260709 13:09:23.625177 29110 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:23.596012 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 70967.7us
I20260709 13:09:23.625576 28839 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.626291 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:23.626530 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.591653 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:23.597647 23416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.014s	sys 0.042s
I20260709 13:09:23.603822 28773 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.608139 28849 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:23.554401 29143 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:23.611925 29173 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.671s	user 0.010s	sys 0.000s
W20260709 13:09:23.615216 29019 consensus_meta.cc:287] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.594s	user 0.003s	sys 0.007s
I20260709 13:09:23.615523 29091 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.616348 29156 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:23.616778 29139 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.618204 29089 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:23.598816 29140 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:23.537962 29061 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:23.625833 22904 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:23.517064 29137 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.701385 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2075) took 1219 ms. Trace:
I20260709 13:09:23.543937 28895 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:23.751029 29185 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.751485 28895 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:23.751781 29185 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.598075 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:23.753456 29146 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:23.753719 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 155743us
I20260709 13:09:23.753944 29146 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:23.754639 29185 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.755862 29146 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:23.729674 22747 raft_consensus.cc:2410] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:09:23.547178 29132 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:23.646517 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.721967 22899 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:23.760200 22754 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:23.597388 29164 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.547355 28990 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:23.766120 22899 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:23.766691 22750 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.003s	sys 0.027s
W20260709 13:09:23.767072 22750 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.003s	sys 0.027s
I20260709 13:09:23.759498 28925 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:23.764496 29139 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:23.771445 28925 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.770089 29091 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.772408 28990 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:23.772565 29091 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.773523 28925 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:23.765712 29031 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:09:23.774741 29166 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.775463 22827 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:23.599562 29178 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:23.649636 28991 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.022s	sys 0.033s
W20260709 13:09:23.653810 29131 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.001s	user 0.021s	sys 0.072s
I20260709 13:09:23.718927 29089 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.660475 28773 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.788245 29132 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.696460 29067 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:23.697553 28665 consensus_meta.cc:287] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.738s	user 0.006s	sys 0.003s
I20260709 13:09:23.719919 29110 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:09:23.627619 23416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.014s	sys 0.042s
I20260709 13:09:23.757368 29140 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.774765 29091 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:23.792025 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 891665us
W20260709 13:09:23.723999 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 97781.3us
I20260709 13:09:23.792729 22899 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:23.793738 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:23.712539 23419 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.017s	sys 0.056s
I20260709 13:09:23.724622 22900 rpcz_store.cc:276] 0709 13:09:22.482218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.482328 (+   110us) service_pool.cc:224] Handling call
0709 13:09:22.482738 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:22.712400 (+229662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.701088 (+988688us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b540035cfd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:23.701305 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424546048}
W20260709 13:09:23.794445 23419 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.017s	sys 0.057s
I20260709 13:09:23.726902 29061 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:23.728569 29137 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.703689 28994 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.729178 29179 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:23.712674 28878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.005s	sys 0.021s
I20260709 13:09:23.730445 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:23.800621 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.729161 22901 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:23.790112 29089 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:23.749648 29022 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.750492 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:23.759472 22755 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:09:23.806833 22755 pending_rounds.cc:85] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:09:23.806546 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:23.807070 22755 pending_rounds.cc:107] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 16.9
W20260709 13:09:23.807144 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56735.9us
I20260709 13:09:23.807443 22755 raft_consensus.cc:2889] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:23.762318 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:23.763751 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.771749 29139 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:23.775688 22903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2077) took 1273 ms. Trace:
I20260709 13:09:23.784009 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.701669 22896 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:23.764923 29000 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.787665 29178 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:23.808528 22903 rpcz_store.cc:276] 0709 13:09:22.502540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.788189 (+285649us) service_pool.cc:224] Handling call
0709 13:09:23.775523 (+987334us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54000596d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:23.775632 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641883520}
I20260709 13:09:23.808959 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:23.784554 28991 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.030s	sys 0.037s
W20260709 13:09:23.790967 23450 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.012s	sys 0.011s
I20260709 13:09:23.725126 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:23.794773 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 832018us
I20260709 13:09:23.797024 29132 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:23.785583 29131 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.021s	sys 0.073s
I20260709 13:09:23.792867 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:23.802996 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:23.823005 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:23.731781 28627 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.765403 29164 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:23.798523 29085 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:09:23.808741 22747 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
W20260709 13:09:23.811625 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 49410.8us
I20260709 13:09:23.804301 29015 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.812145 22754 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:23.796212 29067 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:23.802196 28878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.006s	sys 0.021s
W20260709 13:09:23.813654 23450 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.012s	sys 0.011s
I20260709 13:09:23.795007 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2073) took 1313 ms. Trace:
I20260709 13:09:23.842490 22899 rpcz_store.cc:276] 0709 13:09:22.481652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.482336 (+   684us) service_pool.cc:224] Handling call
0709 13:09:23.137597 (+655261us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400439520. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d72dc0 00007fcb74dfee18 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6
0709 13:09:23.697186 (+559589us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400439520. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d7906f 00007fcb74df1f60 00007fcb74df37de 00007fcb74dff27c 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2
0709 13:09:23.794934 (+ 97748us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702008448}
I20260709 13:09:23.814960 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.824259 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:23.825546 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2080) took 1130 ms. Trace:
I20260709 13:09:23.844475 22901 rpcz_store.cc:276] 0709 13:09:22.695381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.708004 (+ 12623us) service_pool.cc:224] Handling call
0709 13:09:23.825372 (+1117368us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b54003e9254. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:23.825497 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2204337536}
I20260709 13:09:23.826748 22898 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:23.802743 29061 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:23.811940 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:23.812799 22751 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:23.846312 22751 pending_rounds.cc:85] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:09:23.837927 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.833799 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:23.822063 28773 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:23.822220 29091 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:23.797866 28665 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.846447 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:23.828696 29036 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:23.844813 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:23.846593 22751 pending_rounds.cc:107] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:09:23.852341 22751 raft_consensus.cc:2889] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:23.846966 22747 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 9.
I20260709 13:09:23.827438 29131 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:23.816666 29186 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:23.816634 28925 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
I20260709 13:09:23.853999 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:23.858438 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:23.859226 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:23.856268 23419 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:23.856349 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 63583.5us
I20260709 13:09:23.830771 29189 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:23.858741 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:23.828845 29022 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:09:23.848431 29137 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:23.868322 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:23.856904 29036 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.898952 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.863715 28849 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:23.872403 28696 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:23.873549 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:09:23.901363 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 348351us
I20260709 13:09:23.874226 29131 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.854914 29121 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:23.854893 29186 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.877717 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.878265 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.878508 28994 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.878873 29120 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:09:23.864332 29167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.022s	sys 0.020s
I20260709 13:09:23.865972 29188 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:23.852984 29032 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:23.927649 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.903210 29036 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:23.863922 28878 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:23.839241 29179 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:09:23.865229 29189 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.885571 29018 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:23.858834 28978 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:23.852425 29193 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:09:23.886837 29000 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.887408 29068 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:23.848207 28773 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.893273 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:23.863493 29190 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:23.897686 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.900547 28839 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:23.900629 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.902526 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:23.978008 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.854660 29146 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:09:23.982535 29178 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:23.985725 29190 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.973685 28978 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:23.986361 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.987110 29140 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.988204 22797 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:23.988895 28980 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:23.989249 28980 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:23.989601 28980 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:23.990381 28665 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.977264 28878 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:23.991359 29068 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.991762 28878 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.993285 29068 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.993427 29067 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.977270 28773 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:23.994294 28745 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:23.866624 29139 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:23.996431 29193 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:23.996783 29193 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:23.967027 29110 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.997608 29193 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.997946 29110 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.998771 29193 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:23.999872 29110 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:23.989614 29188 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:09:24.000499 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 98056.7us
I20260709 13:09:23.889555 29024 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:23.971913 28953 tablet_copy_client.cc:538] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:23.890127 29049 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:23.847090 29015 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.909436 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:23.880184 29164 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.012771 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:23.906327 28696 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:23.912367 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:23.861245 29156 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:09:23.924729 22750 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:23.918360 29121 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:23.929035 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.027452 23447 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:23.911773 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:23.936517 23445 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 13:09:24.028420 23445 pending_rounds.cc:85] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 15
I20260709 13:09:23.899232 28563 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:23.890393 29083 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.020s
I20260709 13:09:23.902009 29137 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:24.009572 29188 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:23.905364 29184 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:23.938174 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:23.952075 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.043444 22753 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
W20260709 13:09:24.044055 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.044090 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:23.908581 29186 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:24.049093 23452 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:09:24.049748 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.050046 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.051111 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.051208 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.051951 29186 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.052613 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.054739 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.055365 22824 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:24.056175 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.056699 22824 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:24.056514 29137 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:23.977655 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 84471.7us
I20260709 13:09:24.065922 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:23.911440 29120 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.973501 29018 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:23.986161 29190 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:23.975950 29194 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:09:23.927757 29032 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:24.078877 29032 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.004987 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:23.911226 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:24.013494 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 93616.7us
I20260709 13:09:23.972734 29189 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:24.020066 22896 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:24.020895 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.021631 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.022976 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:23.880947 29143 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:24.023231 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.023808 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.025532 22825 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:24.025712 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.025983 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:23.929828 28992 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.079s
I20260709 13:09:23.918394 29031 consensus_queue.cc:1048] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
W20260709 13:09:24.021961 22719 connection.cc:256] RPC call timeout handler was delayed by -1.5157868090000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 13:09:24.028820 23445 pending_rounds.cc:107] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 13:09:24.094867 23445 raft_consensus.cc:2889] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:24.094950 22719 connection.cc:256] RPC call timeout handler was delayed by -1.0171868090000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
I20260709 13:09:23.902236 29030 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.029927 22828 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:09:24.006904 29024 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:24.102239 29024 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:24.102833 29204 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:09:24.030822 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.001958 28978 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:24.107229 29197 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 13:09:24.036523 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.036659 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:24.037232 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 125529us
I20260709 13:09:24.091801 28892 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:09:24.037443 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.038769 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.111027 22826 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:24.111630 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.112668 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:23.910279 29167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.022s	sys 0.020s
I20260709 13:09:24.012514 29049 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:24.113895 29049 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:24.115136 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 2877) took 1057 ms. Trace:
I20260709 13:09:24.115480 22752 rpcz_store.cc:276] 0709 13:09:23.057242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.059406 (+  2164us) service_pool.cc:224] Handling call
0709 13:09:24.114944 (+1055538us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b540011b7d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:24.115059 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2210275584}
I20260709 13:09:24.116580 29019 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:24.018850 29047 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:24.058352 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.059650 29188 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.038522 28563 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.134799 28563 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:24.062278 29121 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.063258 29184 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:24.063426 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.063850 29172 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:23.974961 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.069905 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.070441 23447 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:24.070942 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.151057 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.072022 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.072347 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.073510 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.076772 28696 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.082527 22755 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:24.152705 22826 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:09:24.084471 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.086189 23453 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.089934 22901 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:09:24.087513 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:24.093225 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.093815 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.073989 29120 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.095988 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:24.101496 29030 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.081256 29018 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:24.118288 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.118476 29139 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.130517 28991 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:24.130729 29047 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:24.190421 29047 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:24.132143 28892 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:24.193456 28892 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.193734 29015 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.194430 28892 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.194847 29184 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.195598 28892 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.135401 29186 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.134399 23444 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:24.059651 28980 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:24.136593 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.198210 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.199097 28980 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.199481 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:24.199844 22830 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 10: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:24.137018 29191 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:24.200940 28563 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:24.201113 28992 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:24.201377 28563 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:24.139887 22827 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:24.081601 29061 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:24.141999 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.117785 22823 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
W20260709 13:09:24.149020 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.158212 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 86252.7us
I20260709 13:09:24.205760 23453 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
W20260709 13:09:24.159922 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 72514.1us
I20260709 13:09:24.210426 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.130254 23445 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
W20260709 13:09:24.162957 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67034.6us
I20260709 13:09:24.160434 28696 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:24.165798 29030 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:24.154772 28773 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.174449 29188 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.174793 29024 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.176719 29172 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:24.176829 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:24.182741 28953 tablet_bootstrap.cc:492] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:09:24.178870 29131 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:24.186285 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.139575 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:24.118279 28895 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.213737 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:24.204979 29015 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:24.220922 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.205464 22755 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:24.224311 22897 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:24.231916 22898 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:24.233690 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:24.234180 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.267295 22905 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:24.233723 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.201615 28992 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.237077 22900 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: [15.15-15.15]
I20260709 13:09:24.185107 28991 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:24.234741 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.259212 22831 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:24.269670 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:24.206374 28563 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.202641 29139 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:24.267869 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.269536 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48668.9us
I20260709 13:09:24.284132 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.271260 22905 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:09:24.261941 22904 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
W20260709 13:09:24.274341 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 97576us
W20260709 13:09:24.278820 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45172.8us
I20260709 13:09:24.288159 22748 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:24.234627 29193 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:24.215770 22901 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:24.288111 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:24.290781 29103 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:09:24.290931 29203 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260709 13:09:24.287918 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.299032 28563 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.299912 28978 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:24.301795 29212 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:09:24.304900 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:24.307711 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:24.359930 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.365533 23449 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:24.343472 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:24.327210 29110 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:24.327718 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.363809 28563 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:24.342531 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.370227 28773 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:24.374042 29136 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:24.385490 29136 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.386317 29136 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.387548 29136 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:24.271072 29015 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.390342 29190 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.390789 29015 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.392294 29190 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.351357 29091 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:09:24.375097 22904 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:09:24.270274 28895 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.308274 29061 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:24.306663 29103 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.284185 29132 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:24.282658 29034 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:24.296929 29036 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.231616 29032 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.376408 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.401855 23447 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:09:24.356003 22831 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:24.326901 29178 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:09:24.327943 29166 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:09:24.281994 28696 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:24.380208 22900 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:24.328136 29164 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:09:24.381008 22905 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:24.405927 28895 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.383944 23449 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 15. (term mismatch)
I20260709 13:09:24.384802 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.414402 23449 pending_rounds.cc:85] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 14
I20260709 13:09:24.414683 23449 pending_rounds.cc:107] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 15.15
I20260709 13:09:24.414273 22899 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:24.395526 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:24.395977 29212 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:09:24.396682 29091 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:24.307732 29131 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 13:09:24.419342 29049 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.420495 22896 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 20: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:24.417606 29091 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:24.400292 28991 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.360121 28992 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.398973 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.260509 29186 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:09:24.404213 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.436414 23448 raft_consensus.cc:2410] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:09:24.442163 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.436899 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:24.440472 22831 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:24.428238 29032 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.327556 29214 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.067s
W20260709 13:09:24.393878 29199 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:09:24.405027 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:24.445640 22898 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 15: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:24.446630 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.447494 22898 raft_consensus.cc:2368] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
I20260709 13:09:24.347615 28627 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:09:24.448796 28627 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:24.449716 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:24.400101 29191 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:09:24.429689 29024 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:24.449226 28627 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:24.429883 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 125032us
I20260709 13:09:24.452226 28627 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.421653 29091 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.453095 29184 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:24.434799 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:24.453855 29091 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:24.408471 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.410820 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.454636 28986 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:24.455276 28986 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:24.455968 22905 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: [16.16-16.16]
I20260709 13:09:24.456209 28986 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.328259 29188 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 13:09:24.435781 28992 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:24.429529 29183 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:24.459678 28978 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:24.460459 29089 consensus_meta.cc:287] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.646s	user 0.008s	sys 0.001s
I20260709 13:09:24.460858 29089 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.462486 22903 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:24.463176 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.397063 29143 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:24.490891 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:24.463107 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.492496 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.328821 29193 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.399494 29034 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.418651 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.418519 29103 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.503798 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.422425 29036 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:24.452932 29210 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.390s	user 0.003s	sys 0.000s
W20260709 13:09:24.381153 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 73508.5us
I20260709 13:09:24.515192 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.464022 22749 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:24.515882 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:24.342789 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:24.516909 22902 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.452955 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:24.491804 28986 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.482857 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.493709 28627 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.494278 22829 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:24.504781 29034 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:24.479940 22827 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:24.465726 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.536398 22748 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:24.454752 22897 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 23: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:24.501250 23446 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:24.501974 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.537439 23448 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:24.538015 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.493574 22898 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:24.421291 29049 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.439983 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.540602 22902 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:24.430512 28991 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:24.504279 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.508486 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.511612 29200 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:09:24.512530 29205 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.437s	user 0.000s	sys 0.002s
I20260709 13:09:24.511633 29032 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.430030 29068 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:24.408946 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:24.518700 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 65808.5us
I20260709 13:09:24.550918 22904 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:24.550796 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:24.552038 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:24.414918 23449 raft_consensus.cc:2889] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:24.520423 29189 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.527077 22824 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:24.424782 29120 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.532310 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.559055 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.560019 22829 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:24.541921 22831 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:24.544370 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:24.544544 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.557346 22824 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:24.564191 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:24.543272 29214 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 13:09:24.565196 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.565768 22829 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:24.565846 29083 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:24.567304 29214 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:24.567739 29214 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.569558 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.562579 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.570304 29139 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.543358 29200 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.570634 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.571383 29200 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.572530 29200 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.572757 29083 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:24.547073 29188 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:24.573724 29188 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:24.573777 29083 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:24.574343 29083 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.574684 29188 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.575213 29083 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.16, Last appended by leader: 16, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.576815 29083 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.560593 28895 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:24.577781 28895 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:24.535552 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.578539 22827 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:24.548359 22898 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:24.561277 29089 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.506034 29143 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:24.549265 22897 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:24.554294 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.580910 29143 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.582029 23446 raft_consensus.cc:2410] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:09:24.582931 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.584069 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.585955 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:24.587621 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.590916 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.591468 29120 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:24.556767 29184 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:09:24.593382 29110 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:24.593779 29018 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.595830 29018 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:24.550726 29068 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:24.562884 29190 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:24.596518 29068 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:24.559679 29132 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:24.597368 29132 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.599783 29132 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.601541 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.491118 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.555698 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.602586 22830 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.602835 22753 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:09:24.553030 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:24.529294 28878 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:24.525275 28892 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:24.440677 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.618391 23451 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:24.568738 29214 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.579110 22904 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
W20260709 13:09:24.604285 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51318.6us
I20260709 13:09:24.624259 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.606187 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.551885 22902 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:24.625283 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.608371 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:24.608726 22829 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:24.586926 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.609995 22825 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:24.638015 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.640164 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.642832 22824 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.619330 29186 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:24.620177 28892 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.643904 29186 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:24.644307 29186 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:24.644582 28892 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.645790 28892 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:24.617630 28878 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:24.649644 28878 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.650255 29189 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:24.652186 29218 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.244s	user 0.002s	sys 0.000s
I20260709 13:09:24.652247 29034 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.578562 28895 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.653636 28925 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:24.654305 28925 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.615372 22903 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:24.619951 23452 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:24.565780 29193 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:24.643312 29209 log.cc:927] Time spent T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.563s	user 0.002s	sys 0.000s
I20260709 13:09:24.637094 29015 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:24.657536 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.663213 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:24.610528 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51573.8us
I20260709 13:09:24.690065 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:24.692646 22751 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:09:24.670109 22830 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:24.676848 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:24.679342 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.695652 22753 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.683422 23447 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:24.593801 29110 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:24.584416 29183 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:24.657294 29132 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:09:24.699191 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.649009 29068 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:24.700179 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 91878us
I20260709 13:09:24.709417 29068 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.710096 28563 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:24.710734 28925 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:24.711026 28923 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:24.712690 29034 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:24.712809 28923 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:24.713500 29167 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:24.714032 28923 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:24.714401 28923 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.714565 29030 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:24.714982 29030 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.704969 29110 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:24.715268 28923 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.715857 29030 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.716500 28923 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:24.556135 29032 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.699936 29015 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:24.722987 29032 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.723601 29015 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:24.723816 28563 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:24.724248 28563 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:24.725019 28563 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.726343 28563 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:24.729310 22822 raft_consensus.cc:2410] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:09:24.704766 29146 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:09:24.734163 29146 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:24.734478 29146 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:24.735181 29146 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.736249 29146 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.698975 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.669591 29120 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:24.656432 29184 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:24.713246 29018 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:09:24.747059 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48149.6us
I20260709 13:09:24.751549 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.752504 23448 raft_consensus.cc:2410] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:09:24.753410 23449 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:24.754873 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.737515 22753 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:24.689127 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:24.741705 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:24.743183 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:24.762378 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 73316.5us
I20260709 13:09:24.762758 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.704573 29036 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:24.765722 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.771230 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:24.773826 22903 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:24.738487 28925 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:09:24.776651 22755 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:24.777078 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.764947 29189 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:24.697001 29185 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:09:24.744940 29018 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.799378 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.800144 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:24.781574 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.766462 29032 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:24.771263 29120 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:24.811520 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.783390 29024 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:24.821074 23449 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 25.
W20260709 13:09:24.833412 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.833954 29032 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:09:24.835146 29221 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.472s	user 0.000s	sys 0.004s
I20260709 13:09:24.835829 29032 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:24.836404 29032 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:24.809026 29018 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.814227 29189 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:24.844007 29189 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.814534 29036 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:09:24.846417 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 222812us
I20260709 13:09:24.847400 28990 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:24.848104 28990 consensus_queue.cc:1243] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:09:24.814425 29186 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.849792 29186 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:24.850803 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.851079 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.851980 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.852159 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.852880 22830 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:24.853022 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.844007 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:24.853588 22829 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:09:24.814872 29120 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:24.854661 29120 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.856201 22824 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:24.858016 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.859601 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.863248 22901 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:09:24.879832 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:24.867501 23444 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:24.909502 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:24.885776 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.910509 22901 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:24.911000 22830 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:24.877099 23446 raft_consensus.cc:2410] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 17 because replica has last-logged OpId of term: 16 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 15.
I20260709 13:09:24.877780 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:24.877111 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:24.909312 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:24.913210 23453 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:24.916489 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:24.916611 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.863886 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.862191 22829 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:24.923910 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.924019 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.929016 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.868947 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:24.899894 29113 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.011s	sys 0.011s
I20260709 13:09:24.937369 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.934470 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:24.960618 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:24.944778 22824 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:24.963701 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.966089 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.966719 22825 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:24.962769 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:24.967743 22824 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:24.969691 22830 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:24.970780 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:24.971730 22830 raft_consensus.cc:2410] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:09:24.972734 22830 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:24.866959 29178 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:24.983192 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:24.984675 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50290.9us
W20260709 13:09:24.988425 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 64508.4us
I20260709 13:09:24.927878 28990 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:09:24.963761 29113 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.011s	sys 0.013s
I20260709 13:09:24.991887 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:24.993041 22901 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:24.866922 29139 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:24.933486 28773 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:24.958113 29091 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:24.868613 29110 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.879484 28878 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:24.995972 29121 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:24.963168 29049 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:24.963618 29143 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:25.036059 29184 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:25.044826 29184 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:25.045466 29184 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:25.027366 29212 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.053s	user 0.008s	sys 0.000s
I20260709 13:09:25.030917 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:24.879330 28986 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.140s
I20260709 13:09:25.056260 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.057510 29185 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:24.993157 29036 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:25.057880 29185 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:24.876031 28980 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 13:09:25.058305 29185 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:25.058699 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.058799 28980 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:25.059309 28980 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:25.059871 29121 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:25.060706 29121 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:24.886211 29032 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.029006 29091 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.062014 29121 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:25.062255 29091 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:25.062928 28991 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:25.063326 28991 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:25.064097 28991 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.053401 22898 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:25.065477 29183 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:25.025746 22824 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:25.065892 29183 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:25.065634 29185 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.059392 29137 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:25.066764 29183 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.067216 29137 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:25.067847 29183 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:25.067796 29137 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:25.060422 28980 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.062034 29032 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:25.069099 28980 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:25.074107 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.075173 22749 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:25.052240 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 174557us
W20260709 13:09:25.075213 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.076326 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:25.077242 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.037740 29139 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:25.065202 28991 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:25.084769 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:25.091142 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 128753us
I20260709 13:09:25.096187 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:25.087333 29139 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.085692 28978 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:25.124310 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:25.125952 22901 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:25.127133 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.087646 29131 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:25.110919 29139 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:25.248201 22901 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:25.249297 29110 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:25.256739 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:25.260877 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:25.268255 29250 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:25.274227 28878 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:25.099699 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.260545 29137 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.300441 23451 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
W20260709 13:09:25.292145 29228 log.cc:927] Time spent T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
W20260709 13:09:25.301720 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45027.7us
I20260709 13:09:25.089097 28986 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
W20260709 13:09:25.322280 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 222656us
W20260709 13:09:25.323175 22719 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:25.301538 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:25.125815 28978 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:25.323861 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:25.324458 22901 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:25.325596 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:25.326368 29200 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:25.317246 29229 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:09:25.309397 28892 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:25.356940 22901 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
W20260709 13:09:25.336048 29237 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
W20260709 13:09:25.336951 29241 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.287s	user 0.000s	sys 0.000s
W20260709 13:09:25.337877 29232 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
I20260709 13:09:25.339481 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:25.340144 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 208628us
I20260709 13:09:25.321918 29250 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:25.329381 28978 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:25.342574 22905 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:09:25.334794 29132 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.378538 22905 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:09:25.361785 22825 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:25.350268 28563 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:25.383128 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:25.376467 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:25.385623 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.374366 28878 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:25.292472 29103 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:09:25.367233 22901 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:09:25.329085 29186 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:25.372218 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:25.394515 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:25.362792 28892 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.381560 22824 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:09:25.389186 22794 leader_election.cc:400] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
I20260709 13:09:25.397591 22794 leader_election.cc:403] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:09:25.375697 29146 consensus_meta.cc:287] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.000s
I20260709 13:09:25.392401 29254 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:25.399621 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.382221 28563 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.408697 28980 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
I20260709 13:09:25.409253 28878 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:25.427397 28878 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:25.428309 28878 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.390897 29248 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:25.429497 29248 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.429845 29248 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:25.431671 22827 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [19.10-20.11]
I20260709 13:09:25.435456 29250 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.378564 28895 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:09:25.438086 28895 raft_consensus.cc:3055] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:25.438409 28895 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:25.439208 28895 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.440367 28895 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:25.384213 29132 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.447116 29132 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:25.448390 29219 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.097s	user 0.004s	sys 0.001s
I20260709 13:09:25.451722 29248 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.453075 29214 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:25.460637 28923 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:25.461040 28923 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:25.462646 29248 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:25.463912 29222 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.119s	user 0.000s	sys 0.014s
I20260709 13:09:25.464968 29094 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.465509 29094 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.468322 23419 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:25.470700 22751 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 11. Current term is 12. Ops: []
W20260709 13:09:25.415059 29018 consensus_meta.cc:287] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.576s	user 0.003s	sys 0.006s
I20260709 13:09:25.386837 28986 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
W20260709 13:09:25.419305 29246 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:09:25.387614 29189 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.010s	sys 0.000s
I20260709 13:09:25.421375 22825 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:09:25.329129 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:25.372323 29247 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:25.424746 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.423372 28563 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:25.486865 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 114777us
W20260709 13:09:25.513451 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 88632.3us
I20260709 13:09:25.513736 22825 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:25.514163 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.516324 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:25.516692 22825 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:25.451524 29184 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.399410 28892 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:25.501935 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.510417 29120 consensus_meta.cc:287] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.001s
I20260709 13:09:25.477789 29251 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:25.520823 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:25.527237 22902 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:25.522073 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.522040 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:25.512807 29031 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:09:25.503474 29189 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.010s	sys 0.000s
I20260709 13:09:25.520463 29248 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:25.461923 28923 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.526422 29120 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.505460 29018 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:25.530790 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.538700 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:25.523586 29146 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:25.553802 29260 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:25.520138 28980 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:25.526713 29243 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:09:25.525490 29137 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:25.553853 22902 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:25.577589 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:25.581856 22754 raft_consensus.cc:1240] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:25.583480 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1287) took 1057 ms. Trace:
I20260709 13:09:25.583133 29120 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:25.571156 29247 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:25.615562 29247 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:25.577795 29248 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:25.622107 29248 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:25.586215 29255 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:09:25.603824 29184 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:25.578151 29146 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:09:25.589571 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 65526.1us
W20260709 13:09:25.565796 29091 consensus_meta.cc:287] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 13:09:25.603775 29139 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.605427 28980 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:25.607710 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1275) took 1167 ms. Trace:
I20260709 13:09:25.541759 29094 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:25.599536 22831 rpcz_store.cc:276] 0709 13:09:24.526483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.571419 (+ 44936us) service_pool.cc:224] Handling call
0709 13:09:24.571720 (+   301us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:24.844381 (+272661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:24.844425 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:24.845149 (+   724us) log.cc:844] Serialized 52 byte log entry
0709 13:09:25.052878 (+207729us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:25.085530 (+ 32652us) log.cc:844] Serialized 14 byte log entry
0709 13:09:25.090893 (+  5363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.090986 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:25.583218 (+492232us) raft_consensus.cc:1685] finished
0709 13:09:25.583245 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.583410 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":277399,"prepare.queue_time_us":32806,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2983,"raft.queue_time_us":297578,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":12555,"spinlock_wait_cycles":9239296,"thread_start_us":201075,"threads_started":1,"wal-append.queue_time_us":520460}
W20260709 13:09:25.612805 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.689162 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.694366 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:25.695088 22755 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:25.682983 22823 rpcz_store.cc:276] 0709 13:09:24.440771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.440870 (+    99us) service_pool.cc:224] Handling call
0709 13:09:24.441190 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:24.443671 (+  2481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:24.443701 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:24.444156 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:09:24.505358 (+ 61202us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:24.505741 (+   383us) log.cc:844] Serialized 14 byte log entry
0709 13:09:24.506201 (+   460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.506270 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:24.843968 (+337698us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b18003020e0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb78ccf58f 00007fcb71d0e571 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd
0709 13:09:25.605344 (+761376us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b18003020e0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1
0709 13:09:25.605986 (+   642us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b18003020e0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:25.607500 (+  1514us) raft_consensus.cc:1685] finished
0709 13:09:25.607520 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.607655 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1837,"prepare.queue_time_us":302,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":8203,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":6885,"spinlock_wait_cycles":26147456,"thread_start_us":60755,"threads_started":1,"wal-append.queue_time_us":81629}
I20260709 13:09:25.697877 22827 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:25.529893 29132 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.154s
I20260709 13:09:25.677805 29184 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:25.702203 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 233939us
I20260709 13:09:25.702772 22748 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:25.703637 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:25.705870 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:25.681943 29047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.300s	user 0.002s	sys 0.011s
I20260709 13:09:25.711474 23453 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 13:09:25.712718 22751 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:09:25.685492 28992 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:25.710289 22748 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 22. Current term is 23. Ops: []
W20260709 13:09:25.710099 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.715999 22753 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:09:25.717177 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.718319 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.710062 29047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.334s	user 0.003s	sys 0.012s
I20260709 13:09:25.721539 22831 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:09:25.713987 29200 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:25.723387 29200 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:25.724051 23452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 1132) took 1678 ms (client timeout 1751 ms). Trace:
W20260709 13:09:25.724416 23452 rpcz_store.cc:269] 0709 13:09:24.045902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.050895 (+  4993us) service_pool.cc:224] Handling call
0709 13:09:25.723842 (+1672947us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54004f3154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:25.723976 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1269209344}
I20260709 13:09:25.716468 29094 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.726898 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3448) took 1065 ms. Trace:
I20260709 13:09:25.727128 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:25.725803 23450 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:25.683496 29091 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.727353 22752 rpcz_store.cc:276] 0709 13:09:24.661960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.662428 (+   468us) service_pool.cc:224] Handling call
0709 13:09:24.662726 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:24.785298 (+122572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:24.786389 (+  1091us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:24.786446 (+    57us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:24.786492 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.726698 (+940206us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540032a854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:25.726739 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.726847 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276192000}
I20260709 13:09:25.715407 29266 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.739166 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:25.740195 22755 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:25.715324 29120 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:09:25.741629 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 412690us
I20260709 13:09:25.751314 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:25.752429 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:25.753108 22719 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.68:42017 timed out after 1.751s (SENT) [suppressed 10 similar messages]
I20260709 13:09:25.755494 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:25.761696 22797 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:25.764106 22747 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:25.726464 29094 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:25.742898 29131 consensus_meta.cc:287] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:09:25.767518 23453 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:09:25.781037 23453 pending_rounds.cc:85] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 12
I20260709 13:09:25.725852 29200 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:25.713332 29184 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.759166 29266 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.771790 28627 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:25.781435 29263 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:09:25.788688 29263 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 22.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:09:25.790220 29261 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.790349 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:25.790678 29261 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.760376 29248 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:09:25.791628 28627 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:25.792639 29261 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:25.792536 28627 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.794418 29262 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:25.794872 29262 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:25.750094 29120 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:25.722383 28892 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.797581 22831 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:25.783643 23453 pending_rounds.cc:107] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:09:25.799508 23453 raft_consensus.cc:2889] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:25.738593 28980 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:09:25.785006 29247 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.808750 29247 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:25.796502 29214 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.786942 29266 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:25.809656 28892 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:25.809818 29262 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 22.20, Last appended by leader: 20, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.810638 23448 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:09:25.793151 29261 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:25.814595 29247 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:25.815088 29217 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.465s	user 0.005s	sys 0.001s
I20260709 13:09:25.815192 29247 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.782354 29131 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:25.816263 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3072) took 1722 ms. Trace:
I20260709 13:09:25.816612 22826 rpcz_store.cc:276] 0709 13:09:24.094224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.153422 (+ 59198us) service_pool.cc:224] Handling call
0709 13:09:24.153679 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:24.153862 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:24.153886 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:24.154532 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 13:09:24.303343 (+148811us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:24.303377 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.303435 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:25.814604 (+1511169us) spinlock_profiling.cc:243] Waited 67.7 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:25.815960 (+  1356us) raft_consensus.cc:1685] finished
0709 13:09:25.815983 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.816212 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155671424,"thread_start_us":134377,"threads_started":1,"wal-append.queue_time_us":181311}
W20260709 13:09:25.787597 28953 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:09:25.759307 28986 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
W20260709 13:09:25.820039 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 66958.5us
I20260709 13:09:25.795507 29260 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:09:25.819550 29261 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:25.837217 29261 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.838447 29261 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:25.784046 29200 consensus_queue.cc:1059] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
W20260709 13:09:25.849774 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 87052.5us
I20260709 13:09:25.768512 29146 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.849176 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:25.852056 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:25.803419 28563 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.854562 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.809701 29214 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.853106 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:25.835152 29184 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:25.857095 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.847580 29018 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:25.867386 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 115739us
I20260709 13:09:25.869827 29094 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:25.871508 29214 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:25.871446 28978 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.872329 23453 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:09:25.873711 29091 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:25.847163 29120 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.875813 29120 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:25.877496 22828 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 21. Ops: []
W20260709 13:09:25.877948 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.858021 29139 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:25.879063 28978 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:25.879453 28563 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:25.857831 29251 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:25.880162 23452 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:25.882189 23452 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:09:25.889526 29271 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:09:25.890699 29030 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:25.889021 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1613) took 1350 ms. Trace:
I20260709 13:09:25.891332 23444 rpcz_store.cc:276] 0709 13:09:24.538865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.605988 (+ 67123us) service_pool.cc:224] Handling call
0709 13:09:25.888864 (+1282876us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b5400684054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:25.888968 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541063424}
W20260709 13:09:25.892560 28991 consensus_meta.cc:287] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.787s	user 0.007s	sys 0.003s
I20260709 13:09:25.856562 29121 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:25.894809 29269 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:25.895202 29269 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:25.858134 29047 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:25.896440 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:25.903555 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:25.909214 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:25.932337 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:25.933894 23447 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:25.933701 29258 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:25.934232 23447 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:25.934573 29258 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:25.935387 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:25.935974 29258 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:25.936427 29258 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:25.935016 23447 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.937533 23447 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:25.895926 29269 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.936072 23452 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 14: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:25.944458 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:25.946271 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:25.897710 28878 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:25.947717 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:25.948951 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:25.941557 29261 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:25.937198 29258 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.944032 29121 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:25.951917 29184 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:25.951887 29139 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:25.926546 28895 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:09:25.941810 29018 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:09:25.959518 29258 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:25.960541 29269 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:25.963717 29261 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:25.981963 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1528) took 1557 ms. Trace:
I20260709 13:09:25.982091 29261 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:25.961084 29279 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
W20260709 13:09:25.968380 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.955242 28980 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:09:25.966637 29139 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:25.974920 29233 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.931s	user 0.003s	sys 0.000s
I20260709 13:09:25.982348 22898 rpcz_store.cc:276] 0709 13:09:24.424616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.580037 (+155421us) service_pool.cc:224] Handling call
0709 13:09:24.580335 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.065562 (+485227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.981766 (+916204us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54001b6054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:25.981913 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087374592}
I20260709 13:09:25.967214 29121 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:25.956671 23447 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:09:25.986806 29167 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:25.987406 22831 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:09:25.979861 29247 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:25.989425 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:25.990850 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1616) took 1451 ms. Trace:
I20260709 13:09:25.995849 22754 raft_consensus.cc:2410] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:09:25.996299 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.022835 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.006830 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:26.022471 23446 rpcz_store.cc:276] 0709 13:09:24.539546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.586691 (+ 47145us) service_pool.cc:224] Handling call
0709 13:09:25.990672 (+1403981us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54006833d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:25.990798 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550324096}
I20260709 13:09:26.008893 29089 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:26.008252 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2803) took 1557 ms. Trace:
I20260709 13:09:26.014075 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.032217 22901 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:26.033299 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:26.034760 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.017195 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.022043 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:26.061717 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:26.002351 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:26.015358 22899 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:09:26.062776 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.030140 22897 rpcz_store.cc:276] 0709 13:09:24.451075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.580996 (+129921us) service_pool.cc:224] Handling call
0709 13:09:24.581260 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:24.602258 (+ 20998us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:25.022002 (+419744us) log.cc:844] Serialized 14 byte log entry
0709 13:09:26.007928 (+985926us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:26.007979 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:26.008025 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.008098 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:26.008204 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":940473,"prepare.queue_time_us":118067,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":931442,"spinlock_wait_cycles":3953024,"thread_start_us":418870,"threads_started":2,"wal-append.queue_time_us":87096}
I20260709 13:09:26.008301 29143 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.007812 29036 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:26.105423 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 2214) took 1174 ms. Trace:
I20260709 13:09:26.098755 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:26.101938 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.111812 22831 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:09:26.108317 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 1936) took 1449 ms. Trace:
I20260709 13:09:26.066896 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:26.113054 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:26.113956 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:26.114563 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1535) took 1242 ms. Trace:
I20260709 13:09:26.114858 22826 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:26.111230 22827 rpcz_store.cc:276] 0709 13:09:24.931038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.432434 (+501396us) service_pool.cc:224] Handling call
0709 13:09:25.432688 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.698399 (+265711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.105185 (+406786us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b5400091fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:26.105373 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":609312000}
I20260709 13:09:26.113001 22749 rpcz_store.cc:276] 0709 13:09:24.658581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.658748 (+   167us) service_pool.cc:224] Handling call
0709 13:09:24.659053 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.075648 (+416595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.108112 (+1032464us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54001860d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:26.108256 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956426112}
I20260709 13:09:26.075555 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.021723 29251 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.119215 22899 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:26.121222 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.122110 22899 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:26.123401 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:26.114997 22896 rpcz_store.cc:276] 0709 13:09:24.871865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.965882 (+ 94017us) service_pool.cc:224] Handling call
0709 13:09:24.966125 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:24.966333 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 13:09:25.027564 (+ 61231us) log.cc:844] Serialized 14 byte log entry
0709 13:09:25.036411 (+  8847us) log.cc:844] Serialized 14 byte log entry
0709 13:09:25.995787 (+959376us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b180044cfe0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb74df3060 00007fcb74e05716 00007fcb74e16287 00007fcb74e16208
0709 13:09:26.114273 (+118486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:26.114318 (+    45us) raft_consensus.cc:1645] Marking committed up to 20
0709 13:09:26.114367 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.114422 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:26.114518 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":964269,"prepare.queue_time_us":85703,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":968255,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":4413,"spinlock_wait_cycles":35870592,"thread_start_us":118268,"threads_started":1}
W20260709 13:09:26.112581 28953 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:09:26.127841 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.064324 29015 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:26.104461 29240 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.775s	user 0.001s	sys 0.001s
I20260709 13:09:26.066220 28773 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.025215 29258 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 13:09:26.064169 29193 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.101184 29184 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:26.132413 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:26.057900 29164 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.057684 28895 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.101614 29018 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.118392 29188 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:26.059059 29260 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.134307 29032 raft_consensus.cc:2764] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:09:26.136550 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:26.124469 29085 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:26.136149 28892 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:26.153493 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.151597 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.155400 22868 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:26.124615 29143 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.159380 22899 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:26.134610 29167 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:09:26.146582 29264 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.127146 29251 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.164466 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:26.148208 29277 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.146109 29015 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.199635 22901 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:26.131371 29139 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.143328 28773 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:26.191151 29289 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.156s	user 0.002s	sys 0.003s
I20260709 13:09:26.190158 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:26.177891 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:26.166891 29085 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:26.169042 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:09:26.214213 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 45249.6us
I20260709 13:09:26.147145 29193 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.156446 29018 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.196414 29032 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:26.153426 29164 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.218647 22905 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:26.156049 28895 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.162061 28892 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.159876 29110 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:26.218089 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:26.157826 29188 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.229079 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:26.232749 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 63464.3us
W20260709 13:09:26.234757 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 56991.1us
I20260709 13:09:26.159595 29260 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.218170 29184 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:26.158429 29036 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.218523 29085 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:26.239624 29167 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 13:09:26.250478 23447 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:26.193167 29264 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.236384 29131 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:09:26.257220 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.218289 29258 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:09:26.207618 29277 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.241957 29120 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:26.266585 28892 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.263067 29120 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.273447 29120 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:26.276041 29264 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.277544 29164 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.278095 28773 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:26.278650 29085 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.279933 29188 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.281639 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2805) took 1158 ms. Trace:
I20260709 13:09:26.282064 22900 rpcz_store.cc:276] 0709 13:09:25.123292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.124826 (+  1534us) service_pool.cc:224] Handling call
0709 13:09:25.249409 (+124583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:26.028748 (+779339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:26.028783 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:26.064827 (+ 36044us) log.cc:844] Serialized 26 byte log entry
0709 13:09:26.209201 (+144374us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:26.209240 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.209306 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:26.281416 (+ 72110us) raft_consensus.cc:1685] finished
0709 13:09:26.281449 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:26.281586 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":759086,"raft.queue_time_us":759789,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":72826,"threads_started":1,"wal-append.queue_time_us":124964}
I20260709 13:09:26.270912 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:26.258605 29131 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:26.285343 22822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 1280) took 1802 ms (client timeout 2373 ms). Trace:
I20260709 13:09:26.287796 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:26.288744 23447 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 25: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:26.282979 29015 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:26.281622 29193 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.291631 22824 raft_consensus.cc:2410] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 17 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 14.
W20260709 13:09:26.287506 22822 rpcz_store.cc:269] 0709 13:09:24.482436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.482564 (+   128us) service_pool.cc:224] Handling call
0709 13:09:26.285119 (+1802555us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54002f0dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:26.285267 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476678400}
I20260709 13:09:26.282227 29018 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.293916 29131 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:26.305289 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.294263 29258 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:26.306962 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:26.298763 29167 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:26.309782 29167 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:26.310941 29167 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.312424 29167 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:26.307596 28990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.272s	user 0.014s	sys 0.002s
W20260709 13:09:26.314036 28990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.283s	user 0.015s	sys 0.002s
I20260709 13:09:26.314334 28990 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:26.315150 28990 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.319741 28895 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.321631 29251 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.330741 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:26.302367 29299 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:26.339581 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.335541 29143 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.316630 29193 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 13:09:26.305104 29132 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.316969 29121 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.344285 29299 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.321064 29120 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.357420 29299 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:26.357970 29299 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.358400 29299 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.360495 29299 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.361191 29184 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:26.362111 29290 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:26.361873 29184 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.362785 29185 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.340776 29277 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.317090 29258 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.364244 29184 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.365521 29258 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.355209 29132 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.377568 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:26.368917 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 133247us
I20260709 13:09:26.379956 22747 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:26.366619 29189 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:26.355113 29121 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.397771 29132 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.405490 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:26.402558 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:26.404186 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:26.404654 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:26.410532 22904 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 15. Ops: [13.8-13.8]
I20260709 13:09:26.411559 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2089) took 1643 ms. Trace:
I20260709 13:09:26.411965 22904 rpcz_store.cc:276] 0709 13:09:24.767687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.824977 (+ 57290us) service_pool.cc:224] Handling call
0709 13:09:26.405645 (+1580668us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b5800211598. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6eee 0000000000375939 00007fcb79d4a1e0 00007fcb79e130c4 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:09:26.405831 (+   186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:26.411244 (+  5413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.411495 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3249408}
I20260709 13:09:26.406914 22749 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:26.415021 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:26.398586 29302 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:26.416327 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:26.419267 29015 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:26.419775 29015 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.422060 29015 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.422318 29164 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.422895 29164 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.351918 29200 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:26.424091 29200 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:26.424743 29164 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.424901 29200 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.425572 29301 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:26.426046 29200 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:26.428143 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:26.425467 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.429476 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.430034 22896 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:26.430889 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.431221 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:26.431540 22896 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:26.432142 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:26.432811 22752 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:26.433328 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:26.364270 28992 raft_consensus.cc:3055] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:26.445528 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.445765 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.445609 23447 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:26.446585 22902 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:26.447795 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:26.449115 29290 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:26.453373 23447 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:09:26.452253 29290 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.405448 28925 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.468943 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:26.471776 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.473588 22899 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:26.475975 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:26.476111 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.476541 22900 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:26.476972 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:26.478744 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:26.479308 22896 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:26.484665 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:09:26.669698 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.485816 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:26.553143 22898 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
W20260709 13:09:26.675016 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 197017us
W20260709 13:09:26.494596 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.467121 29302 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:09:26.580480 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.443905 28992 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:26.677304 22896 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:09:26.676767 28953 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:09:26.435247 29261 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:09:26.681588 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 261549us
I20260709 13:09:26.482693 28895 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:26.583165 29185 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:26.680178 29302 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.482580 29184 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.202s
I20260709 13:09:26.688053 23448 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:26.500232 28878 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:26.679966 29299 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:26.512862 29300 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.482549 29214 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.695643 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:26.685446 29200 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:26.722359 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.724699 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:26.733814 22896 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 13:09:26.734736 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.508865 29146 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.188s
I20260709 13:09:26.556680 29131 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.484536 28925 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.692706 29185 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:26.697013 29299 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.698335 29261 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.693919 29302 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:26.714831 28895 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:26.806417 29103 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.806927 29103 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:26.786075 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.810518 28923 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:26.813762 29315 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:09:26.738417 29036 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:26.814793 28696 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:26.740186 29120 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.748044 29189 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:26.716051 29314 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.116s
I20260709 13:09:26.699323 28878 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:26.809140 29103 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.751008 28953 tablet_bootstrap.cc:492] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:26.819128 28696 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.715332 29300 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.803692 29299 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:26.758172 29121 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:26.821429 29311 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:09:26.756186 29185 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:26.794570 29261 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:26.737610 29200 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.737720 29214 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:26.854732 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:26.849442 28990 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.061s	sys 0.175s
I20260709 13:09:26.829826 28992 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:26.869848 28696 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:26.871483 29131 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:26.876744 28990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.062s	sys 0.176s
I20260709 13:09:26.890776 29214 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:26.963562 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 1952) took 1021 ms. Trace:
I20260709 13:09:26.964368 22754 rpcz_store.cc:276] 0709 13:09:25.942100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.944246 (+  2146us) service_pool.cc:224] Handling call
0709 13:09:26.963045 (+1018799us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b540011c1d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:26.963336 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112140544}
I20260709 13:09:25.468866 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:26.993402 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 1301) took 1004 ms. Trace:
I20260709 13:09:26.993901 22824 rpcz_store.cc:276] 0709 13:09:25.988558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.989157 (+   599us) service_pool.cc:224] Handling call
0709 13:09:26.992887 (+1003730us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400432e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:26.993145 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618867968}
I20260709 13:09:26.994488 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1283) took 2492 ms. Trace:
I20260709 13:09:26.994931 22828 rpcz_store.cc:276] 0709 13:09:24.501792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.510626 (+  8834us) service_pool.cc:224] Handling call
0709 13:09:25.816121 (+1305495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.877891 (+ 61770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.993515 (+1115624us) spinlock_profiling.cc:243] Waited 61.2 ms on lock 0x7b5400091ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:26.994103 (+   588us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400091ad0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:26.994310 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3142299776}
I20260709 13:09:26.995282 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.006026 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:27.025305 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2098) took 1010 ms. Trace:
I20260709 13:09:27.030323 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.029860 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1543) took 1008 ms. Trace:
I20260709 13:09:27.032037 22901 rpcz_store.cc:276] 0709 13:09:26.013757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.034473 (+ 20716us) service_pool.cc:224] Handling call
0709 13:09:27.022064 (+987591us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400002dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:27.022321 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318399360}
I20260709 13:09:27.032706 22897 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:27.034363 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2096) took 1020 ms. Trace:
I20260709 13:09:27.034936 22897 rpcz_store.cc:276] 0709 13:09:26.013998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.083853 (+ 69855us) service_pool.cc:224] Handling call
0709 13:09:27.022452 (+938599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.033370 (+ 10918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.034109 (+   739us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b5400002dd0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:27.034311 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2157306624}
I20260709 13:09:27.054802 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:27.055610 23444 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:27.067531 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:27.068645 23447 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:27.068562 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.068584 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:27.069953 23445 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:27.070263 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:27.071182 23446 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:27.071630 23445 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:27.071209 23447 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:27.074945 28925 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.075619 23449 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 1160) took 1093 ms. Trace:
I20260709 13:09:27.075846 28925 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:09:27.079201 28773 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:26.875468 28895 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:27.081768 22829 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 3082) took 2218 ms (client timeout 1666 ms). Trace:
W20260709 13:09:27.082309 22829 rpcz_store.cc:269] 0709 13:09:24.863588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.940811 (+ 77223us) service_pool.cc:224] Handling call
0709 13:09:27.081271 (+2140460us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b54002c9a54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:27.081509 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4804999424}
I20260709 13:09:27.084364 28953 tablet_bootstrap.cc:492] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:09:27.084451 29015 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.085526 29015 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.088039 28563 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:27.088560 28563 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:27.090200 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 1152) took 1369 ms. Trace:
I20260709 13:09:27.090626 23453 rpcz_store.cc:276] 0709 13:09:25.720285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.802000 (+ 81715us) service_pool.cc:224] Handling call
0709 13:09:25.802272 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.872655 (+ 70383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.089659 (+1217004us) spinlock_profiling.cc:243] Waited 69.8 ms on lock 0x7b54005e1ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:27.090030 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160465664}
I20260709 13:09:27.093266 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1678) took 2753 ms. Trace:
I20260709 13:09:27.093796 23450 rpcz_store.cc:276] 0709 13:09:24.339730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.339870 (+   140us) service_pool.cc:224] Handling call
0709 13:09:25.725589 (+1385719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.728076 (+  2487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.092769 (+1364693us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54004f3150. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:27.093030 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3132791168}
I20260709 13:09:27.096187 23451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1698) took 1405 ms. Trace:
I20260709 13:09:27.096621 23451 rpcz_store.cc:276] 0709 13:09:25.690353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.690485 (+   132us) service_pool.cc:224] Handling call
0709 13:09:25.690743 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.728515 (+ 37772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:25.728544 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:25.728593 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:25.728644 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.095696 (+1367052us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b5400698f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:27.095781 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.096015 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86358400}
I20260709 13:09:27.100802 29061 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:27.103125 29024 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:27.107779 29024 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.125687 28563 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.076036 23449 rpcz_store.cc:276] 0709 13:09:25.981622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.981776 (+   154us) service_pool.cc:224] Handling call
0709 13:09:25.982052 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:25.982380 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:25.982407 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:25.982801 (+   394us) log.cc:844] Serialized 26 byte log entry
0709 13:09:27.074582 (+1091781us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b18003b3440. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb749a34bc 00007fcb71cf148b 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1
0709 13:09:27.074935 (+   353us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:27.075008 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.075111 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:27.075152 (+    41us) raft_consensus.cc:1685] finished
0709 13:09:27.075163 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.075422 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1687,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":13592,"spinlock_wait_cycles":4317056,"thread_start_us":22519,"threads_started":1,"wal-append.queue_time_us":52206}
I20260709 13:09:27.106333 29031 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.129205 29031 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.136458 28992 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:27.149004 29121 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.149246 29132 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:27.149713 29132 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:27.150030 29131 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.150723 29131 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.151288 29132 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.152684 29132 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:27.149739 29121 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.168208 29146 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.175729 28978 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.169502 29146 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.176695 28978 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.179579 29200 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.181916 28978 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.030514 22905 rpcz_store.cc:276] 0709 13:09:26.014508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.081105 (+ 66597us) service_pool.cc:224] Handling call
0709 13:09:26.081505 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:26.219350 (+137845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.024866 (+805516us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54000596d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:27.025140 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314670848}
I20260709 13:09:27.189148 22747 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:09:27.196987 22747 pending_rounds.cc:85] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 10
I20260709 13:09:27.179639 29146 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.210738 22747 pending_rounds.cc:107] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:09:27.080129 28773 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:27.220672 22747 raft_consensus.cc:2889] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:27.187055 29061 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:27.235033 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.239303 28773 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:27.240330 29185 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.241850 29024 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.236860 28563 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:27.235596 29015 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.264529 29120 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:27.269706 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.271315 29031 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.272190 29131 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.270884 29323 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:27.271533 29321 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:27.342666 22899 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.006s	sys 0.010s
W20260709 13:09:27.343256 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 176620us
I20260709 13:09:27.345469 28696 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:27.354045 23444 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:27.349530 29120 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.358486 29295 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:27.360183 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.360621 29120 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.360951 23450 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:27.362097 29266 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:27.362576 29266 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:27.364205 29266 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.367126 28696 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.369398 29185 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:27.371464 28923 consensus_meta.cc:287] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.017s
I20260709 13:09:27.373402 29121 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:27.381294 22899 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.007s	sys 0.010s
I20260709 13:09:27.389091 23447 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:27.383713 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.270387 29258 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:27.369516 28953 ts_tablet_manager.cc:1403] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 3.187s	user 0.053s	sys 0.053s
I20260709 13:09:27.394877 23452 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:09:27.378214 28925 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.396704 22897 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:09:27.392067 23451 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:09:27.404537 29300 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.408200 29325 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.408833 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:27.409404 29325 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.412597 29325 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.412415 28953 raft_consensus.cc:359] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:27.412954 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.410374 22754 raft_consensus.cc:2410] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:09:27.405397 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:27.413539 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:27.396649 28925 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:27.430833 22719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.026s	sys 0.018s
W20260709 13:09:27.431428 22719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.027s	sys 0.018s
I20260709 13:09:27.431967 29258 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:27.435163 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.436528 22749 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:09:27.436987 22749 pending_rounds.cc:85] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 9
I20260709 13:09:27.437227 22749 pending_rounds.cc:107] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:09:27.437485 22749 raft_consensus.cc:2889] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:27.439845 28992 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:27.440429 28992 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.441789 29258 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.440321 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:27.443233 22755 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:27.443611 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.444186 29061 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:27.432008 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 947345us
I20260709 13:09:27.440960 28992 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:27.446946 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:09:27.451139 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.460412 29121 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.461031 29121 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.461737 28878 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.462491 28878 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:27.462867 28878 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.463228 28878 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:27.464078 29121 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.467087 29061 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:27.468997 29300 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:27.440456 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:27.471338 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.471797 29264 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:27.472317 29264 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:27.472800 29264 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:27.473181 29295 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:27.473744 29295 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:27.479146 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.479923 22752 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:27.481000 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.479557 22650 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 13 to 15, pending config changed. New cstate: current_term: 15 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } } pending_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:27.427776 29266 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:27.404522 29260 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:27.483177 29313 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.591s	user 0.001s	sys 0.002s
I20260709 13:09:27.423375 28953 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:09:27.488214 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.497045 23452 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:27.495460 22754 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:27.506304 22755 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:27.509011 28990 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.510427 28990 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.512876 29301 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:27.513875 29301 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.513628 28953 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:27.516505 29305 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:27.516731 29247 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:27.517200 29305 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.517511 29247 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.519323 29305 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.519546 29247 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.520390 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.520488 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.521885 22756 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:09:27.521801 29015 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:27.522642 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.523623 22896 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:27.483166 28992 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.525071 29305 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:27.525907 29305 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:27.519925 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.527052 28992 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.527346 22904 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:27.528062 29305 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.519565 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.529278 22903 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
W20260709 13:09:27.529682 29299 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.663s	user 0.010s	sys 0.005s
I20260709 13:09:27.530260 29139 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:27.530184 29299 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.530647 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.531322 22901 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:27.532166 29262 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:27.532480 29299 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.522670 29015 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.535732 29015 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.483784 29323 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.539608 29323 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.539644 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.539922 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.540648 23446 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:27.541498 29247 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.533182 28990 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.542173 29247 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:27.543224 29261 consensus_meta.cc:287] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.675s	user 0.003s	sys 0.011s
I20260709 13:09:27.544322 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:27.544782 29319 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.659s	user 0.001s	sys 0.000s
I20260709 13:09:27.546875 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.557147 22754 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:27.569543 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.400399 29321 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:27.547930 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.549130 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.509364 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.549841 29251 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.551268 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.571851 29251 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.572464 22828 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:27.572580 22750 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:09:27.573505 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:27.574168 22748 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:27.574781 29251 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.551537 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.551930 28696 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:27.553776 22753 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:09:27.557713 22756 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:27.557756 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.558420 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.537038 29132 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:27.562026 22747 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:27.466795 29121 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.034s
I20260709 13:09:27.568578 23446 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:27.579373 23447 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:27.516893 29277 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.581971 29061 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:27.582257 22829 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:27.571915 23453 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 12: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:27.582909 29314 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:27.575667 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.577052 22824 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:27.575307 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.581753 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.577529 22753 pending_rounds.cc:85] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 12
I20260709 13:09:27.584401 22753 pending_rounds.cc:107] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:09:27.584693 22753 raft_consensus.cc:2889] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:27.574927 28925 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:27.577492 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 71350.9us
I20260709 13:09:27.581521 28978 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.589121 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.580237 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:27.581820 28696 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:27.585093 22756 pending_rounds.cc:85] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 9
I20260709 13:09:27.593243 22756 pending_rounds.cc:107] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:09:27.516803 29301 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:27.593643 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.593699 22756 raft_consensus.cc:2889] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:27.603577 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.575330 28773 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:09:27.604251 23452 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:27.582954 29061 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:27.577723 29015 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:27.606688 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.582340 29277 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.583479 29314 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:27.618880 22903 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:27.567339 29325 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:09:27.623749 22749 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 23: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:27.628552 29120 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.635118 29120 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.637040 29120 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.638456 29247 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.639034 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:27.639357 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.620639 29061 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:27.640733 29302 consensus_meta.cc:287] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.772s	user 0.032s	sys 0.000s
W20260709 13:09:27.641297 29328 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:09:27.645222 23447 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 17 LEARNER]: Advancing to term 18
I20260709 13:09:27.646231 29321 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:27.585778 29261 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.647945 28878 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.674062 28563 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:09:27.660279 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:27.624126 29301 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:27.666432 22829 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:27.627774 29314 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:27.649509 22650 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 8 to 10, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 10 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:27.668845 23453 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:09:27.672780 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:27.647025 28925 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:27.674358 22828 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:27.675395 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.594497 29185 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:09:27.674235 22825 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:27.651073 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.703645 22824 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:09:27.620527 29132 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.693830 23446 raft_consensus.cc:2410] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 22 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:09:27.678936 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:27.699839 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.600675 28990 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:27.713588 22755 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:27.656571 28978 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.717005 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:27.673668 29015 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.729681 29321 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:27.730005 22897 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:09:27.730185 29321 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:27.731216 29015 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:27.731351 29321 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.732477 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.732816 29321 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:27.740401 22749 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:27.695165 29301 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.754848 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:27.752985 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.674408 29302 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.769588 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.770274 28773 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:27.771402 22823 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:27.771972 29276 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:27.771430 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.764922 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.772588 22828 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:09:27.773242 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:27.826293 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.775599 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.840662 23445 raft_consensus.cc:2368] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18.
I20260709 13:09:27.766317 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:27.841306 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66476.6us
I20260709 13:09:27.754667 29301 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:27.775902 22828 pending_rounds.cc:85] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 12
I20260709 13:09:27.848048 22828 pending_rounds.cc:107] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:09:27.848392 22828 raft_consensus.cc:2889] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:27.749234 29015 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.712643 28925 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.856335 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.857002 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:27.857597 22755 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:27.858994 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:27.859597 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.860276 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:27.861019 23453 raft_consensus.cc:2368] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:27.861452 22830 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:09:27.861806 22830 pending_rounds.cc:85] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 12
I20260709 13:09:27.862011 22830 pending_rounds.cc:107] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:09:27.862226 22830 raft_consensus.cc:2889] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:27.863520 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.864464 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.869519 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:27.872922 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 199884us
I20260709 13:09:27.771497 29083 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.178s
I20260709 13:09:27.840354 28773 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:27.874783 29265 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:09:27.887034 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.887650 23453 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:27.888423 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.888955 23453 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:27.889573 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:27.889791 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.890182 22822 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:27.890337 23453 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 10.
I20260709 13:09:27.890784 28773 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:27.893986 28895 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:27.894366 22651 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 336) took 20589 ms (client timeout 15000 ms). Trace:
I20260709 13:09:27.895279 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:27.894956 28895 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.879377 23419 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:27.896773 29265 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:27.897150 29265 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.897544 29265 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:27.897500 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:27.909585 22749 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:27.869988 29262 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:09:27.921998 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.901490 28696 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:27.851116 28990 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.911559 28563 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:27.926430 28696 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.959291 22822 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:27.959291 22823 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:09:27.970158 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:27.971971 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 74653.7us
I20260709 13:09:27.919852 29015 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:27.925733 29262 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.974545 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:27.908787 29061 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:27.960660 28696 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:27.975261 22897 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:28.005453 22822 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:27.984344 29061 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:27.971539 29277 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.013679 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:28.016170 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:28.016800 29189 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:28.017509 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:27.894829 22651 rpcz_store.cc:269] 0709 13:09:07.304978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:07.305245 (+   267us) service_pool.cc:224] Handling call
0709 13:09:08.005874 (+700629us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54000ab4d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d65090 00007fcb74e03f3a 00007fcb7abba1cc 00007fcb7acb837e 00007fcb791792a4 00007fcb79179232 00007fcb79179161 00007fcb791790dc 00007fcb79178392 00007fcb7313648c 00007fcb731358c6
0709 13:09:27.894073 (+19888199us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:18.383860 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:18.569517 (+185657us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:18.569585 (+    68us) write_op.cc:435] Acquired schema lock
0709 13:09:18.569613 (+    28us) tablet.cc:662] Decoding operations
0709 13:09:18.830379 (+260766us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:18.830451 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:18.830479 (+    28us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:09:18.831948 (+  1469us) tablet.cc:701] Row locks acquired
0709 13:09:18.831997 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:09:18.832162 (+   165us) write_op.cc:270] Start()
0709 13:09:18.832300 (+   138us) write_op.cc:276] Timestamp: P: 1783602558832098 usec, L: 0
0709 13:09:18.832321 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:09:18.833080 (+   759us) log.cc:844] Serialized 8474 byte log entry
0709 13:09:20.404162 (+1571082us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:20.660579 (+256417us) op_driver.cc:464] REPLICATION: finished
0709 13:09:21.074063 (+413484us) write_op.cc:301] APPLY: starting
0709 13:09:21.106426 (+ 32363us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:22.204015 (+1097589us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b180002a6a0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb74b3a0ec 00007fcb74b3caf7 00007fcb74b3dfb0 00007fcb788521bb 00007fcb78852532 00007fcb78899b93 00007fcb78746de3 00007fcb787466ab 00007fcb78747e77
0709 13:09:23.471297 (+1267282us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:23.471341 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:24.018212 (+546871us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:24.058118 (+ 39906us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:09:24.058163 (+    45us) write_op.cc:312] APPLY: finished
0709 13:09:24.058586 (+   423us) log.cc:844] Serialized 188 byte log entry
0709 13:09:24.580188 (+521602us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb71cf10ac 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b
0709 13:09:25.035534 (+455346us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:25.054485 (+ 18951us) write_op.cc:454] Released schema lock
0709 13:09:25.088944 (+ 34459us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:25.310230 (+221286us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:25.318036 (+  7806us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":200854,"reported_tablets":44,"spinlock_wait_cycles":1181453184,"child_traces":[["op",{"apply.queue_time_us":267472,"apply.run_cpu_time_us":61571,"apply.run_wall_time_us":4129027,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91907,"num_ops":21,"prepare.queue_time_us":134887,"prepare.run_cpu_time_us":42550,"prepare.run_wall_time_us":622104,"raft.queue_time_us":141817,"raft.run_cpu_time_us":19983,"raft.run_wall_time_us":299911,"replication_time_us":1793391,"spinlock_wait_cycles":96604928,"thread_start_us":1385013,"threads_started":5,"wal-append.queue_time_us":1423347,"wal-append.run_cpu_time_us":44280,"wal-append.run_wall_time_us":3599821}]]}
I20260709 13:09:28.018677 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:09:28.019275 29338 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:09:28.020906 28953 ts_tablet_manager.cc:1434] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.616s	user 0.011s	sys 0.000s
W20260709 13:09:28.021231 28953 ts_tablet_manager.cc:1467] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Tablet startup took 623ms
I20260709 13:09:28.021605 29277 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:28.022179 29277 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:28.023164 29277 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.017552 28978 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.026798 28978 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:28.027315 28978 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:28.028250 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.029817 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.015411 29131 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.032940 29189 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:28.033989 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:28.034507 29164 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:09:28.070196 22897 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:28.034896 28990 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.035233 29251 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:28.036535 28925 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:28.037508 29302 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:28.038633 29015 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:27.982524 28895 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:28.040676 29266 consensus_meta.cc:287] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.539s	user 0.007s	sys 0.001s
I20260709 13:09:28.041419 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1307) took 1812 ms. Trace:
I20260709 13:09:27.981979 28878 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:27.980484 22650 catalog_manager.cc:5697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.045167 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.047168 28978 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.048852 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.052740 29305 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:09:28.056777 29337 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:09:28.025455 29339 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:28.058609 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.059437 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:28.060801 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.061061 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.064167 29024 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:27.984850 29132 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.015156 29139 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
W20260709 13:09:28.072381 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 322461us
I20260709 13:09:28.085855 22747 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:09:28.086712 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.103368 22871 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.102998 22650 catalog_manager.cc:5697] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:28.115207 22650 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.105106 29189 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:28.108479 29061 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:28.108858 28563 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:28.109208 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.100944 22901 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:28.117115 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:28.131064 22751 raft_consensus.cc:2410] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:09:28.132668 22871 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.087918 22827 rpcz_store.cc:276] 0709 13:09:26.105705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.118254 (+ 12549us) service_pool.cc:224] Handling call
0709 13:09:26.118520 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:26.239870 (+121350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:26.239900 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:26.241169 (+  1269us) log.cc:844] Serialized 26 byte log entry
0709 13:09:27.010519 (+769350us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b18001ecee0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:09:27.240192 (+229673us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:27.240287 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.541944 (+301657us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54003813d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:27.541992 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:27.917522 (+375530us) raft_consensus.cc:1685] finished
0709 13:09:27.917543 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:27.917676 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":163,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":6289,"spinlock_wait_cycles":282574464,"thread_start_us":200894,"threads_started":1,"wal-append.queue_time_us":1133237}
I20260709 13:09:28.134357 22650 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 18 to 19, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
W20260709 13:09:28.144471 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 110645us
I20260709 13:09:28.145229 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.130515 28563 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.100530 29131 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.113240 29265 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.115319 29339 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.168198 22650 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:28.178298 22650 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 15 to 17, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.179643 29339 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.180191 22650 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:28.180830 29339 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:28.173516 29015 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:28.181725 29015 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:28.182129 29015 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:28.183609 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.150548 28563 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:28.190196 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.198390 23451 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:28.208231 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.146368 29061 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.230139 22747 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:28.147753 29024 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.237715 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:28.239945 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.245173 28925 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:28.245728 28925 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:28.246713 28925 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:28.021605 28953 ts_tablet_manager.cc:1469] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Trace:
0709 13:09:16.713670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.720222 (+  6552us) service_pool.cc:224] Handling call
0709 13:09:16.890976 (+170754us) ts_tablet_manager.cc:934] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
0709 13:09:20.252213 (+3361237us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:20.452036 (+199823us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:20.457902 (+  5866us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:20.479396 (+ 21494us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:20.479699 (+   303us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:20.480000 (+   301us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:20.480057 (+    57us) inbound_call.cc:177] Queueing success response
0709 13:09:24.182351 (+3702294us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:09:27.405262 (+3222911us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:09:27.408560 (+  3298us) tablet_replica.cc:261] Starting instrumentation
0709 13:09:27.409740 (+  1180us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":369,"raft.queue_time_us":38559,"spinlock_wait_cycles":35969920,"tablet-copy.queue_time_us":46902,"tablet-download-pool.queue_time_us":97788,"tablet-download-pool.run_cpu_time_us":1652,"tablet-download-pool.run_wall_time_us":2886839,"thread_start_us":313213,"threads_started":2}
I20260709 13:09:28.198925 28773 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.256903 28773 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.258214 29321 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.274255 29131 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.274014 29321 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.276427 29321 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:28.276510 29024 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.259519 29258 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.278193 29258 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.279976 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:28.280514 29258 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:28.261809 29264 consensus_meta.cc:287] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.788s	user 0.006s	sys 0.006s
I20260709 13:09:28.281211 29264 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:28.282415 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.282791 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:28.236609 29295 consensus_meta.cc:287] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.760s	user 0.008s	sys 0.005s
I20260709 13:09:28.283354 22822 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:28.283970 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.284608 22747 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
W20260709 13:09:28.286559 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 110455us
I20260709 13:09:28.287431 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:28.262274 22650 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 8 to 12, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 12 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.289752 29301 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.290319 22650 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:28.291879 29120 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:28.292346 22896 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 24: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:28.283290 29264 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:28.292971 29323 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:28.292634 29120 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.293411 29264 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:28.293563 29323 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.280676 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.290817 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.295006 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.295156 22823 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:28.295749 29323 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.279925 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.296113 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.296283 29323 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.297257 29265 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:28.297346 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.298049 22831 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:28.298569 22823 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:09:28.298841 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.299384 22831 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:28.299616 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.300168 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.300400 22719 leader_election.cc:400] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:28.300745 22719 leader_election.cc:403] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:09:28.301667 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.302815 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.302919 29305 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:28.303331 29305 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:28.303831 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.304401 22824 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:28.305627 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.305927 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:28.306222 22826 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:28.305653 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:28.306972 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.307193 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:28.307574 22826 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:28.308401 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.307334 22751 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:09:28.308884 22826 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:28.309597 29120 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.310361 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.310657 29261 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:28.311550 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.312175 22828 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:28.301689 29247 consensus_queue.cc:1059] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:09:28.313917 29247 raft_consensus.cc:3055] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:28.314312 29247 raft_consensus.cc:740] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:28.314363 29262 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:28.291932 22650 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 16 to 17, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:28.315192 29247 consensus_queue.cc:260] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.315768 22650 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:28.316385 29247 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:28.305143 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.317747 22824 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:28.301182 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.320996 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:28.322568 29339 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:28.268460 29061 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.303267 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.324456 28992 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.325137 23452 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:28.297492 28895 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:09:28.325910 29325 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:28.326169 29301 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.326663 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.283533 29295 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.304579 29305 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.329855 29341 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:28.296885 29323 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.300635 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.323583 23419 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.317045 22650 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 14 to 15, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 15 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:28.318331 28696 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.336369 22751 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:28.338572 29139 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:09:28.338557 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 319867us
I20260709 13:09:28.339004 28992 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.346117 28563 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.357149 23446 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:28.338762 29167 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:28.356127 23419 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:28.352387 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 211683us
I20260709 13:09:28.357874 22827 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:28.355420 22650 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 14, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 14 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:28.370217 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.356889 29083 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:28.351013 29258 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:28.376617 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.376722 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.377765 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:28.382035 29269 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.026s	sys 0.012s
W20260709 13:09:28.382403 29269 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.026s	sys 0.012s
I20260709 13:09:28.388903 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:28.375514 29314 consensus_meta.cc:287] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.001s
W20260709 13:09:28.395888 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 92779.5us
I20260709 13:09:28.393360 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.399817 22794 leader_election.cc:400] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18.
I20260709 13:09:28.400255 22794 leader_election.cc:403] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:28.392143 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.396852 22797 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:28.403672 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.396647 22650 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.374444 29321 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:28.411970 28696 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.413343 29015 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:28.415472 29015 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:28.417225 29305 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:28.421456 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 100569us
I20260709 13:09:28.422375 29295 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:28.422912 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:28.424497 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.378688 29083 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.425300 29083 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:28.429454 22896 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:28.430419 29350 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.369730 29341 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.430701 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:28.431241 29350 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:28.432541 29341 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.433188 29030 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:28.409255 29139 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:28.433903 29139 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.434262 29139 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:28.435432 28563 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:28.435827 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1632) took 1050 ms. Trace:
I20260709 13:09:28.436223 23444 rpcz_store.cc:276] 0709 13:09:27.385310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.394089 (+  8779us) service_pool.cc:224] Handling call
0709 13:09:27.478257 (+ 84168us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:27.605307 (+127050us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:27.605356 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:27.606574 (+  1218us) log.cc:844] Serialized 26 byte log entry
0709 13:09:27.640009 (+ 33435us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b18001d1640. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb78ccf58f 00007fcb71d0e571 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd
0709 13:09:27.966550 (+326541us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b18001d1640. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:09:28.197708 (+231158us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:28.197746 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:28.197818 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:28.435628 (+237810us) raft_consensus.cc:1685] finished
0709 13:09:28.435649 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:28.435765 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18724,"raft.queue_time_us":30904,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4317,"spinlock_wait_cycles":22959616,"thread_start_us":230962,"threads_started":1,"wal-append.queue_time_us":773226}
I20260709 13:09:28.437573 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.376521 29258 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.438166 29258 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:28.409282 29314 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.439417 22797 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:28.440491 22902 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:28.440513 29314 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.444520 29348 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:28.445508 29348 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:28.446334 29348 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:28.447527 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:28.448243 22749 raft_consensus.cc:2393] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 23: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:28.448948 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.449728 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.450045 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.409103 29189 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:28.449887 29277 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:09:28.432971 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.449720 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.449720 22749 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:28.454797 22898 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:28.455889 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:28.456050 29350 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:28.457073 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.456387 29262 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:28.477397 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:28.474759 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 196356us
I20260709 13:09:28.477896 22650 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 17 to 19, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 19 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.476366 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.484412 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.484962 23452 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:28.486445 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:28.486915 23452 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:28.404685 28992 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:28.490036 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.492303 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.493644 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:28.494159 23444 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:28.492890 22896 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:28.511924 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:28.512038 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.498150 22650 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 12, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:28.512962 23452 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:28.463433 28563 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:28.507714 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.519140 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.527594 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.528235 22897 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:28.490931 28990 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:28.474939 29321 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:28.477556 28895 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:09:28.527796 29360 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:09:28.513928 22650 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:09:28.555387 29347 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:09:28.558393 22868 leader_election.cc:336] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.666s (SENT)
I20260709 13:09:28.474901 29185 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
W20260709 13:09:28.597465 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 164558us
I20260709 13:09:28.559119 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.598879 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.586792 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.589906 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:28.602391 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 110337us
I20260709 13:09:28.584410 22903 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:28.560994 22898 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:28.585228 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:28.589576 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 76502.7us
I20260709 13:09:28.605346 22897 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:28.607424 22901 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:28.613205 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:28.606454 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:28.608047 22900 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:28.611097 22899 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:28.614799 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:28.590060 29338 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:09:28.626263 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67892.5us
I20260709 13:09:28.627252 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.628278 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:28.629451 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.630012 22901 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:28.630517 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.631132 23447 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 10.
I20260709 13:09:28.633038 23447 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:09:28.633414 29362 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:28.634522 29121 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:28.635124 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.635725 23452 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:28.511660 29131 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:28.518888 29251 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:28.520319 28563 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.604700 28990 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.635667 29349 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:28.640748 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.641906 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.643007 29362 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:28.643240 23450 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:28.643841 22914 tablet_copy_service.cc:342] P 3b626c41600d4737a178ebab8a7ebfc2: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-18a127a20cfa4496bad51cb634bd0d0f received from {username='slave'} at 127.0.0.1:42490
I20260709 13:09:28.582439 29189 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:28.639467 23444 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:09:28.649569 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.652210 22747 raft_consensus.cc:2410] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:09:28.644577 22650 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:28.665555 22650 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 13 to 14, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 14 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:28.657287 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.649722 29030 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:28.657588 22901 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:28.657694 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.656499 22797 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.651721 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.661844 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.667959 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.664335 23448 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:28.644550 29251 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.675565 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.676074 22896 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:28.650116 28563 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:28.676892 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.655833 28990 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:28.678104 22898 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:28.615458 29258 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.666601 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.677816 22905 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 25.
I20260709 13:09:28.641014 29164 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:09:28.680438 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.697712 29305 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:28.697999 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:28.698776 29340 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:28.699131 29340 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.699290 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:28.686748 29342 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:28.700325 29342 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:28.678702 29362 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:28.701159 29342 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.702019 29362 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 22.20, Last appended by leader: 20, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.704344 29262 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:28.654028 29323 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.681138 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.688102 29355 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.705885 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.706508 22897 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 12.
I20260709 13:09:28.691013 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.666016 29189 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:28.717479 28992 raft_consensus.cc:2764] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:09:28.718736 29321 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.720347 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:28.721558 28925 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:28.722114 28925 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:28.722962 28925 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.694839 23449 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:28.695495 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.695991 29325 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:28.699466 29340 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:28.683209 22900 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:09:28.690413 22871 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.672654 22914 tablet_copy_service.cc:434] P 3b626c41600d4737a178ebab8a7ebfc2: ending tablet copy session c45e1805f049467697a7358fb566de2f-18a127a20cfa4496bad51cb634bd0d0f on tablet 18a127a20cfa4496bad51cb634bd0d0f with peer c45e1805f049467697a7358fb566de2f
W20260709 13:09:28.738620 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48298.3us
I20260709 13:09:28.704775 29262 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.743027 29262 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:28.743225 29269 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:28.743633 29269 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:28.744539 29269 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:28.745426 23451 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:28.745564 29269 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:28.746151 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.746477 29266 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:28.747078 29266 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:28.686512 29015 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:28.747761 29015 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:28.747898 28696 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:28.748245 29015 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:28.747931 29266 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.673197 29030 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.753507 29030 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:28.679973 29251 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:28.713758 29355 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.710551 28978 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:28.714898 23452 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:09:28.706441 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:28.759341 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:28.759676 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 53301.8us
I20260709 13:09:28.760166 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.760705 22897 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:28.635733 29139 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.747305 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.748386 22905 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:28.765112 22900 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:28.765355 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:28.714511 29083 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.769474 22903 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:28.757692 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.763652 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.758522 29167 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.771971 22901 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:28.782220 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 123388us
I20260709 13:09:28.788004 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.788086 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:28.788828 23446 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:28.790607 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:28.817366 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:28.792702 22899 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:28.768014 29164 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:28.844660 29164 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:28.845103 29164 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:28.845698 29083 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:28.849429 22795 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.21.31.126:35001, user_credentials={real_user=slave}} blocked reactor thread for 56934.9us
I20260709 13:09:28.850240 29139 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.852277 29277 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18.
W20260709 13:09:28.854063 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 83120.7us
I20260709 13:09:28.857486 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:28.758673 28978 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.861020 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:28.862257 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.785012 29261 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:28.780817 28992 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:28.867708 28978 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:28.787588 29325 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:28.872351 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:28.873503 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:28.884397 22751 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
W20260709 13:09:28.885998 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 229539us
I20260709 13:09:28.888017 22901 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:28.888774 22897 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:28.888942 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:28.872931 29185 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:28.868650 29131 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:28.868469 29305 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.873339 29261 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.906922 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.907323 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.907490 22826 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:28.892143 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.902465 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:28.875447 29258 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:28.873584 28992 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:28.908103 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.789036 29167 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.915431 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.886905 29132 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:28.915673 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.916232 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:28.893023 29277 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:28.917388 22898 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:09:28.919937 22823 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:28.868818 29323 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:09:28.925985 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:28.926162 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:28.890396 29349 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:28.919934 22822 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:28.926599 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:28.927157 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.890199 29139 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:28.903352 29131 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.929524 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.944747 22825 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:28.914201 29261 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:28.914179 28992 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:28.948668 22900 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:28.953689 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:28.954236 22756 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:28.930572 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.893352 28895 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:09:28.928398 29132 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.892565 29189 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.959060 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.955714 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.961447 22901 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:28.965536 22755 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:09:28.934466 29185 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:28.965921 22755 pending_rounds.cc:85] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 5
I20260709 13:09:28.966189 22755 pending_rounds.cc:107] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:09:28.966456 22755 raft_consensus.cc:2889] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:28.972658 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:28.949241 29131 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:28.974035 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:28.977216 22904 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:28.977783 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:28.957890 29321 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:28.986071 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46839.1us
I20260709 13:09:28.961575 28878 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:28.961766 29132 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:28.933732 29305 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.960938 29083 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:28.932390 29349 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.971902 29185 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:28.999709 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.988004 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.999603 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:28.938611 29258 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:09:28.949913 29277 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.010437 22828 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:28.970978 29139 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:28.972160 29300 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:09:28.994024 28878 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:29.002889 29342 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:28.989415 29341 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 13:09:29.016844 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 153910us
I20260709 13:09:28.988657 29061 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:09:28.994174 29339 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:09:29.004560 29185 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:29.032516 29261 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.036332 29261 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.036584 29167 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.038997 22899 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:29.015552 28696 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:29.018153 28878 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:29.070930 22823 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:29.018436 29323 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:09:29.074306 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50454.7us
I20260709 13:09:29.076053 29379 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:29.076645 29379 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:29.005940 29015 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.015349 29325 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:09:29.094347 22749 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:09:29.098063 22898 raft_consensus.cc:1240] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:29.099491 22898 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:09:29.085755 29373 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:09:29.104430 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.106761 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.107467 22749 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:29.112592 22747 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:09:29.113427 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.114292 22755 raft_consensus.cc:2368] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:29.121775 29383 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:29.097784 29382 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:29.134995 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:29.136175 28563 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:29.132460 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 144552us
I20260709 13:09:29.144981 22827 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:29.137670 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:29.149772 23453 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:29.150135 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.153052 22749 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:09:29.136274 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:29.150992 22747 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:29.156489 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.156848 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.157517 22823 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:29.159988 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.104475 29369 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:09:29.161671 23416 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.160494 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.104413 29030 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.165055 23447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f881fed251c4393ac053086bd572549"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:29.165820 23447 raft_consensus.cc:606] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.166121 23447 raft_consensus.cc:614] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:29.166397 29030 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.162014 28563 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:29.170316 29374 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:09:29.146353 29383 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:29.146588 29382 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:29.172075 29382 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:29.172465 29277 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.177714 29189 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.178221 29349 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.178721 29139 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.132453 28696 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:29.177155 22756 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:29.180469 23446 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:09:29.173158 22896 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:29.217527 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.252995 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.156009 29167 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:29.218221 22831 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:29.254565 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.256606 22823 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:29.257894 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.258584 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.220201 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.242498 29321 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:29.219698 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:09:29.260828 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 261340us
I20260709 13:09:29.262925 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.260871 22822 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:29.162002 29384 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:29.266647 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:29.267976 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 50109.6us
I20260709 13:09:29.215616 29341 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:29.270555 29323 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:29.274003 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.274616 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.242682 29083 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:09:29.275775 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 78643.8us
I20260709 13:09:29.260905 29305 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.279062 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.281405 22822 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
W20260709 13:09:29.272944 29262 consensus_meta.cc:287] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 13:09:29.280081 22754 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:29.218932 29251 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.284754 22754 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:29.267046 29167 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:29.215917 29132 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:29.286821 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:29.290011 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:29.273455 29384 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:29.296525 29384 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:29.296842 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:29.297569 22896 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:29.298270 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:29.298727 22896 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:29.301347 29388 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:29.301867 29388 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.303809 29388 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.271813 29030 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
W20260709 13:09:29.287571 28990 consensus_meta.cc:287] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.001s
W20260709 13:09:29.304342 29370 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:09:29.304931 29390 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:29.304730 28990 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.306190 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.306861 28990 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.307657 22827 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 24: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:29.309497 29164 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:29.292320 29015 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.292044 28696 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.311640 29164 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.292572 22898 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:09:29.276870 29323 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:29.281780 29350 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:29.294900 29382 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:29.306126 29387 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:09:29.322504 22754 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.317111 22896 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:29.319154 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.292428 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:29.295542 29269 consensus_meta.cc:287] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.550s	user 0.004s	sys 0.004s
I20260709 13:09:29.276808 29321 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:29.345481 22750 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:09:29.346472 29167 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.288635 29262 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.350888 28978 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:29.352437 23451 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:09:29.324199 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.324424 22823 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:29.336407 22825 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:29.335659 22905 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:29.278978 29341 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.341286 29251 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.341544 29132 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:29.341660 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 75090.4us
I20260709 13:09:29.344893 29321 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.337813 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:29.354920 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.352973 22754 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:29.371388 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.358726 22828 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:29.314833 29083 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:29.325762 29323 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:29.339108 29388 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:09:29.364223 29340 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.000s
W20260709 13:09:29.369671 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 232026us
I20260709 13:09:29.325084 29350 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:29.372180 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 89451.5us
I20260709 13:09:29.373950 22747 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:29.350895 29295 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:29.361609 22830 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:29.377003 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 1977) took 1023 ms. Trace:
I20260709 13:09:29.391217 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:29.388090 29350 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:29.394116 29262 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.394999 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:29.395771 23449 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 24: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:29.395627 29396 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:09:29.396512 29396 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:29.396934 29396 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:29.397650 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:29.398202 23447 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:29.394063 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:29.411356 23447 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 11.
I20260709 13:09:29.412055 23452 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:29.390281 22751 rpcz_store.cc:276] 0709 13:09:28.353386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:28.353498 (+   112us) service_pool.cc:224] Handling call
0709 13:09:29.376862 (+1023364us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540032a854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:29.376953 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865184640}
I20260709 13:09:29.333685 28992 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.415130 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:29.425235 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 135287us
I20260709 13:09:29.430543 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:29.427850 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.451064 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:29.450021 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:29.451715 23450 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:29.408700 29350 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:29.455227 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.452359 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:29.457271 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.458312 29083 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.462834 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:29.459484 28878 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.389448 29295 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:09:29.462884 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.389650 29388 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:29.459597 28992 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.387939 29121 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 13:09:29.467825 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.468842 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.469416 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.388702 29340 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:29.484628 22756 raft_consensus.cc:2368] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
I20260709 13:09:29.485188 22822 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:29.493535 29378 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:29.493937 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.488232 23419 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:29.501459 29399 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:29.502595 28878 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.503405 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.465648 29164 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:29.498370 28978 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.507498 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.508433 22824 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:29.508800 22829 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:29.466993 29390 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:29.509406 29295 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.510376 29388 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.487612 29167 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:29.471809 28990 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
W20260709 13:09:29.505247 29397 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:09:29.516606 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.469290 29305 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:09:29.517827 22747 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:29.516908 29167 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:29.519943 29185 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.484218 29396 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.527565 29382 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.487149 29383 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.539139 22753 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:29.498811 29030 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:09:29.493961 29378 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:29.534014 22822 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:29.543900 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:29.545974 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:29.547557 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.545689 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.545976 29391 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:09:29.489190 29384 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.550565 29167 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:29.563057 29402 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:09:29.564069 29131 consensus_meta.cc:287] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.002s
I20260709 13:09:29.544869 29378 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:29.574823 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:29.513705 29406 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:09:29.579219 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:29.573500 29185 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.581688 29340 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.582546 29341 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.584702 29167 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.587102 29061 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:29.587495 29061 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:29.587996 29061 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:29.576709 29402 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:29.588989 29402 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:29.591394 29384 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.574162 29396 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:29.602329 29355 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.615339 29323 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.616463 22749 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:29.618709 29323 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.623885 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:29.624713 29410 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.101s	user 0.003s	sys 0.000s
I20260709 13:09:29.625401 29131 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:29.626891 29300 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:29.627058 28895 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:09:29.627427 29300 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.628661 29350 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:29.629542 22748 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:29.630496 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:29.626592 29061 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:29.639488 28895 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:29.640808 29417 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:09:29.627444 29131 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:29.641585 29323 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.642132 29323 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.642369 29355 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:29.642177 23445 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:09:29.641147 22905 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: []
W20260709 13:09:29.643371 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68672.2us
I20260709 13:09:29.640889 28895 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.647018 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:29.647449 29323 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.644418 29355 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:29.647892 28895 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.648280 29355 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:29.649605 29120 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:29.649998 29120 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:29.650424 29061 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:29.650913 29120 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.652652 29120 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:29.653466 29408 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.071s	user 0.002s	sys 0.000s
I20260709 13:09:29.653955 29300 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:29.548641 28925 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 13:09:29.665768 29355 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.658649 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:29.673127 22902 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:29.659821 22899 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:09:29.660332 22905 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:29.660602 22903 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:29.653955 29379 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:29.667812 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1648) took 1185 ms. Trace:
W20260709 13:09:29.681754 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 198144us
I20260709 13:09:29.681499 23453 rpcz_store.cc:276] 0709 13:09:28.482207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:28.482420 (+   213us) service_pool.cc:224] Handling call
0709 13:09:29.667612 (+1185192us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54000b13d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:29.667732 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1436514048}
I20260709 13:09:29.644240 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:29.685081 23447 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:09:29.687348 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.688274 23444 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
W20260709 13:09:29.689872 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 223251us
I20260709 13:09:29.698426 22797 leader_election.cc:400] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:29.700021 22797 leader_election.cc:400] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
I20260709 13:09:29.699939 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.700358 22797 leader_election.cc:403] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:29.703888 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.704983 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:29.715937 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.720710 22823 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:29.721261 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:29.716665 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.717677 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.717844 22754 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:29.729256 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:29.729684 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.730340 22899 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:29.731609 23448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:29.732048 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.732143 23448 raft_consensus.cc:606] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.732869 23448 raft_consensus.cc:614] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:29.734624 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.725579 22756 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:09:29.736061 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.736685 22756 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:29.737334 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.737839 22829 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:09:29.730943 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.743429 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:29.742616 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:29.754621 22755 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 11: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:29.755125 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:29.755452 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.756023 22822 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:29.756455 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.757480 22756 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:29.760623 22830 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:29.760632 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:29.714478 28992 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:29.719879 29266 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:09:29.771621 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:29.726043 29388 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:29.765743 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:29.719306 29379 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:29.719728 29262 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:29.769052 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:29.725010 29300 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:29.760032 22823 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:29.770576 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:29.782649 22897 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:29.776049 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:29.781914 29164 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:29.787101 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:29.781890 29300 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:29.788184 29341 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:29.788533 29300 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:29.788614 29341 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:29.789177 29341 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:29.789834 22825 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:29.793797 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:29.783427 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:29.800770 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:29.803625 22755 raft_consensus.cc:2368] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:29.725274 29277 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:09:29.812536 22897 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:29.738516 29349 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:09:29.728482 29402 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:29.738268 29321 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
W20260709 13:09:29.789837 29407 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:09:29.784037 29406 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:29.768013 29384 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:09:29.814076 29362 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:09:29.865636 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 138835us
I20260709 13:09:29.807313 29262 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:09:29.847412 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 203201us
I20260709 13:09:29.853101 23452 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:29.834080 29350 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:29.940639 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1720) took 1075 ms. Trace:
I20260709 13:09:29.800206 29340 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:09:29.943171 22896 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22. Ops: []
W20260709 13:09:29.954248 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 167252us
I20260709 13:09:29.943681 23446 rpcz_store.cc:276] 0709 13:09:28.865618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:28.867465 (+  1847us) service_pool.cc:224] Handling call
0709 13:09:28.878769 (+ 11304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:29.218071 (+339302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:29.940172 (+722101us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b5400672fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:29.940544 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693270912}
I20260709 13:09:30.007642 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:30.009454 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 171390us
W20260709 13:09:30.011109 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53934.5us
W20260709 13:09:30.014438 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.955078 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:29.873553 29362 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:29.834601 28696 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.140s
W20260709 13:09:30.035480 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 218968us
I20260709 13:09:30.038158 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:29.942485 29406 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:29.964514 29395 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:09:30.030191 29408 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:09:30.048991 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.055348 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1325) took 1148 ms. Trace:
I20260709 13:09:30.052634 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:30.057542 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 102557us
I20260709 13:09:30.058218 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.030586 29362 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:30.078053 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:30.075126 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:30.044961 29386 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:09:30.114254 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 83068.7us
I20260709 13:09:30.047935 29323 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:30.122758 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.027333 29390 consensus_queue.cc:1059] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:09:30.054704 29395 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:30.134512 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76360.8us
I20260709 13:09:30.068928 29083 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:30.135476 29416 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:09:30.135782 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.078118 29378 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.136837 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:30.059720 29348 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:30.073184 29401 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:30.137521 23446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:30.138079 23446 raft_consensus.cc:606] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:30.117997 29386 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:30.135263 29083 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.070052 29061 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:30.056166 22826 rpcz_store.cc:276] 0709 13:09:28.921379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:28.929900 (+  8521us) service_pool.cc:224] Handling call
0709 13:09:28.939400 (+  9500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:29.091037 (+151637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:29.091070 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:29.102062 (+ 10992us) log.cc:844] Serialized 26 byte log entry
0709 13:09:29.369220 (+267158us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:09:29.470165 (+100945us) log.cc:844] Serialized 14 byte log entry
0709 13:09:29.512399 (+ 42234us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b180048c940. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e522 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:29.875490 (+363091us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b180048c940. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0b455 00007fcb74df3060 00007fcb74e05716 00007fcb74e16287 00007fcb74e16208
0709 13:09:30.055008 (+179518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:30.055140 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:30.055177 (+    37us) raft_consensus.cc:1685] finished
0709 13:09:30.055187 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:30.055298 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80175,"prepare.queue_time_us":100427,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":2425,"raft.queue_time_us":219800,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":15576,"spinlock_wait_cycles":50131840,"thread_start_us":563103,"threads_started":4,"wal-append.queue_time_us":232807}
I20260709 13:09:30.141482 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.050233 29185 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:09:30.069201 29355 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.072345 29379 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:30.053961 29305 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 13:09:30.117373 28696 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:30.127365 29323 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:30.190526 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:30.192225 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55448.6us
I20260709 13:09:30.142712 29396 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:30.109062 29381 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:09:30.060290 28978 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 13:09:30.197184 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:30.198444 22829 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:09:30.147025 29061 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:30.224514 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:30.235720 29061 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.21, Last appended by leader: 21, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.232404 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:30.239240 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:30.238984 29341 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.186223 29269 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:30.158236 29314 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:30.241935 29314 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:30.242386 29314 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:30.150435 29015 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:30.185879 29321 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:30.194084 29406 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.233117 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:30.195204 29379 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:30.233309 29402 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:30.234637 29355 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:30.251296 29355 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.251368 29379 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.252892 22824 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:30.196184 29323 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:30.253309 29355 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:30.250761 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:30.248879 22825 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:30.255049 22829 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:09:30.237146 29061 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:30.257475 29415 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:30.261968 29269 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:30.262360 29269 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:30.262707 29323 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.263176 29269 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.264873 29323 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:30.124719 29262 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:30.266288 22902 consensus_meta.cc:287] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.003s
I20260709 13:09:30.266719 22902 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:30.267522 22899 raft_consensus.cc:1240] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: [15.8-18.9]
I20260709 13:09:30.283090 29083 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:30.283963 29386 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:30.284844 29266 consensus_meta.cc:287] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 13:09:30.285300 29386 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:30.285983 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.285921 29378 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:30.288224 29430 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
W20260709 13:09:30.290270 29411 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
W20260709 13:09:30.291466 29418 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:09:30.182575 28895 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:09:30.315454 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 240404us
I20260709 13:09:30.280352 29323 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:09:30.304661 29419 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:09:30.285281 29266 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17.
I20260709 13:09:30.274127 29120 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:30.327979 29120 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.319173 28978 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:09:30.329877 29120 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.319962 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:30.349728 22827 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
W20260709 13:09:30.363549 29413 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.520s	user 0.003s	sys 0.000s
W20260709 13:09:30.369709 29421 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.413s	user 0.000s	sys 0.001s
I20260709 13:09:30.380030 29434 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:30.380446 29434 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:30.318562 29323 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:30.318369 29412 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
I20260709 13:09:30.381690 29323 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:30.381376 29434 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.382683 29434 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:30.383219 29431 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:09:30.383456 28992 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.383962 28992 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.386067 28992 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:30.386811 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 66933.2us
I20260709 13:09:30.387497 29266 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:30.387799 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:30.388018 29266 raft_consensus.cc:697] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:30.389019 29266 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.390847 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:30.391664 29378 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.277698 29341 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:30.389046 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:30.394484 29378 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.399607 23453 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:30.413467 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:30.414265 22823 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:30.414564 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.397784 29412 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:30.418406 29429 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:30.418959 29429 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.420126 29395 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.421487 29301 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:30.422308 29391 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:30.422706 29391 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:30.423556 29391 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.424706 29391 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:30.426216 29378 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:30.415987 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.406558 29341 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:30.427654 29341 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:30.421120 29426 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
I20260709 13:09:30.429596 29341 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.427712 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:30.432227 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.433259 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:30.433874 23453 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:30.434468 29362 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.434787 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:30.435388 23453 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:30.435482 29425 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.241s	user 0.000s	sys 0.000s
I20260709 13:09:30.405943 29350 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:30.436126 29350 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.437353 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:30.437897 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.438021 23446 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:30.438956 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:30.439456 23446 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:30.439615 29384 raft_consensus.cc:3055] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:30.440001 29384 raft_consensus.cc:740] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:30.400141 28925 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.441583 28925 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.443004 23453 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:30.443471 28925 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:30.443588 29167 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.444011 28925 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:30.444082 29167 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.445853 23447 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 25. Current term is 26. Ops: []
W20260709 13:09:30.447450 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.447635 29391 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:30.447695 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:30.448382 29391 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:30.449434 23449 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:09:30.402055 29381 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:09:30.453881 29437 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:09:30.401955 28978 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:30.435560 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:30.464061 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:30.465489 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 55962.9us
I20260709 13:09:30.421020 29429 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.469687 22826 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:09:30.460992 23448 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:30.473946 23445 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:09:30.430114 29341 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:30.477802 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:30.463271 29015 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:30.483006 23445 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:09:30.485917 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.464920 29395 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:30.486898 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:30.487736 23447 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:30.440819 29384 consensus_queue.cc:260] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:30.487142 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:30.438091 29350 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.478129 29429 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:30.417166 29314 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.496623 23445 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:09:30.444538 28925 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.486014 29377 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:30.501058 22747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.003s	sys 0.023s
W20260709 13:09:30.505008 22747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.004s	sys 0.023s
I20260709 13:09:30.509552 22900 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 13:09:30.483043 29437 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 13:09:30.495643 29386 raft_consensus.cc:2764] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
W20260709 13:09:30.516157 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.523097 22830 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:09:30.536484 29167 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:30.512027 29384 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:30.569620 23447 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:09:30.539631 29401 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.508646 29377 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.576555 23446 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 24.
I20260709 13:09:30.537714 29391 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.563210 29355 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.559921 29435 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 13:09:30.560493 29386 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:09:30.585985 29439 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:30.589432 29439 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.580386 29139 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.577497 23451 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 23. Current term is 24. Ops: []
W20260709 13:09:30.598048 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 147507us
W20260709 13:09:30.608969 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 458773us
I20260709 13:09:30.598865 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:30.579330 28990 consensus_queue.cc:1059] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:09:30.596413 29362 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.584506 29355 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.639119 22904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2845) took 1018 ms. Trace:
I20260709 13:09:30.639207 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.642112 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:30.647181 23448 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 26: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:30.630623 29061 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:30.644533 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:30.644840 29441 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:30.612455 29139 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.580206 29401 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.656509 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:30.656961 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:30.639746 22904 rpcz_store.cc:276] 0709 13:09:29.620781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.631464 (+ 10683us) service_pool.cc:224] Handling call
0709 13:09:29.631776 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:29.631986 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:09:29.774707 (+142721us) log.cc:844] Serialized 14 byte log entry
0709 13:09:30.420500 (+645793us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:30.446378 (+ 25878us) spinlock_profiling.cc:243] Waited 59.8 ms on lock 0x7b1800426040. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb787f33c7 00007fcb74e059a7 00007fcb74e150bf 00007fcb74e15038
0709 13:09:30.446624 (+   246us) log.cc:844] Serialized 14 byte log entry
0709 13:09:30.638571 (+191947us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:30.638628 (+    57us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:09:30.638770 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:30.638862 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:30.639050 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":504581,"prepare.queue_time_us":895195,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":9024,"raft.queue_time_us":49353,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":21701,"spinlock_wait_cycles":149469312,"thread_start_us":409063,"threads_started":3,"wal-append.queue_time_us":349359}
I20260709 13:09:30.641044 29378 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.659238 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:30.653425 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:30.634699 29429 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 13:09:30.660379 23448 raft_consensus.cc:2393] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 13: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:30.645550 28925 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:30.664073 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:30.671357 29435 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.671867 29435 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.670415 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:30.656826 29314 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.674340 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:30.675307 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:30.676466 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:30.658638 29061 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:30.660250 29445 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.669721 28895 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:30.660784 29185 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:30.667667 29120 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:30.685279 29120 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:30.684242 29445 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.685803 29120 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:30.680483 29396 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:30.694588 29445 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.666220 29378 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.698755 23447 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:09:30.699134 29378 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:30.712378 29414 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.870s	user 0.000s	sys 0.005s
I20260709 13:09:30.714982 29269 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:30.717303 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:30.717697 29401 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:30.688050 29300 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.899s	user 0.011s	sys 0.001s
I20260709 13:09:30.721144 22797 leader_election.cc:400] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:30.736950 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2936) took 1829 ms. Trace:
I20260709 13:09:30.689908 29377 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:30.693037 29396 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:30.737571 22748 rpcz_store.cc:276] 0709 13:09:28.905610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:28.905729 (+   119us) service_pool.cc:224] Handling call
0709 13:09:28.905979 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:29.629949 (+723970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:30.734864 (+1104915us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54002684d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:30.735016 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663513472}
I20260709 13:09:30.739372 29300 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:30.751075 29061 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:30.753293 29445 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:30.761384 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 87111.7us
I20260709 13:09:30.753993 29295 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:30.763584 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:30.723317 29185 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:30.765676 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:30.771778 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:30.769110 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:30.772871 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:30.781174 22750 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:30.783365 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:30.788527 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:30.789270 22751 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:30.795609 22898 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:30.798760 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:30.738544 29435 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:30.766309 29396 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:30.808349 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:30.794055 29424 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:09:30.780359 28895 consensus_queue.cc:1059] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:09:30.779789 29445 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.821656 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:30.822438 22754 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:30.825795 22751 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:30.809610 29185 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:09:30.803108 28978 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:09:30.846225 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:30.848960 22830 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:30.816439 29458 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:30.818727 29323 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:30.852984 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.859370 29445 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:30.863205 22749 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:30.864364 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:30.806509 29120 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.847045 29355 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:30.849903 29139 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.874511 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:30.875097 22747 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:30.867424 29305 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:30.870919 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:30.851977 29458 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:30.876096 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:30.878585 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 103320us
I20260709 13:09:30.855566 29323 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:30.879336 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.879503 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:30.849687 28895 consensus_queue.cc:1243] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:30.883723 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:30.880323 22752 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:30.881763 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.873600 28696 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:30.882594 29185 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:30.892714 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:30.877413 29305 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:30.895090 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:30.880242 29355 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:30.879848 29458 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:30.883287 29349 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:30.885788 29384 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:30.891676 28696 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:30.893692 29185 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:30.905944 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2252) took 1171 ms. Trace:
I20260709 13:09:30.907725 22827 rpcz_store.cc:276] 0709 13:09:29.731714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.731848 (+   134us) service_pool.cc:224] Handling call
0709 13:09:30.902531 (+1170683us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54002c8b54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:30.902641 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369381760}
I20260709 13:09:30.895686 29467 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:30.898747 29355 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:30.895972 29450 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:30.906314 28696 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:30.904147 29384 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:30.916266 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:30.918730 22827 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:30.914183 29355 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:30.910977 29396 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:30.925189 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.917384 29300 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:30.929904 29355 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.934903 22871 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:30.936043 28696 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.939606 22753 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:09:30.943539 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:30.944465 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:30.945135 22751 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:30.947109 23445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:30.947834 23445 raft_consensus.cc:606] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:30.950053 22896 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:09:30.957849 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:30.959380 22897 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:30.961081 22900 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:09:30.941051 29384 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:30.965622 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:30.971156 22905 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 15. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:09:30.950932 29450 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:30.952598 29396 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:30.918092 29305 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:30.989871 29305 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:30.992985 23445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:30.951258 29300 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:09:30.968300 29061 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.967206 29185 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:30.986212 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:30.993626 23445 raft_consensus.cc:606] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:30.993386 29369 consensus_queue.cc:1175] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6a5191aaff76469bbf6c4735dc9e6509" responder_term: 18 status { last_received { term: 15 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: [17.10-17.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:31.006520 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:31.113034 23452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1656) took 1567 ms (client timeout 1667 ms). Trace:
I20260709 13:09:31.041682 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:31.119612 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:31.123380 22905 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:31.124145 22751 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:31.124805 23445 raft_consensus.cc:614] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:31.124935 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:31.125514 22751 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:31.044328 22897 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:09:31.121539 23452 rpcz_store.cc:269] 0709 13:09:29.545611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.545748 (+   137us) service_pool.cc:224] Handling call
0709 13:09:31.112872 (+1567124us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b5400583754. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:31.112980 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703321472}
I20260709 13:09:31.130662 22749 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:31.138278 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.140151 23450 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:31.142472 22749 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:31.137368 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:31.042510 29395 consensus_meta.cc:287] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.000s
I20260709 13:09:30.993672 29450 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:31.145561 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.146899 22899 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:31.113154 29083 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.043239 29348 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:31.004822 29350 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:09:31.152113 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 252692us
I20260709 13:09:31.150130 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.136838 29379 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.108830 29355 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:31.171909 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.058090 29424 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 13:09:31.159504 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.158764 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2141) took 1404 ms. Trace:
I20260709 13:09:31.158836 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:31.179383 22750 raft_consensus.cc:2393] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:31.157586 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.005487 29120 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.162619 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:31.139302 29349 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:31.168442 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.057845 29030 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 13:09:31.156942 22751 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:31.144320 29422 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:31.172677 23452 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:31.057849 29388 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 13:09:31.004876 29061 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.095254 29458 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.178962 22898 rpcz_store.cc:276] 0709 13:09:29.747313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.747439 (+   126us) service_pool.cc:224] Handling call
0709 13:09:29.772793 (+ 25354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:30.795984 (+1023191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.151238 (+355254us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003e9754. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:31.151401 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2351792384}
W20260709 13:09:31.182683 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.184861 22747 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:31.206468 22747 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:31.137141 28992 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:09:31.151861 29396 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.151859 29295 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:31.218905 29295 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:31.219502 29295 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:31.221385 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.221820 29450 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.155351 29185 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:31.177074 29464 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 13:09:31.154101 29348 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:31.229566 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:31.230484 29348 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.162358 29350 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:31.154073 29466 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:31.231878 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:31.165866 29467 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:31.232642 29350 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.234563 29350 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:31.193399 29349 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:31.199088 23450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38064 (request call id 1655) took 1656 ms (client timeout 1683 ms). Trace:
I20260709 13:09:31.213279 29396 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.214128 29164 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:31.214198 22794 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:31.266443 29427 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
W20260709 13:09:31.276896 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 62799.3us
I20260709 13:09:31.277690 22794 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:31.210371 29445 raft_consensus.cc:3055] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:31.278366 29445 raft_consensus.cc:740] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:31.232942 29467 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:31.239244 23450 rpcz_store.cc:269] 0709 13:09:29.542634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.542782 (+   148us) service_pool.cc:224] Handling call
0709 13:09:31.198948 (+1656166us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54006dea54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:31.199037 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3504128384}
I20260709 13:09:31.207520 22750 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:09:31.281004 22750 pending_rounds.cc:85] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 15
I20260709 13:09:31.281270 22750 pending_rounds.cc:107] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:09:31.281536 22750 raft_consensus.cc:2889] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:31.158836 29083 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.168181 29402 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:09:31.258347 29349 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.265177 22753 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:31.269655 29474 raft_consensus.cc:2764] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 13:09:31.178136 29379 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.292371 29185 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.293066 29379 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.275024 22752 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:09:31.240556 29377 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:09:31.284230 29458 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.203994 29422 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.265038 29164 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.298429 29422 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.299692 29164 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.276261 29139 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:31.301663 29139 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.303594 29139 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.285884 29395 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:31.292840 29185 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.314323 23444 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:31.279309 29445 consensus_queue.cc:260] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.289027 29401 raft_consensus.cc:2804] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:31.271078 29399 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:09:31.320541 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.335853 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.307437 29030 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:09:31.341650 22822 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:09:31.350119 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.351086 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.351914 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.353937 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:31.358342 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 173544us
I20260709 13:09:31.359609 22827 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:09:31.362416 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.362524 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.363195 22822 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:31.363631 22827 raft_consensus.cc:2410] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 18 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 14.
I20260709 13:09:31.378510 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.379436 22823 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:31.346273 29401 raft_consensus.cc:697] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:31.348803 29269 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:31.349787 29120 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:31.389542 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.390152 22829 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:31.390436 29439 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.358752 29445 raft_consensus.cc:3060] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:31.392802 29295 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.396498 29295 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.401739 29384 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.403779 29384 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.404829 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.405462 22830 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:31.405701 22831 raft_consensus.cc:2368] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19.
I20260709 13:09:31.406466 29061 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.383224 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:31.412281 22900 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:31.391894 29396 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:31.429309 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46156.3us
I20260709 13:09:31.432296 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:31.436437 22823 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:09:31.436905 22823 pending_rounds.cc:85] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 12
I20260709 13:09:31.437186 22823 pending_rounds.cc:107] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:09:31.438259 22823 raft_consensus.cc:2889] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:31.438623 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:31.438724 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:31.440737 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.443743 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.444525 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.448539 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:31.429181 29030 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:09:31.453346 29083 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:31.453781 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:31.461220 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:31.462064 23444 raft_consensus.cc:2368] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21.
I20260709 13:09:31.471915 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.439595 29396 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:09:31.483691 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 134318us
I20260709 13:09:31.486310 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.490209 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:31.493197 22750 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:31.498821 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.500733 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.489535 22748 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:09:31.426669 29120 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.427958 29269 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.501848 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.501281 22896 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:31.515285 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.512468 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.516958 22823 raft_consensus.cc:2410] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:09:31.501499 22825 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:31.506147 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.518656 23444 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:31.521304 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.521414 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.523200 23419 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:31.523645 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:31.524237 23451 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:31.524109 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:31.525094 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.443684 29321 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:09:31.492729 29384 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.532277 22827 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:31.514468 29396 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:31.544342 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.544978 22825 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:31.546645 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.548959 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:31.566357 22827 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:31.562542 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:31.533798 29427 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:31.582893 22825 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:31.600360 29402 raft_consensus.cc:993] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:31.570458 28696 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:31.560338 29185 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.575884 29427 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.535171 29384 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.573482 29482 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.560635 29469 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:31.666428 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 49959.1us
I20260709 13:09:31.572345 29480 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.667133 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:31.668013 29483 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 13:09:31.668696 29481 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:09:31.669677 29388 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:31.669812 29185 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:31.670230 29388 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.670897 28992 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:31.671281 28992 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:31.672060 28992 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:31.656256 29485 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 13:09:31.676491 29487 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.002s
I20260709 13:09:31.548018 29478 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:31.526202 29401 consensus_queue.cc:237] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.611898 29396 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:31.626085 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48014.4us
I20260709 13:09:31.636548 28563 raft_consensus.cc:1081] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:31.622289 29479 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:31.660704 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:31.681957 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.682252 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.670715 29185 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:31.682854 22750 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:31.666236 29499 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 13:09:31.688203 29478 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:31.690140 22797 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:31.686224 29469 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:31.692315 29484 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:09:31.693274 29491 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:09:31.693784 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:31.694032 29469 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.694960 29269 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:31.695119 29490 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:09:31.696697 29269 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:31.696775 29369 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:09:31.697821 29493 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:09:31.698134 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.698681 22826 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:31.699281 22755 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:31.701541 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:31.701704 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:31.702296 22896 raft_consensus.cc:2393] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:31.705107 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:31.705927 23445 raft_consensus.cc:3060] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:31.697110 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.709785 28563 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:31.683708 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.685508 22749 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:09:31.665995 29427 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:31.711149 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44099.8us
I20260709 13:09:31.711494 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:31.711635 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.673239 28992 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:31.719741 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:31.720580 29422 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:31.721840 29422 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:31.722806 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:31.724169 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:31.724795 29083 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:31.725544 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.716508 23444 raft_consensus.cc:1240] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:09:31.729892 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.730613 22825 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:31.730511 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.731559 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:31.731916 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.731590 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:31.733867 22828 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:31.690927 29185 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:31.738050 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:31.735211 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.741933 22905 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:31.744913 22750 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:31.713555 29277 raft_consensus.cc:2804] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:31.713802 29269 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:31.751225 29384 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.751703 29139 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.837809 29139 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.839787 29139 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.752156 29266 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.840430 29266 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.841140 23450 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:31.842422 29266 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.752195 29305 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.843219 29305 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.683434 23451 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:31.752789 29427 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.754168 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:31.742797 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:31.717352 29479 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:31.746974 29083 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:31.687964 29482 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.688153 29480 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:31.874289 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 120231us
I20260709 13:09:31.778760 29478 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:31.730222 29402 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:09:31.785872 29466 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:31.785140 29185 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.787423 29499 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:31.800554 29348 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:31.810513 29502 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:09:31.811743 29424 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:09:31.812865 29498 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:09:31.814296 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:31.817613 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.820144 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.894127 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:31.896039 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:31.896780 23447 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:31.744827 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:31.833948 29277 raft_consensus.cc:697] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:31.833683 28563 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.752971 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.903201 28563 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.857936 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.859603 23452 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:09:31.868716 29120 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.870421 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.911525 22827 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:31.817404 29388 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.879899 29482 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.879966 22747 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:31.846215 29321 raft_consensus.cc:2764] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:09:31.885298 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:31.871294 29083 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.872074 29269 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.936944 29083 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:31.937552 22822 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:31.938580 29269 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.939181 29269 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:31.939606 29269 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.940647 22826 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:31.944051 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.906695 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.876579 29305 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.881053 29266 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:31.910534 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:31.949388 29350 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:09:31.949824 28696 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:31.950281 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:31.950296 28696 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:31.951311 29277 consensus_queue.cc:237] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.952281 29061 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:31.952363 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:31.883051 29466 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:31.883050 29369 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.953748 29369 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:31.956110 29369 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:31.891121 29424 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:31.891238 29384 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.956787 29424 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:31.956920 29384 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.959414 29384 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.892195 29499 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:31.951390 28696 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.953054 29061 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:31.960901 29499 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.20, Last appended by leader: 20, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.953773 29083 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:31.933888 29480 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:31.969985 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:31.970222 29083 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.970743 22755 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:31.944180 29427 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:31.972678 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:31.974470 28563 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:31.977672 29384 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:31.978247 29384 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.978567 29488 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:31.978747 29384 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:31.979336 29488 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:31.901497 29185 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:31.984815 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:31.924248 22827 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
W20260709 13:09:31.989622 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 251977us
I20260709 13:09:31.990610 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:31.993691 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:31.995095 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:31.999210 23445 raft_consensus.cc:2468] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:31.932346 29448 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:32.009464 28563 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:32.011219 22827 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:32.013264 22827 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:31.999524 29466 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.015760 23416 leader_election.cc:400] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19.
I20260709 13:09:32.016218 23416 leader_election.cc:403] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:09:32.017611 29466 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.018916 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:32.019789 29391 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:32.021598 29391 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:09:32.025357 29083 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19.
I20260709 13:09:32.025676 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:31.989292 29458 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:32.051848 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:32.062553 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:32.062728 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:32.063735 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:32.065160 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.067289 23447 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:32.069649 28563 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.070461 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:32.082240 29448 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:32.082875 29448 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:32.084450 29494 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.329s	user 0.000s	sys 0.004s
I20260709 13:09:31.999413 29479 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.086156 29448 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:32.086942 29448 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:32.087705 29448 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:32.008036 29350 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:32.010074 29061 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:31.999045 29439 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:32.070140 28563 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.095242 28563 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.008590 29269 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:32.098740 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 379116us
I20260709 13:09:32.099164 23447 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:32.107895 22830 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:32.115694 22719 leader_election.cc:400] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21.
I20260709 13:09:32.116123 22719 leader_election.cc:403] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:09:32.071137 29391 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:32.117167 23447 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:09:32.119467 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:32.125145 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:32.126112 23447 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:09:32.061012 29427 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:09:32.101184 29355 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.146253 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:32.155534 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:32.135857 29473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.025s	sys 0.040s
I20260709 13:09:32.133014 29466 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21.
I20260709 13:09:32.172811 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.174775 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1585) took 1227 ms. Trace:
I20260709 13:09:32.175297 22902 rpcz_store.cc:276] 0709 13:09:30.942995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.943328 (+   333us) service_pool.cc:224] Handling call
0709 13:09:32.170272 (+1226944us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54001b6054. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:32.170431 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967571328}
I20260709 13:09:32.137796 29391 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:32.182211 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3519) took 1240 ms. Trace:
I20260709 13:09:32.168013 29120 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:32.165771 29482 raft_consensus.cc:2764] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:09:32.168146 29473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.025s	sys 0.040s
I20260709 13:09:32.182780 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:32.182570 22753 rpcz_store.cc:276] 0709 13:09:30.941856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.941992 (+   136us) service_pool.cc:224] Handling call
0709 13:09:30.952927 (+ 10935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.289159 (+336232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.181987 (+892828us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54000776d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:32.182159 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717576064}
I20260709 13:09:32.185272 29488 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.186770 22825 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:32.172645 29486 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:32.178426 29467 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:32.214740 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:32.220754 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:32.221949 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:32.222064 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:32.225237 22825 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:32.216133 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:32.234436 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:32.143873 29355 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.264855 29488 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:32.265735 29488 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.265991 29469 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:09:32.266180 29488 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.266505 29469 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:32.213758 29120 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:32.220358 29473 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:32.281374 22825 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:32.239912 29480 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:32.267197 29369 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:32.291685 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:32.314047 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 175772us
I20260709 13:09:32.315943 22827 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:32.279570 29478 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:32.316752 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 94889.1us
I20260709 13:09:32.317677 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:32.193118 29448 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.276590 29305 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:32.290369 29473 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:32.294943 29369 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.341614 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:32.298396 29401 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:32.335173 29427 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:32.342599 29369 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:32.339799 29473 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:32.343709 29480 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:32.371837 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:32.350756 29350 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.376866 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:32.339533 29305 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.296532 29391 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.377652 29507 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:32.377822 29478 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:32.377750 29277 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.378172 29120 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.388645 29505 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:32.412935 29505 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.413271 29505 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:32.381574 29350 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:32.414366 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:32.415094 22830 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:32.415704 29350 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:32.420156 22755 raft_consensus.cc:2410] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:09:32.391335 23450 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:32.425015 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 2004) took 1345 ms. Trace:
I20260709 13:09:32.393676 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:32.427942 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:32.429502 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.425521 22752 rpcz_store.cc:276] 0709 13:09:31.079999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.184359 (+104360us) service_pool.cc:224] Handling call
0709 13:09:31.211669 (+ 27310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.295696 (+ 84027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.422015 (+1126319us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b540032a854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:32.424948 (+  2933us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144719616}
I20260709 13:09:32.436189 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:32.442414 29350 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.443040 29350 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.341892 29469 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.450758 29305 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:32.458004 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:32.391860 29517 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:09:32.353731 29401 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:32.390787 29503 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:09:32.476753 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:32.391017 29513 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
W20260709 13:09:32.391425 29506 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:09:32.466274 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.494892 22902 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:32.492865 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.379935 29377 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.498919 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.419883 29479 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:32.473697 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.506524 22897 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:32.474560 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:32.409881 29488 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:32.475317 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:32.469964 29269 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.397710 29478 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.473826 28696 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.524544 28696 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.526345 28696 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:32.527570 28696 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:32.527971 28696 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:32.528666 28696 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.409008 29439 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:32.373216 29427 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.531342 29427 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.531800 29427 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.532238 29427 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.390398 29531 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:32.533581 29531 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.535557 29531 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:32.410400 29355 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:32.501881 29083 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.537024 29486 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:32.537158 29083 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.537765 29486 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:32.539215 29083 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.539216 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.543922 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.544947 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.482214 23452 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:09:32.547369 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:32.548985 22901 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:32.553752 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:32.484215 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 149684us
I20260709 13:09:32.556087 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:32.557509 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:32.558395 29061 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:32.558512 29401 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.559324 29030 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:32.560484 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:32.560938 29030 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:32.398104 29277 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.563019 29277 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:32.563458 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:32.563905 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.564158 22750 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:56780
I20260709 13:09:32.564823 22750 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:32.565160 22750 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:32.565824 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.507484 29377 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.566668 22748 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:32.412407 29424 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.568037 29377 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:32.568743 29424 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:32.569427 29479 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:32.511384 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:32.569669 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 2985) took 1075 ms. Trace:
W20260709 13:09:32.570308 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 59046.5us
I20260709 13:09:32.571461 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:32.570207 22751 rpcz_store.cc:276] 0709 13:09:31.494011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.494187 (+   176us) service_pool.cc:224] Handling call
0709 13:09:31.521644 (+ 27457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:31.532253 (+ 10609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:31.532298 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:31.546460 (+ 14162us) log.cc:844] Serialized 26 byte log entry
0709 13:09:31.988052 (+441592us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:31.988103 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.988196 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:32.569393 (+581197us) raft_consensus.cc:1685] finished
0709 13:09:32.569428 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:32.569602 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2462,"raft.queue_time_us":6553,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":3536,"thread_start_us":363989,"threads_started":1,"wal-append.queue_time_us":606617}
I20260709 13:09:32.571890 29458 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:32.570323 29479 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:32.573930 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.574896 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47110 (request call id 1352) took 1046 ms. Trace:
I20260709 13:09:32.575608 29520 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:32.575415 22822 rpcz_store.cc:276] 0709 13:09:31.528171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.699557 (+171386us) service_pool.cc:224] Handling call
0709 13:09:32.574661 (+875104us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54002f0dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:32.574808 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471561472}
I20260709 13:09:32.576253 29520 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.576664 29520 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:32.579619 29061 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.580106 29061 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.394428 29507 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:32.497054 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.582551 22902 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:09:32.499761 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.451794 29519 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:32.459287 29139 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:32.509416 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.399269 29266 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:32.516399 29527 consensus_queue.cc:1059] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:32.517377 29488 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.634707 29030 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:32.588903 22750 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:32.519954 29348 consensus_meta.cc:287] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.000s
I20260709 13:09:32.520735 29269 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:32.635277 29030 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:32.636495 22750 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:32.637106 29269 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.522924 29478 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.637475 22824 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:32.523044 28563 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:32.636404 29030 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:32.638952 29528 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.077s	user 0.012s	sys 0.008s
I20260709 13:09:32.523327 29525 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:32.639619 29525 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.640340 29061 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:32.584836 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:32.563993 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:32.577060 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:32.628612 22904 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:32.643232 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.569137 22748 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:32.617354 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:32.620998 22902 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:09:32.615852 22896 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 13.
W20260709 13:09:32.519109 29384 consensus_meta.cc:287] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.000s
I20260709 13:09:32.496832 29350 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:32.642405 22822 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:32.635147 22826 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:32.638636 28563 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.661991 23448 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:32.662101 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.662470 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:32.664815 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.665935 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.666616 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.666751 22900 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:32.669447 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.665372 22899 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
W20260709 13:09:32.615828 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 55422.6us
W20260709 13:09:32.658464 29529 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.004s
W20260709 13:09:32.660451 29533 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.091s	user 0.000s	sys 0.003s
I20260709 13:09:32.672926 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:32.673804 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.676347 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:32.676090 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:32.681361 22751 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:32.676437 22756 raft_consensus.cc:1275] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:09:32.665387 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.678107 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:32.659010 29083 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:32.690887 23445 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:32.644249 29305 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:32.690237 28563 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.628077 29519 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:32.689658 22900 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
W20260709 13:09:32.676831 29521 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.084s	user 0.004s	sys 0.000s
I20260709 13:09:32.694625 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:32.697367 29478 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:32.699813 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:32.795833 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 119569us
I20260709 13:09:32.702287 29525 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.702435 29269 raft_consensus.cc:3055] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:32.663938 29384 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:32.704617 29388 raft_consensus.cc:993] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:32.706754 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38478 (request call id 3527) took 1024 ms. Trace:
I20260709 13:09:32.707652 29520 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.711196 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.712248 29505 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.716017 22750 raft_consensus.cc:1275] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:32.815505 22747 rpcz_store.cc:276] 0709 13:09:31.682111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.683486 (+  1375us) service_pool.cc:224] Handling call
0709 13:09:32.706590 (+1023104us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400046854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:32.706701 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371931392}
W20260709 13:09:32.790261 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 97489.6us
W20260709 13:09:32.795058 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 104857us
W20260709 13:09:32.795192 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 85940us
I20260709 13:09:32.635167 29488 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.798202 22829 raft_consensus.cc:2410] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:09:32.803056 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1747) took 1093 ms. Trace:
W20260709 13:09:32.804385 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 368272us
I20260709 13:09:32.817869 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:32.806545 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:32.806090 22822 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:09:32.703258 29427 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:32.716790 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.717175 29305 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.796415 29519 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:32.823998 23446 rpcz_store.cc:276] 0709 13:09:31.712171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.712351 (+   180us) service_pool.cc:224] Handling call
0709 13:09:31.712681 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.712918 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:09:32.136583 (+423665us) log.cc:844] Serialized 14 byte log entry
0709 13:09:32.687978 (+551395us) log.cc:844] Serialized 14 byte log entry
0709 13:09:32.801292 (+113314us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:32.802709 (+  1417us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:32.802770 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.802858 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:32.802994 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113101,"prepare.queue_time_us":461879,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":100400,"raft.queue_time_us":158781,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"thread_start_us":944564,"threads_started":3,"wal-append.queue_time_us":432144}
I20260709 13:09:32.806202 29401 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.833811 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:32.805284 29295 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:32.582577 29507 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.805999 29269 raft_consensus.cc:740] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:32.844776 22831 raft_consensus.cc:2368] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:32.845940 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:32.802394 29537 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:32.841756 29402 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.858356 29402 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.858753 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:32.821669 29480 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.860020 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:32.860327 29402 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:32.842154 29305 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:32.848548 29531 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:09:32.857709 29164 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:32.848273 29401 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.859817 29480 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:32.860146 29489 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:32.842126 29536 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:32.850553 22823 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:09:32.851890 22822 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:09:32.830845 29424 raft_consensus.cc:1081] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:32.869982 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.830966 29083 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:32.889546 29083 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.891016 29350 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:32.861279 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:32.894752 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:32.807416 29478 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.858682 29384 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:32.848950 29348 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:32.888571 29538 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:09:32.873476 29537 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:32.834345 28992 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:09:32.861249 29388 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:32.864010 29395 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:32.910166 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 64287.9us
I20260709 13:09:32.872516 29402 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:09:32.914398 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.920346 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 3148) took 1149 ms. Trace:
I20260709 13:09:32.861047 29061 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:09:32.912434 29277 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:32.937103 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3150) took 1165 ms. Trace:
I20260709 13:09:32.946748 22827 rpcz_store.cc:276] 0709 13:09:31.771744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.334607 (+562863us) service_pool.cc:224] Handling call
0709 13:09:32.583044 (+248437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:32.583244 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.583280 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:32.583332 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:32.583374 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.583441 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:32.936892 (+353451us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54002f0dd0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:32.937031 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570523904}
I20260709 13:09:32.948787 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:32.950177 22827 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:32.921784 29369 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.000s
I20260709 13:09:32.934204 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:32.939600 22824 rpcz_store.cc:276] 0709 13:09:31.771180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.814005 (+ 42825us) service_pool.cc:224] Handling call
0709 13:09:32.920191 (+1106186us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b5400089b54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:32.920295 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1711921408}
I20260709 13:09:32.956758 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:32.941325 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.959784 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.961292 22747 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:32.941529 22748 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:56780
I20260709 13:09:32.963367 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.964874 22825 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:32.929476 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:32.928398 29473 consensus_meta.cc:287] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.546s	user 0.002s	sys 0.007s
I20260709 13:09:32.966801 22752 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:32.967839 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.980221 22747 raft_consensus.cc:2410] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:09:32.981278 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 21 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.921093 28978 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:32.964252 22756 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:32.921660 29030 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:32.924858 29388 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.941782 29537 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:32.345829 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:32.986131 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:32.946364 29277 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:32.966517 29501 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.975101 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:32.920816 29164 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
W20260709 13:09:33.004249 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 74876us
I20260709 13:09:32.989307 28978 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:32.987051 29439 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:09:32.986160 29473 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.993021 29388 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:33.019136 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.944252 29486 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.037575 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:33.000703 29277 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:32.976234 29369 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:32.900116 29536 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.068528 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:33.060632 22719 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.609s (SENT) [suppressed 10 similar messages]
I20260709 13:09:32.992146 29030 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:33.072613 22753 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:33.010552 29422 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:33.070931 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:33.082993 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 221719us
I20260709 13:09:33.084721 22748 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:33.047284 28978 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:33.087589 22830 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:33.087610 22748 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:33.089207 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 231355us
I20260709 13:09:33.000944 29537 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.077425 29530 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:33.078016 29424 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:33.075640 29030 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:33.120620 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:33.120136 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 264492us
I20260709 13:09:33.090000 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:33.129833 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:33.081696 29520 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.085796 29539 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:33.149827 28696 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:33.150345 28696 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:33.150700 28696 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:33.149724 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60252.6us
I20260709 13:09:33.151574 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:33.154454 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:33.007656 29501 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:33.156740 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.133603 29486 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.157812 23448 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 22: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:33.177675 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.120760 29355 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:09:33.177659 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.185742 23453 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:33.187173 29537 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.186945 29251 consensus_queue.cc:1048] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:33.188264 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56780 (request call id 2028) took 1123 ms. Trace:
I20260709 13:09:33.188870 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2262) took 1687 ms. Trace:
I20260709 13:09:33.206102 22828 rpcz_store.cc:276] 0709 13:09:31.501407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.744469 (+243062us) service_pool.cc:224] Handling call
0709 13:09:33.188679 (+1444210us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400433354. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:33.188801 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063695616}
W20260709 13:09:33.166162 29544 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.246s	user 0.001s	sys 0.001s
I20260709 13:09:33.165678 23447 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:09:33.158946 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.150727 29488 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.156330 29401 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.156795 29478 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.138165 29530 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.120891 29061 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:09:33.167129 29480 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.214445 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:33.189376 29164 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:33.155335 29489 consensus_queue.cc:1048] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:09:33.196234 29486 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:33.205729 22755 rpcz_store.cc:276] 0709 13:09:32.065044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.142858 (+ 77814us) service_pool.cc:224] Handling call
0709 13:09:33.188076 (+1045218us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54003381d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:33.188210 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564811520}
I20260709 13:09:33.279814 23445 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:09:33.215389 29552 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:09:33.280339 29448 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.012s	sys 0.002s
W20260709 13:09:33.291275 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 136861us
I20260709 13:09:33.306243 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:33.291781 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 139881us
I20260709 13:09:33.296056 28696 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:33.300761 23445 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:33.266988 29530 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:33.308238 23445 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.314639 23445 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:33.325253 22896 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:09:33.327250 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.297856 29401 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:09:33.307183 29448 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.013s	sys 0.003s
I20260709 13:09:33.303704 29486 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:33.328789 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.359897 22896 raft_consensus.cc:2410] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:09:33.362761 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1361) took 1147 ms. Trace:
I20260709 13:09:33.363296 22822 rpcz_store.cc:276] 0709 13:09:32.215801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.898547 (+682746us) service_pool.cc:224] Handling call
0709 13:09:32.918716 (+ 20169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:32.918927 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:09:33.134849 (+215922us) log.cc:844] Serialized 14 byte log entry
0709 13:09:33.361164 (+226315us) log.cc:844] Serialized 14 byte log entry
0709 13:09:33.361873 (+   709us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:33.362061 (+   188us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:09:33.362194 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.362399 (+   205us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.362674 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1236,"thread_start_us":433176,"threads_started":2}
I20260709 13:09:33.329630 28696 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:33.331688 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:33.366603 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.366990 28696 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.334247 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:33.334820 29549 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:09:33.301769 29489 consensus_queue.cc:1243] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 18.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:09:33.352586 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:33.365139 29562 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.365814 29536 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.366145 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.388628 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:33.389029 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:33.389297 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:33.389807 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 234831us
I20260709 13:09:33.390270 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:33.390767 22896 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:33.367570 28696 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.370162 22898 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:33.380539 22900 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:33.394150 28696 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.394692 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:33.396806 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:33.396538 22751 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:09:33.398703 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.396508 28978 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.370193 29534 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.401571 28978 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.386755 23445 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:33.402076 29534 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:33.403180 29305 consensus_meta.cc:287] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.005s
I20260709 13:09:33.403537 22896 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:33.403554 29305 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:33.404260 29534 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.362654 29384 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:33.405228 29539 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:33.405146 29384 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.405637 29539 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.405995 29539 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:33.407183 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.407972 22896 raft_consensus.cc:2410] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 20 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:09:33.360378 29486 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:33.359488 29355 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:33.410396 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.432404 22901 raft_consensus.cc:2376] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17. Re-sending same reply.
I20260709 13:09:33.412592 22756 raft_consensus.cc:2393] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:33.430204 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:33.430294 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.433830 22903 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:33.362106 29473 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.311373 29556 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 13:09:33.442934 22896 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 25. Current term is 26. Ops: []
I20260709 13:09:33.444152 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.444794 22896 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:09:33.437790 23416 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:33.446267 22896 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:09:33.446673 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:33.455503 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:33.461958 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:33.462224 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.462836 22822 raft_consensus.cc:2368] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17.
I20260709 13:09:33.463152 22901 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 25 because replica has last-logged OpId of term: 23 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 20.
I20260709 13:09:33.465659 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:33.380899 29369 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.429755 29561 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:33.476119 29561 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.408090 29305 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.479048 29561 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.479710 29561 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:33.480165 29561 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.480563 29561 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:33.467175 22751 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:09:33.466553 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.483626 22904 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:33.431768 29355 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.489562 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:33.359850 29401 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.497176 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:33.496300 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:33.500587 22747 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:09:33.500638 22756 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:09:33.403743 29542 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:09:33.508116 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:33.509174 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.509618 22751 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:33.464686 29422 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:33.467509 29083 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:33.515333 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:33.518055 22750 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:33.511575 22748 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.498819 29355 raft_consensus.cc:3060] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:33.521944 29083 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:09:33.532366 29555 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.261s	user 0.003s	sys 0.002s
I20260709 13:09:33.533579 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1357) took 1327 ms. Trace:
I20260709 13:09:33.533682 29541 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:33.534108 22829 rpcz_store.cc:276] 0709 13:09:32.205866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.826061 (+620195us) service_pool.cc:224] Handling call
0709 13:09:32.845152 (+ 19091us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:32.855771 (+ 10619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.855803 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:32.856233 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 13:09:33.002302 (+146069us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:33.002339 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.002406 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.533366 (+530960us) raft_consensus.cc:1685] finished
0709 13:09:33.533388 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.533520 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10240,"raft.queue_time_us":12805,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":145718,"threads_started":1,"wal-append.queue_time_us":282420}
I20260709 13:09:33.536077 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:33.536655 22829 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:33.538261 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1607) took 1001 ms. Trace:
I20260709 13:09:33.543272 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:33.537806 22831 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1360) took 1324 ms. Trace:
I20260709 13:09:33.538719 22897 rpcz_store.cc:276] 0709 13:09:32.536466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.583013 (+ 46547us) service_pool.cc:224] Handling call
0709 13:09:32.583307 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:32.583635 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.583671 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:32.645005 (+ 61334us) log.cc:844] Serialized 26 byte log entry
0709 13:09:33.077984 (+432979us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:33.266090 (+188106us) log.cc:844] Serialized 14 byte log entry
0709 13:09:33.436086 (+169996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.436177 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.538043 (+101866us) raft_consensus.cc:1685] finished
0709 13:09:33.538062 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.538191 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36204,"prepare.queue_time_us":459436,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":177636,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":34293,"thread_start_us":666569,"threads_started":3,"wal-append.queue_time_us":836648}
I20260709 13:09:33.494171 29061 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:09:33.410064 29562 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.553794 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:33.554584 22903 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 13.
I20260709 13:09:33.545279 22831 rpcz_store.cc:276] 0709 13:09:32.215469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.856431 (+640962us) service_pool.cc:224] Handling call
0709 13:09:32.856750 (+   319us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:32.857121 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:33.088324 (+231203us) log.cc:844] Serialized 14 byte log entry
0709 13:09:33.300346 (+212022us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:33.311778 (+ 11432us) log.cc:844] Serialized 52 byte log entry
0709 13:09:33.311962 (+   184us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:33.312006 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.312105 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.537577 (+225472us) raft_consensus.cc:1685] finished
0709 13:09:33.537602 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.537748 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22778,"prepare.queue_time_us":508365,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":21214,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":127,"thread_start_us":358606,"threads_started":2,"wal-append.queue_time_us":251023}
I20260709 13:09:33.557468 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:33.542126 29559 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:33.530447 29501 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.560585 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:33.559362 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 69898.7us
I20260709 13:09:33.567411 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.567612 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:33.568269 22897 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 22: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:33.568343 22756 raft_consensus.cc:2468] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
W20260709 13:09:33.595178 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:33.666891 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:33.735100 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 167559us
I20260709 13:09:33.735100 22747 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:33.739192 29580 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.223s
I20260709 13:09:33.741495 29542 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:33.742516 29562 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.744464 29401 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.738605 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3154) took 1953 ms. Trace:
I20260709 13:09:33.754328 22823 rpcz_store.cc:276] 0709 13:09:31.785429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.691513 (+906084us) service_pool.cc:224] Handling call
0709 13:09:32.691799 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:32.878152 (+186353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.738313 (+860161us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400432e54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:33.738532 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364364544}
I20260709 13:09:33.756093 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:33.757061 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:33.721259 29501 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.769207 22823 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:33.776113 22823 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:33.755466 29578 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:33.786190 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:33.566342 29559 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.788669 22831 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:09:33.783272 29565 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.321s	user 0.001s	sys 0.001s
I20260709 13:09:33.781405 29578 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:33.771765 29539 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:33.789777 29519 consensus_meta.cc:287] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.929s	user 0.008s	sys 0.003s
I20260709 13:09:33.779270 29501 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:33.822346 29469 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:33.822860 29530 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.825032 29530 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:33.826406 29559 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.836633 23447 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:33.826465 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 2031) took 1296 ms. Trace:
I20260709 13:09:33.837533 22754 rpcz_store.cc:276] 0709 13:09:32.529563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.688595 (+159032us) service_pool.cc:224] Handling call
0709 13:09:32.706726 (+ 18131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:32.707134 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.707173 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:32.707696 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:09:32.903324 (+195628us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:32.903362 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.904315 (+   953us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.437374 (+533059us) raft_consensus.cc:1685] finished
0709 13:09:33.437395 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.826151 (+388756us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b5400046850. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:33.826398 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2472,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":40923008,"thread_start_us":195322,"threads_started":1,"wal-append.queue_time_us":382308}
I20260709 13:09:33.839442 22829 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:33.841897 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1363) took 1549 ms. Trace:
I20260709 13:09:33.842367 22824 rpcz_store.cc:276] 0709 13:09:32.292759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.969635 (+676876us) service_pool.cc:224] Handling call
0709 13:09:32.970182 (+   547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:32.970541 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.970581 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:32.971056 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:09:33.158431 (+187375us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:33.158468 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.158536 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.841658 (+683122us) raft_consensus.cc:1685] finished
0709 13:09:33.841682 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.841820 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":9674,"thread_start_us":158079,"threads_started":1,"wal-append.queue_time_us":373221}
I20260709 13:09:33.805514 29519 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.829010 29536 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:33.849408 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.853204 22896 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:33.853780 22896 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:33.857048 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 71021.8us
W20260709 13:09:33.858084 29277 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.781s	user 0.005s	sys 0.005s
I20260709 13:09:33.858866 22896 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.866119 22896 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:33.872893 29388 consensus_meta.cc:287] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.823s	user 0.006s	sys 0.002s
I20260709 13:09:33.867996 29277 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:33.881510 29536 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:33.873580 29591 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 13:09:33.883936 29519 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.873297 29388 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:33.886068 29030 consensus_meta.cc:287] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.727s	user 0.005s	sys 0.005s
I20260709 13:09:33.886732 23444 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 23: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:33.886595 29030 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.850504 29578 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:33.889695 29578 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:33.877715 29561 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.890996 29388 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.898466 22753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 2024) took 1891 ms. Trace:
I20260709 13:09:33.889210 29277 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.898916 22753 rpcz_store.cc:276] 0709 13:09:32.006644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.407365 (+400721us) service_pool.cc:224] Handling call
0709 13:09:32.407614 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.073094 (+665480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.896759 (+823665us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b54003368d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:33.898403 (+  1644us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528784256}
I20260709 13:09:33.911502 22896 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:09:33.911878 22896 pending_rounds.cc:85] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 11
I20260709 13:09:33.912124 22896 pending_rounds.cc:107] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:09:33.880869 29355 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:33.912357 22896 raft_consensus.cc:2889] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:33.913210 29424 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.763s	user 0.009s	sys 0.002s
I20260709 13:09:33.914348 22748 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.869685 29486 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.871471 29505 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.861578 29576 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:09:33.925082 29424 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:33.925693 29501 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.933748 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:33.934301 22749 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:33.937220 22747 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:33.927541 29486 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.928222 23452 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:09:33.928900 22900 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:33.929421 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:33.937458 22902 raft_consensus.cc:2368] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:33.922596 29061 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.935060 29539 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.939728 29527 raft_consensus.cc:3055] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:33.939636 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:33.940505 22754 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:33.927248 29550 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.935s	user 0.006s	sys 0.009s
I20260709 13:09:33.943125 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:33.941227 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:33.941799 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:33.945374 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:33.945539 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 21 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.946102 23451 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:09:33.947223 22896 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:33.947796 29388 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:33.948436 29388 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.950811 29388 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:33.953183 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:33.954759 22719 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.016s	sys 0.008s
W20260709 13:09:33.955143 22719 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.016s	sys 0.008s
W20260709 13:09:33.955433 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 894747us
I20260709 13:09:33.962029 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:33.962986 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.963425 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:33.964167 23445 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:33.944022 29061 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.971006 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.971623 29061 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:33.972314 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:33.970685 29030 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.972887 22747 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:33.964784 29269 consensus_queue.cc:260] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.973723 29269 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:33.977152 28696 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:33.977625 28696 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:33.940251 29527 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:33.978538 28696 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:33.979460 29527 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:33.980508 29527 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:33.981670 22753 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:09:33.979878 28696 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:33.983773 29355 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:33.987571 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.993038 23451 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:33.995276 29030 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:33.995893 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.995795 29030 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:33.996143 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:33.996507 22899 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:33.996841 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3153) took 2216 ms. Trace:
I20260709 13:09:33.997216 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:33.997189 22830 rpcz_store.cc:276] 0709 13:09:31.780913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.415991 (+635078us) service_pool.cc:224] Handling call
0709 13:09:32.416324 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.088002 (+671678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.996649 (+908647us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b54001f90d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:33.996791 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1543310208}
I20260709 13:09:33.997826 29030 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:33.998297 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:33.999329 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.002275 23445 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:34.006378 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:34.006919 23451 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:34.007179 23451 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:33.996099 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1365) took 1378 ms. Trace:
I20260709 13:09:34.000224 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.008610 22827 rpcz_store.cc:276] 0709 13:09:32.618144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.993276 (+375132us) service_pool.cc:224] Handling call
0709 13:09:32.993823 (+   547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.158527 (+164704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:33.158555 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:33.158602 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:33.158655 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.995901 (+837246us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54001ee6d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:33.995945 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.996050 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378384128}
W20260709 13:09:34.010732 29573 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.455s	user 0.002s	sys 0.000s
I20260709 13:09:34.013823 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.014505 22899 raft_consensus.cc:2410] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:09:34.014914 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.015902 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.016352 22900 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:34.018115 22901 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:34.018206 29561 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:34.018164 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 2268) took 2492 ms. Trace:
I20260709 13:09:34.018865 29561 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:09:34.019330 22826 rpcz_store.cc:276] 0709 13:09:31.526116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.991941 (+465825us) service_pool.cc:224] Handling call
0709 13:09:31.992239 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:32.660875 (+668636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.017939 (+1357064us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b540003fcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:34.018115 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470114176}
I20260709 13:09:34.019806 29083 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:34.020243 29083 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:34.021029 29083 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.022213 29083 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:34.007979 23451 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.026050 23451 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:34.026145 29545 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 1.034s	user 0.002s	sys 0.010s
I20260709 13:09:34.016338 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.030077 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.030555 22899 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:34.034857 22747 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:09:34.035591 22900 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:34.036540 22900 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:34.036885 22900 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:34.037081 22754 raft_consensus.cc:2410] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:09:34.037497 29466 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.038007 29466 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.038472 23445 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 22 index: 20. Preceding OpId from leader: term: 24 index: 21. (index mismatch)
I20260709 13:09:34.040143 29466 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.042569 29384 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.044067 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:34.045585 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:34.064921 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.065433 22752 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:34.046622 29581 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:09:34.048066 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.051563 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.066771 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.067087 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.053188 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.057703 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:34.068341 23447 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:34.069736 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:34.070815 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:34.071466 22752 raft_consensus.cc:2393] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:34.071672 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.077486 23448 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:34.058627 22748 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:34.059659 22749 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:09:34.063301 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.063802 23453 raft_consensus.cc:1240] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:09:34.078241 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 192766us
I20260709 13:09:34.081938 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.052702 22900 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.084025 22797 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:34.090440 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.091003 22898 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:34.091068 22797 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:34.091935 22752 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:34.073029 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.072180 29139 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:34.094484 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.097656 23445 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:34.095230 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.077411 22899 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 27.
I20260709 13:09:34.099227 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.099803 22899 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:34.099890 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:34.100559 22897 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:34.101425 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:34.005944 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:34.110235 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 104344us
I20260709 13:09:34.123281 22900 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:34.130153 29546 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:34.130576 29546 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:34.097182 29266 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.131261 29266 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.131405 29546 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.132512 29546 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:34.158183 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.158375 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.158973 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:34.159169 22897 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:34.159539 23452 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:34.160359 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 49069.7us
W20260709 13:09:34.160861 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 59557.5us
I20260709 13:09:34.133510 23449 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:09:34.157112 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.133428 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1366) took 1226 ms. Trace:
I20260709 13:09:34.163745 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:34.164789 22903 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:34.109122 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:34.138301 22754 raft_consensus.cc:1217] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 12.7->[15.8-19.10]   Dedup: 15.8->[17.9-19.10]
W20260709 13:09:34.165889 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56880.3us
I20260709 13:09:34.166643 22754 pending_rounds.cc:85] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:09:34.166083 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.166903 22719 leader_election.cc:400] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
W20260709 13:09:34.225530 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 75655.4us
I20260709 13:09:34.267681 22719 leader_election.cc:403] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:09:34.184928 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.164069 22828 rpcz_store.cc:276] 0709 13:09:32.907062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.207069 (+300007us) service_pool.cc:224] Handling call
0709 13:09:33.207449 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:33.303370 (+ 95921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:33.303404 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:33.311021 (+  7617us) log.cc:844] Serialized 26 byte log entry
0709 13:09:33.464597 (+153576us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:09:33.494910 (+ 30313us) log.cc:844] Serialized 14 byte log entry
0709 13:09:33.495125 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.998713 (+503588us) spinlock_profiling.cc:243] Waited 89.6 ms on lock 0x7b54002c9554. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:33.998766 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:34.046024 (+ 47258us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:34.133187 (+ 87163us) raft_consensus.cc:1685] finished
0709 13:09:34.133207 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:34.133374 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8284,"prepare.queue_time_us":34743,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":7232,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":17411,"spinlock_wait_cycles":224122112,"thread_start_us":95320,"threads_started":1,"wal-append.queue_time_us":441571}
I20260709 13:09:34.180104 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:34.166942 22754 pending_rounds.cc:107] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 13:09:34.271540 22754 raft_consensus.cc:2889] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:34.174062 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.273103 22753 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:34.276857 22899 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:34.279304 23451 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:34.279346 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:34.280320 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 113538us
W20260709 13:09:34.281286 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 107346us
I20260709 13:09:34.282954 22719 leader_election.cc:400] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17.
I20260709 13:09:34.283330 22719 leader_election.cc:403] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:09:34.287101 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.289296 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:34.089965 29474 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:34.276754 22900 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:09:34.133217 29139 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.098479 29348 raft_consensus.cc:2804] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:34.296559 28696 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:34.301985 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.163219 29030 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:34.133487 29572 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:34.307502 29561 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:09:34.307204 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 127180us
I20260709 13:09:34.308931 22898 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:34.309861 29384 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:34.311414 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.318747 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:34.133498 29355 consensus_queue.cc:1175] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6a5191aaff76469bbf6c4735dc9e6509" responder_term: 15 status { last_received { term: 15 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:34.157341 29061 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.149s
I20260709 13:09:34.312726 22822 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:34.313741 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:34.314368 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.315243 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:34.315429 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:34.315491 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.304448 29139 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:34.157460 29277 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:34.318347 29030 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.216960 29399 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.221487 29266 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.316846 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.325111 29348 raft_consensus.cc:697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:34.322603 28696 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:09:34.354887 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.354977 28696 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:34.323175 22903 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:34.243731 29576 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:34.324064 22899 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46616
I20260709 13:09:34.293478 29578 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:34.331264 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:34.340996 29558 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:34.293201 29480 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:34.295424 29269 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:34.355746 22829 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
W20260709 13:09:34.317721 29531 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 13:09:34.311596 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.220096 29469 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:34.383502 29469 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:34.383956 29469 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:34.364560 29576 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.388249 29576 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:34.338625 22824 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:34.389096 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.390268 29488 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.390360 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:34.390864 29488 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.392562 22824 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:09:34.392830 29488 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:34.337118 22825 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:34.303071 29448 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:34.350749 29427 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:34.343093 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.318858 29396 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17.
W20260709 13:09:34.343901 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.321262 23452 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:34.322144 29560 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:34.350258 22823 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:34.216699 29527 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.349519 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:34.355962 28696 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.357522 22899 raft_consensus.cc:606] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Received request to transfer leadership
I20260709 13:09:34.362589 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.409484 22899 raft_consensus.cc:614] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:34.333940 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.369143 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.305635 29474 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.348222 29030 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:34.376543 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:34.401623 29602 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:09:34.337750 29350 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.028s
W20260709 13:09:34.394665 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 111895us
I20260709 13:09:34.349579 29277 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:34.350952 29561 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:34.341809 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.429997 22826 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:34.402427 23445 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:09:34.404306 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.364866 29266 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:34.339787 29384 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.409713 23448 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:34.410488 22825 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:34.411114 28696 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:34.381493 29486 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:34.370774 29536 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:34.444414 29560 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.462733 29560 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.413791 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.415758 22829 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:34.380096 29269 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:34.464555 29269 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:34.418857 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.354074 29399 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.423566 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2153) took 1058 ms. Trace:
I20260709 13:09:34.401037 29439 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:09:34.453864 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 135194us
I20260709 13:09:34.439023 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:34.471702 22902 rpcz_store.cc:276] 0709 13:09:33.365513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.366390 (+   877us) service_pool.cc:224] Handling call
0709 13:09:34.420331 (+1053941us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b540034b2d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:34.423509 (+  3178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310849408}
I20260709 13:09:34.471830 22830 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:09:34.474303 22794 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:34.483923 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:34.490794 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:34.490998 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 436684us
W20260709 13:09:34.484526 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45579.2us
I20260709 13:09:34.469905 22905 raft_consensus.cc:2368] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17.
I20260709 13:09:34.496379 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:34.496909 22824 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:34.415406 29061 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:09:34.520311 22905 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2159) took 1146 ms (client timeout 1514 ms). Trace:
W20260709 13:09:34.520821 22905 rpcz_store.cc:269] 0709 13:09:33.373805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.392203 (+ 18398us) service_pool.cc:224] Handling call
0709 13:09:34.418757 (+1026554us) spinlock_profiling.cc:243] Waited 6.88 ms on lock 0x7b5800211598. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dc10a0 00007fcb79dc0984 00007fcb7333b6d4 00007fcb7333b662
0709 13:09:34.520245 (+101488us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15817600}
I20260709 13:09:34.498090 22828 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:09:34.522784 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:34.523411 22828 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:34.499047 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.499603 22753 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:34.500792 29479 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:34.501664 29507 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:34.549489 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.502622 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.503731 29505 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:34.451682 29501 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:09:34.411597 29427 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.505420 29519 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:34.505968 29466 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:34.604117 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.606317 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.508154 29469 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:34.508566 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 94839.2us
I20260709 13:09:34.607977 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:34.408782 29448 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.509056 29536 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:34.465211 29560 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:34.467957 29598 log.cc:927] Time spent T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:09:34.509440 29561 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.510295 29139 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:34.415617 29527 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:34.515285 29599 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:09:34.465525 29486 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.517267 29384 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.465957 29580 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:09:34.527483 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:34.533161 29547 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 1.538s	user 0.006s	sys 0.005s
I20260709 13:09:34.535247 29399 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:34.535481 29321 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.476039 29269 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.532547 23445 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:34.538672 29277 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.539403 22652 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 15 to 17, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 17 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:34.596902 29424 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:09:34.540639 29474 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.450523 29348 consensus_queue.cc:237] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:34.542634 29607 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:09:34.429682 29572 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.113s
I20260709 13:09:34.458180 29266 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.530215 29488 raft_consensus.cc:2804] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:09:34.469350 29595 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:09:34.546813 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:34.548220 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.581758 22824 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:34.582362 22828 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:34.588371 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:34.617041 29525 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
W20260709 13:09:34.600631 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 98077.2us
I20260709 13:09:34.603282 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:34.634027 29384 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.623029 29605 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:34.624007 29427 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:34.625221 22827 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:34.625990 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.628176 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.663120 22902 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:34.628417 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.664170 22899 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:34.628777 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:34.624039 29139 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:34.611315 29505 consensus_queue.cc:1243] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:09:34.630896 22822 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:09:34.617131 29531 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:34.637380 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:34.639199 29603 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.259s	user 0.004s	sys 0.000s
I20260709 13:09:34.614712 29519 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.641070 29527 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:34.642812 29604 log.cc:927] Time spent T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 13:09:34.540141 22831 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:34.612646 29083 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:34.691893 29083 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.692726 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.693616 29266 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.694381 29083 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:34.666241 29530 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:09:34.694408 29613 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:09:34.694989 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.695154 29530 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:34.597730 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.626696 29466 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:34.659469 29577 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:09:34.629962 29580 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:34.656093 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 67829.7us
I20260709 13:09:34.698153 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.700414 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:34.657781 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:34.619426 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.625816 29350 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.027s
W20260709 13:09:34.659739 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56523us
I20260709 13:09:34.662480 23447 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
W20260709 13:09:34.655448 29546 consensus_meta.cc:287] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.523s	user 0.013s	sys 0.006s
I20260709 13:09:34.668081 23448 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:34.647214 29321 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.667536 29558 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:34.663712 29384 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.671078 29607 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:34.678340 22753 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:34.691201 29448 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.630214 29486 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:34.702448 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.714705 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:34.674974 29531 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:34.705847 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 86531.7us
I20260709 13:09:34.704308 29527 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:34.703118 29139 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:34.696898 29466 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.707556 22901 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:34.701713 29269 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.716120 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:34.705175 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 47473.9us
I20260709 13:09:34.716424 28563 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:09:34.717828 23452 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 12.
I20260709 13:09:34.717816 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:34.724404 28992 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:09:34.740415 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.740490 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.737696 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:34.725345 29384 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:34.725863 29531 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:34.726147 29611 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:09:34.719543 29527 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.713124 29607 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:34.729009 29030 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.710387 29505 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:34.731379 22652 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:34.707634 29536 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:34.733965 29519 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.709023 29558 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:34.736389 29488 raft_consensus.cc:697] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:34.743737 29277 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:09:34.747296 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:34.751214 29251 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:34.717070 29399 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:34.757700 28563 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 27 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:34.761248 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:34.735019 29469 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.754415 23448 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:34.750691 29139 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:34.750311 29448 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:34.756063 29479 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:34.758711 29527 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:34.759298 29607 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:34.775579 28563 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.785892 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:34.784265 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:34.789757 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:34.793056 29507 raft_consensus.cc:3055] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:34.797530 29507 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:34.793900 29479 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:34.794364 29469 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:34.798715 29469 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:34.799043 29479 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.798739 29630 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:34.800199 29630 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.799567 29469 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.800534 29630 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:34.800726 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.802816 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.802919 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:34.804338 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:34.805471 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:34.807036 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.812144 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.815626 29488 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:34.817212 29593 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:34.818742 29030 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:34.819212 29630 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:34.819837 29629 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:34.821421 29630 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:34.755960 29605 consensus_queue.cc:1059] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:09:34.829767 22749 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 20: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:34.824242 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:34.768615 29505 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:34.776916 29448 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.747381 29530 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.839833 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:34.778822 29251 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:34.841797 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.782212 29501 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
W20260709 13:09:34.851122 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 104410us
I20260709 13:09:34.853799 29321 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:34.857004 29251 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:34.860761 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57939.5us
I20260709 13:09:34.840924 29448 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:34.837262 29576 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:09:34.845059 29593 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:34.849223 29591 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:34.859056 29474 raft_consensus.cc:2764] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:09:34.876880 22719 leader_election.cc:400] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:34.860811 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.884045 22719 leader_election.cc:400] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17.
I20260709 13:09:34.885003 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.859375 29536 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:09:34.888767 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.889991 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:34.890501 22828 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:34.885643 29061 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:34.896874 29061 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:34.899722 29061 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:34.900674 29486 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.906095 29448 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.884464 29479 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:09:34.908236 29448 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:34.908344 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.906515 29399 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:34.892330 29576 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:34.911837 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:34.915270 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 173378us
I20260709 13:09:34.891800 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.893661 29618 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:34.918500 29628 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:09:34.917755 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:34.916111 29560 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:34.922322 29560 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:34.922681 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:34.918984 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:34.925495 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.927799 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:34.955608 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:34.956352 23451 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:34.963958 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:35.024049 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:35.024858 22828 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:35.039759 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 112024us
W20260709 13:09:34.919713 22702 auto_rebalancer.cc:893] Removing replace marker failed: Replica 3b626c41600d4737a178ebab8a7ebfc2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:09:35.041115 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 122193us
I20260709 13:09:34.883210 29505 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.062592 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:35.063805 23451 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
W20260709 13:09:35.064323 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 100472us
W20260709 13:09:35.062028 22702 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:09:34.912027 29448 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:34.893388 29501 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:09:35.071898 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:34.915926 29576 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:34.919152 29591 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:35.071926 22702 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:09:35.086619 22828 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:35.099685 22831 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:34.913796 29592 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.166s
I20260709 13:09:35.062825 29469 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:35.077965 29479 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:35.079311 29561 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:35.077790 29628 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:09:35.132406 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 60668.7us
I20260709 13:09:35.160835 29479 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:35.097981 29605 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:35.134554 29266 raft_consensus.cc:2764] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:09:35.101872 28992 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.163172 28992 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.136897 29630 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 13:09:35.164353 29560 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.165199 28992 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:35.144503 29466 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:35.165908 29486 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.166270 29466 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.161468 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:35.168334 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:35.168535 29466 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.168699 29384 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:35.169224 29384 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.169704 29384 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:35.170243 29277 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:35.170747 29277 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.171154 29277 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:35.103116 29629 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:09:35.174193 29466 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:35.119156 29576 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:35.175745 29061 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.178622 29061 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.175271 29466 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:35.184324 29466 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.195684 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:35.167794 29486 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:35.205561 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:35.205744 29466 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:35.206571 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:35.079388 29593 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.212791 28992 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:34.929095 29618 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.213689 29396 raft_consensus.cc:3055] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:35.214036 28992 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:35.214644 28992 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:35.215258 29618 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.214083 29396 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:35.216274 29396 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.216435 29580 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.217447 29396 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:35.218611 29640 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
I20260709 13:09:35.215279 29536 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.221618 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:35.222219 23449 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:35.221987 29536 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.224045 29536 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:35.225026 22899 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:35.225481 23450 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:35.226166 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:35.226804 22896 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
W20260709 13:09:35.231549 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:35.231750 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:35.232568 22825 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:35.232923 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:35.233724 22827 raft_consensus.cc:2410] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:09:35.236483 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 135738us
I20260709 13:09:35.237411 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:35.239698 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:35.251664 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 21 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:35.252391 23449 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:35.252653 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:35.253471 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:35.254058 22827 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:35.124467 29634 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:35.256254 22896 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:35.257908 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:35.257018 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:35.258109 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:35.277968 22896 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: [13.8-16.9]
I20260709 13:09:35.278378 22826 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:35.258318 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:35.258886 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:35.266628 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:35.254655 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:35.281445 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:35.279460 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:35.274142 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:35.304347 22827 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:35.304029 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:35.312709 22826 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:35.314637 22826 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:09:35.285593 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:35.315886 22823 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:35.320839 22899 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:35.325173 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:35.325899 22905 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:35.326922 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:35.327587 22900 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:35.274192 29480 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:35.332139 22903 raft_consensus.cc:1240] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:35.328920 29639 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:35.331794 29648 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:09:35.338425 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57089.7us
I20260709 13:09:35.250119 29628 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:35.340489 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:35.341266 22831 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 12.
I20260709 13:09:35.339188 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:35.284288 29634 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.313714 29580 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:35.311262 29536 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:35.233239 29486 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.366624 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 21 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:35.370699 22900 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:35.370891 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:35.310444 29530 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:35.329282 29061 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.380069 22823 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:35.273790 29277 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.215421 28992 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.284149 29377 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.090s
I20260709 13:09:35.251466 29527 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.383740 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:35.365383 29607 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.543s	user 0.002s	sys 0.008s
I20260709 13:09:35.385047 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:35.328992 29546 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:09:35.365528 29634 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:35.386181 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 142542us
I20260709 13:09:35.387737 22750 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: [21.17-22.18]
I20260709 13:09:35.353556 29030 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:35.357159 29629 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:09:35.387135 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:35.417948 22829 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: [17.16-17.16]
I20260709 13:09:35.386123 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:35.394163 22826 raft_consensus.cc:1217] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Deduplicated request from leader. Original: 12.7->[13.8-16.9]   Dedup: 13.8->[16.9-16.9]
I20260709 13:09:35.364610 29648 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:35.378000 29628 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:35.398217 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:35.390645 22900 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 20: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:35.392977 22903 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:35.371387 29630 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.027s
I20260709 13:09:35.426601 22897 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:35.419766 22826 raft_consensus.cc:1240] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: [13.8-16.9]
I20260709 13:09:35.417248 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:35.420228 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 162395us
I20260709 13:09:35.401914 22830 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:09:35.371886 29480 consensus_queue.cc:1059] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:09:35.400302 29644 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:35.415445 29030 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:35.422031 29638 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:09:35.385448 29530 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.388383 29061 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.429949 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:35.440418 29486 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.441084 29377 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:35.441681 29377 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:35.446271 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:35.397603 29607 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.424609 28992 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:35.428539 29531 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:35.436440 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:35.383996 29647 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:35.446910 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:35.434609 29630 consensus_queue.cc:1243] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 23.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 13:09:35.451380 22754 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:35.434764 29030 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:35.447031 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:35.451684 29636 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:09:35.436288 29644 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:35.454787 29576 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.457113 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:35.478075 22747 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:35.461390 29427 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:35.442634 29377 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.462191 29648 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.480576 29628 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.487167 29277 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.488437 29469 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:35.492323 29629 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:35.492800 29629 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.463521 23449 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 25.
I20260709 13:09:35.438400 29519 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:35.474351 23452 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:35.466302 29530 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:35.480178 22828 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:09:35.451076 29269 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 13:09:35.495048 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:35.449731 29536 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:35.495704 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:35.496820 23445 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 24. Current term is 25. Ops: [24.21-24.21]
I20260709 13:09:35.497537 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:35.500588 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:35.469523 29647 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.506556 23444 raft_consensus.cc:2393] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 12: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:09:35.505947 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 69788.5us
I20260709 13:09:35.508083 23448 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:35.511590 23448 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:35.513334 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:35.513511 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:35.514684 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:35.515545 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:35.520020 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67026.4us
I20260709 13:09:35.520887 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:35.516391 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:35.544770 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:35.505960 29629 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.499677 29636 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.547500 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:35.549109 22905 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:09:35.549467 29474 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:35.549643 29561 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:35.500015 29519 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:35.475765 29531 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.495623 29634 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.505568 29633 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:35.516780 29655 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:09:35.576663 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:35.522604 29277 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:35.625980 29269 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:35.543622 29427 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:35.493301 29384 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.555253 29629 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:35.670347 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:35.549168 29527 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.554260 29659 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:35.559919 29519 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:35.557253 29424 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:35.686674 22825 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:35.551249 29630 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.132s
I20260709 13:09:35.557104 29636 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.749842 29576 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:35.751279 29466 consensus_meta.cc:287] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.000s
W20260709 13:09:35.752298 29396 consensus_meta.cc:287] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.001s
I20260709 13:09:35.752671 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:35.624666 29266 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.070s
W20260709 13:09:35.753660 29612 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.061s	user 0.001s	sys 0.012s
I20260709 13:09:35.754994 29624 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:35.757407 29507 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.763939 29634 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.739768 29629 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.770649 29061 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:35.508122 29647 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:35.557578 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:35.845223 23450 raft_consensus.cc:2393] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:09:35.849733 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 292306us
I20260709 13:09:35.849110 29507 raft_consensus.cc:3060] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:35.846261 29266 consensus_queue.cc:1243] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 22.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:09:35.855382 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1694) took 1508 ms. Trace:
I20260709 13:09:35.855501 29629 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.856946 29377 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:35.842698 29624 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:35.855806 23453 rpcz_store.cc:276] 0709 13:09:34.348732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.348884 (+   152us) service_pool.cc:224] Handling call
0709 13:09:34.365485 (+ 16601us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:34.365935 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:34.365962 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:34.394183 (+ 28221us) log.cc:844] Serialized 52 byte log entry
0709 13:09:34.573730 (+179547us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:34.573786 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.573871 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:35.753003 (+1179132us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:35.855082 (+102079us) raft_consensus.cc:1685] finished
0709 13:09:35.855106 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:35.855322 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12224,"raft.queue_time_us":2930,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":22159,"spinlock_wait_cycles":24739200,"thread_start_us":151156,"threads_started":1,"wal-append.queue_time_us":251276}
W20260709 13:09:35.851270 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 303854us
I20260709 13:09:35.859908 29636 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:35.867296 29624 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.868417 29624 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:35.860772 29655 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:35.861095 23451 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:35.852092 29634 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:35.857986 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:35.677796 29644 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.742764 29536 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.879748 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:35.875952 29634 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:35.885700 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:35.889132 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:35.890169 23447 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:35.881443 29266 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:35.891404 29266 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.882403 29348 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:09:35.891736 29266 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:35.883869 29607 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.881675 29527 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:35.900390 23447 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:35.879009 29636 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:09:35.894166 29634 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:35.922288 29396 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.922751 29396 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.873878 29377 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.897122 29527 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:35.924727 29527 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.925638 29519 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.928359 29519 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.927867 29647 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.938413 29624 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:35.937158 29656 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.939102 29656 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.939692 29527 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:35.939785 29647 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:35.941380 29656 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:35.942004 29624 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:35.942853 29624 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.943981 29624 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:35.759833 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:35.945117 29634 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.949193 29634 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:35.936981 28992 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.953896 28992 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.957423 29396 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:35.957669 28992 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:35.960227 22650 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 343) took 18234 ms (client timeout 15000 ms). Trace:
I20260709 13:09:35.960932 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 3033) took 1160 ms. Trace:
I20260709 13:09:35.961437 29396 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.961329 22749 rpcz_store.cc:276] 0709 13:09:34.800098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.804065 (+  3967us) service_pool.cc:224] Handling call
0709 13:09:35.960734 (+1156669us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b54003cebd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:35.960851 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39749504}
I20260709 13:09:35.965879 29624 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.967922 29083 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:35.968343 29083 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:35.969169 29083 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:35.971433 29083 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:35.962160 29396 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:35.976161 29266 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:35.983351 29030 consensus_meta.cc:287] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.001s
I20260709 13:09:35.987422 29377 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:35.960644 22650 rpcz_store.cc:269] 0709 13:09:17.726191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:17.726420 (+   229us) service_pool.cc:224] Handling call
0709 13:09:35.960157 (+18233737us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:29.167212 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:29.167892 (+   680us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:29.167954 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:09:29.167971 (+    17us) tablet.cc:662] Decoding operations
0709 13:09:29.495429 (+327458us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:29.495502 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:29.495526 (+    24us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:09:29.496987 (+  1461us) tablet.cc:701] Row locks acquired
0709 13:09:29.497038 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:09:29.497204 (+   166us) write_op.cc:270] Start()
0709 13:09:29.497360 (+   156us) write_op.cc:276] Timestamp: P: 1783602569497146 usec, L: 0
0709 13:09:29.497380 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:09:29.498173 (+   793us) log.cc:844] Serialized 8540 byte log entry
0709 13:09:30.388062 (+889889us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b1800074800. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e522 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:30.710593 (+322531us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:30.911260 (+200667us) op_driver.cc:464] REPLICATION: finished
0709 13:09:31.344140 (+432880us) write_op.cc:301] APPLY: starting
0709 13:09:31.344269 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:31.596073 (+251804us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:31.596117 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:31.811551 (+215434us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:31.811989 (+   438us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:09:31.812034 (+    45us) write_op.cc:312] APPLY: finished
0709 13:09:31.812532 (+   498us) log.cc:844] Serialized 188 byte log entry
0709 13:09:31.824336 (+ 11804us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:31.859306 (+ 34970us) write_op.cc:454] Released schema lock
0709 13:09:31.882798 (+ 23492us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:31.966099 (+ 83301us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:31.966134 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":731128,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":430202,"apply.run_cpu_time_us":30382,"apply.run_wall_time_us":588288,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91907,"num_ops":21,"prepare.queue_time_us":128669,"prepare.run_cpu_time_us":15018,"prepare.run_wall_time_us":568729,"raft.queue_time_us":142813,"raft.run_cpu_time_us":18067,"raft.run_wall_time_us":291982,"replication_time_us":1402074,"spinlock_wait_cycles":27941760,"thread_start_us":552311,"threads_started":4,"wal-append.queue_time_us":256882,"wal-append.run_cpu_time_us":34394,"wal-append.run_wall_time_us":3059961}]]}
W20260709 13:09:36.118086 22827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.013s	sys 0.032s
W20260709 13:09:36.118496 22827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.013s	sys 0.032s
W20260709 13:09:36.118611 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:35.964774 29384 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.109014 29501 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:36.109724 29350 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:36.106792 29507 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:36.108872 29527 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:36.129819 29350 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:36.132964 29266 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.135521 29507 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.137236 29350 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.138509 29507 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.138809 29350 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:36.134150 29608 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.434s	user 0.010s	sys 0.012s
I20260709 13:09:36.135434 29527 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.142644 29527 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:36.132597 29668 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:09:36.144835 22822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3169) took 1828 ms. Trace:
I20260709 13:09:36.023257 29624 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.116974 29030 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.120159 29488 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:36.126389 29501 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:36.177349 29030 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.177135 22822 rpcz_store.cc:276] 0709 13:09:34.316740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.337789 (+ 21049us) service_pool.cc:224] Handling call
0709 13:09:34.352775 (+ 14986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:34.666585 (+313810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:36.144636 (+1478051us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400231ed4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:36.144785 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638947584}
I20260709 13:09:36.150892 29650 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:36.177348 29624 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:36.169154 29266 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.179340 29266 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.179145 29501 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.21, Last appended by leader: 21, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.169903 22902 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:09:36.178545 29650 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:36.170269 22901 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 2901) took 1541 ms (client timeout 1625 ms). Trace:
I20260709 13:09:36.170456 22827 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: [18.13-19.14]
I20260709 13:09:36.170663 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:36.185595 22901 rpcz_store.cc:269] 0709 13:09:34.628372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.628577 (+   205us) service_pool.cc:224] Handling call
0709 13:09:36.169452 (+1540875us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b54000d0f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:36.170217 (+   765us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97202944}
I20260709 13:09:36.186762 22826 raft_consensus.cc:2410] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 20 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:09:36.188077 22827 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:36.189606 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.190215 22825 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:36.181056 29501 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:36.192061 22825 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:36.193881 22826 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:36.194906 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.195708 29650 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.195750 22822 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:36.199515 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:36.200556 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.201092 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.201871 22749 raft_consensus.cc:2410] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:09:36.203307 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:36.203820 23447 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:36.203944 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:36.204813 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:36.207019 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:36.207507 23448 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:36.211232 23447 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:36.211964 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:36.216390 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:36.221555 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.226383 29030 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:36.226557 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.227177 29030 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:36.227521 29030 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:36.228060 23444 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:36.228724 29616 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.229204 29616 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.229982 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.231186 29616 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.233002 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:36.233582 22753 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:09:36.228313 29030 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.235463 22749 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:09:36.237286 22749 pending_rounds.cc:85] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 13
I20260709 13:09:36.237615 22749 pending_rounds.cc:107] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:09:36.237846 22749 raft_consensus.cc:2889] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:36.236619 22753 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:36.195847 29266 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.244217 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3035) took 1435 ms. Trace:
I20260709 13:09:36.244462 22747 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:09:36.246933 22747 pending_rounds.cc:85] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 11
I20260709 13:09:36.248776 22747 pending_rounds.cc:107] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:09:36.249415 22747 raft_consensus.cc:2889] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:36.248771 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:36.246803 22751 rpcz_store.cc:276] 0709 13:09:34.809271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.809755 (+   484us) service_pool.cc:224] Handling call
0709 13:09:34.838576 (+ 28821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:34.838811 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:34.847467 (+  8656us) log.cc:844] Serialized 14 byte log entry
0709 13:09:34.847853 (+   386us) log.cc:844] Serialized 14 byte log entry
0709 13:09:35.437939 (+590086us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b18000b0320. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:36.171008 (+733069us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b18000b0320. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb749b1d70 00007fcb749a5293 00007fcb7499acea 00007fcb7499b23a 00007fcb74e05a2a 00007fcb74e150bf 00007fcb74e15038
0709 13:09:36.243473 (+ 72465us) log.cc:844] Serialized 14 byte log entry
0709 13:09:36.243916 (+   443us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:36.243962 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:36.244014 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:36.244069 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:36.244171 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1052,"prepare.queue_time_us":623795,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":1308,"raft.queue_time_us":1619,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":5143,"spinlock_wait_cycles":12654208,"thread_start_us":71351,"threads_started":1}
I20260709 13:09:36.241014 23448 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:36.243961 29525 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:36.244922 29624 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:36.259650 29558 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.261101 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:36.261745 29665 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.429s	user 0.006s	sys 0.000s
I20260709 13:09:36.256982 29628 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:36.264560 29615 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:09:36.265008 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 53097.8us
W20260709 13:09:36.268340 29677 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:09:36.251626 29295 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:09:36.272060 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.275796 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.255584 29030 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:36.283150 29615 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:36.284466 29531 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.243785 29377 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 13:09:36.288225 29637 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.154s	user 0.029s	sys 0.040s
I20260709 13:09:36.279335 29628 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:36.289965 29628 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:36.284471 29615 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.299070 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:36.299592 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.282085 29558 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:36.282115 22824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3184) took 1254 ms. Trace:
I20260709 13:09:36.301492 22753 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:09:36.278537 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:36.304385 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:36.305509 22754 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:36.310117 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:36.315423 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:36.310392 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:36.316669 22748 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:36.317799 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:36.311655 22824 rpcz_store.cc:276] 0709 13:09:35.028086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.028232 (+   146us) service_pool.cc:224] Handling call
0709 13:09:35.028619 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:35.087889 (+ 59270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:35.087936 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:35.105590 (+ 17654us) log.cc:844] Serialized 26 byte log entry
0709 13:09:36.172280 (+1066690us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b1800049480. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:09:36.192375 (+ 20095us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:36.268852 (+ 76477us) log.cc:844] Serialized 14 byte log entry
0709 13:09:36.269151 (+   299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:36.269245 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:36.281806 (+ 12561us) raft_consensus.cc:1685] finished
0709 13:09:36.281918 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:36.282045 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58780,"prepare.queue_time_us":61181,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":88881,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":18627,"spinlock_wait_cycles":83130240,"thread_start_us":95917,"threads_started":2,"wal-append.queue_time_us":1064872}
I20260709 13:09:36.320655 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.268803 29624 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:36.321609 22747 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:36.320695 22749 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:36.322540 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.323108 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:36.269025 29402 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:36.323318 29650 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:36.326072 29652 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:36.326182 29633 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:36.332590 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.333835 22751 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
W20260709 13:09:36.338779 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.343001 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.346374 22754 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
W20260709 13:09:36.347654 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.348874 29546 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:36.349617 29546 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.350401 29580 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.350678 29061 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:36.350909 29580 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.351238 29061 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.351817 29546 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.352476 29546 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.352887 29546 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.353259 29061 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.353502 22899 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:36.354046 29649 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.354701 29649 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.355127 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:36.355686 22900 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:36.355792 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.356264 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:36.356705 29649 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.356959 22902 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:36.357388 29649 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.357578 29576 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:36.357933 29649 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.358404 22897 raft_consensus.cc:1275] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:09:36.358512 29576 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:36.359191 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.359738 23450 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:36.359875 29649 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.360399 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.357625 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:36.357944 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:36.353535 29577 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.361171 22905 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:36.361460 29577 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.362398 29580 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:36.363425 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.354107 22899 raft_consensus.cc:606] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:36.364414 22899 raft_consensus.cc:614] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:36.364809 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:36.365607 29660 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:36.366055 29660 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:36.366797 29660 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.21, Last appended by leader: 21, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.367898 22905 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:36.359799 29576 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:36.369133 29576 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:36.369760 22903 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:09:36.370134 22903 pending_rounds.cc:85] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 15
I20260709 13:09:36.369908 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:36.371052 29061 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:36.371466 29061 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:36.371660 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:36.371877 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:36.372519 22896 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:36.369894 29576 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.374444 29576 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:36.374879 22902 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:36.375370 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:36.381752 22899 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:36.375883 23450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:36.382627 23450 raft_consensus.cc:606] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:36.383867 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 105411us
I20260709 13:09:36.370528 22903 pending_rounds.cc:107] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:09:36.385221 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.385577 22903 raft_consensus.cc:2889] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:36.385798 22754 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:36.377457 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.378221 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.331641 29624 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:36.363525 29577 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:36.340045 29634 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.384498 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:36.391256 22900 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:36.392271 22748 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:36.339916 29652 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:36.391404 22794 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:36.397374 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.371919 29061 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:36.379848 29580 consensus_queue.cc:1175] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c45e1805f049467697a7358fb566de2f" responder_term: 17 status { last_received { term: 17 index: 16 } last_committed_idx: 14 last_received_current_leader { term: 17 index: 16 } }
I20260709 13:09:36.381892 29650 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:36.398919 29480 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:36.394591 29350 raft_consensus.cc:2764] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:09:36.339622 29402 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.389722 29479 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:36.411326 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:36.397969 29251 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:36.413898 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:36.397811 29616 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:36.398842 29652 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:36.406661 29505 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:36.419610 29480 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.428601 29295 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:36.430013 29672 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:36.430373 29480 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.430632 23419 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:36.430503 29672 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.432446 29672 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.432920 29474 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.405707 29670 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:36.398123 29634 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.433421 29251 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:36.437809 29083 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:09:36.440995 29561 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:36.421229 29399 raft_consensus.cc:2804] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:36.419683 29479 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:36.454957 29402 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:36.455847 23450 raft_consensus.cc:1240] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:36.436012 29525 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:09:36.470902 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.478410 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.433426 29505 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.438074 29474 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:36.466202 29399 raft_consensus.cc:697] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:36.470171 29486 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:36.446885 29670 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.475484 29572 consensus_queue.cc:1059] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:09:36.523131 29507 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.530064 29616 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:36.559835 29670 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.528232 29251 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.564999 29605 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:36.565726 29605 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:36.565395 29399 consensus_queue.cc:237] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.566515 29605 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:36.567507 29683 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:09:36.523653 29486 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.472846 29650 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.572621 29505 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.539407 29546 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.574276 29486 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:36.583702 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 168071us
I20260709 13:09:36.542814 29635 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.567562 29605 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:36.586743 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:36.574479 29474 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:36.587689 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2907) took 1199 ms. Trace:
I20260709 13:09:36.588685 29696 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:36.589238 29696 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.521627 29479 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.591719 22794 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:36.581161 29628 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:36.623076 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.551774 29507 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.628163 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:36.601804 29139 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:36.600960 29694 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:09:36.634078 29507 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.613619 29576 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:36.655210 29388 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:36.601796 29266 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:36.655766 29388 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:36.656085 29266 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.657508 29479 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:36.656755 29388 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.591429 29635 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.660070 29635 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:36.660348 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.660688 29388 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:36.661705 29061 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.628252 29616 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.658429 29266 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.664058 29061 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:36.675350 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 47299us
W20260709 13:09:36.638671 29690 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:09:36.652927 29666 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.796s	user 0.006s	sys 0.012s
I20260709 13:09:36.655346 29576 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.640208 29628 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:36.640204 29634 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.651446 29696 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.649514 29655 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.666649 29321 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:36.695994 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:36.629704 29699 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:36.676534 29266 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:36.651870 29624 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.651000 29652 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.706452 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:36.724080 28696 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:36.725831 28696 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.730443 28696 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.735086 29634 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:36.739058 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:36.739303 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.739650 22826 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:36.738867 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.739940 22825 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:36.740343 22822 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:36.749015 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.735621 29634 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:36.704312 29696 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:36.749487 22753 raft_consensus.cc:3060] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:36.710531 29576 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.706636 29699 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:36.731261 29424 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:36.756748 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:36.698891 29399 consensus_queue.cc:1048] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
W20260709 13:09:36.752202 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 137255us
I20260709 13:09:36.732983 29083 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:36.699438 29635 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:09:36.761265 22753 raft_consensus.cc:2468] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 13.
I20260709 13:09:36.765769 22826 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:09:36.706234 29694 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:09:36.700392 29321 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.774624 29083 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.757006 29634 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:36.770658 29652 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.780544 29083 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:36.781533 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.788900 22827 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:09:36.783538 23444 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:36.783617 23450 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:09:36.790083 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.786289 23446 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 13:09:36.786911 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.700491 29655 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.787423 29576 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:36.784176 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.790768 22750 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:36.793444 22830 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:36.796061 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.796665 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.769264 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:36.797472 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.851929 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:36.869984 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 100864us
W20260709 13:09:36.870628 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.798064 22829 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:09:36.870054 22751 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
W20260709 13:09:36.800529 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.871591 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.872455 22829 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:09:36.872716 23448 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 17. Ops: []
I20260709 13:09:36.805673 22823 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:36.872605 22825 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 13.
I20260709 13:09:36.800611 22752 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:09:36.874369 22752 pending_rounds.cc:85] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 11
I20260709 13:09:36.874539 22754 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:36.848335 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.874752 22752 pending_rounds.cc:107] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:09:36.873756 22829 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:36.875385 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.883810 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:36.768869 29635 consensus_queue.cc:1243] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:09:36.888394 22752 raft_consensus.cc:2889] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:36.891064 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.894133 22824 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:09:36.895152 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.892277 22752 pending_rounds.cc:107] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:09:36.896304 22752 raft_consensus.cc:2889] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:36.896862 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.876418 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.847370 29083 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:36.770675 29624 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:36.777341 29321 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:36.890829 29592 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.747s	user 0.003s	sys 0.025s
I20260709 13:09:36.896960 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.875487 29628 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.875881 29576 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.908679 29628 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.909390 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.910338 29576 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:36.910565 29628 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.912153 29650 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:36.913203 29648 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:36.913585 29648 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:36.913805 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:36.914500 23444 raft_consensus.cc:2368] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13.
I20260709 13:09:36.914367 29648 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.915575 29648 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:36.875404 29469 raft_consensus.cc:476] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Not starting pre-election -- already a leader
I20260709 13:09:36.917407 29639 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:36.918278 29639 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:36.918639 29639 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:36.919413 29639 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.891582 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:36.920317 29634 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:36.876732 29384 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:36.893555 29266 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:36.906805 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.768343 29424 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.927381 22830 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:36.931540 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.932305 22830 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:36.850773 29699 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:36.907559 29592 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.764s	user 0.003s	sys 0.025s
I20260709 13:09:36.893354 29618 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:09:36.920378 29639 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:36.905999 29652 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:36.953814 23450 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:36.926087 29616 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:36.930335 29266 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:36.959399 29266 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:36.959473 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:36.964847 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.965466 23449 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:36.898312 29706 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:09:36.969666 29266 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.942106 29469 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:09:36.934844 29630 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:09:36.945235 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:36.945871 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:36.946640 29650 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:36.946857 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:36.946928 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:36.973419 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:36.945003 22822 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:36.991905 22901 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:36.992694 22903 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:36.935722 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:36.980444 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.992686 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:36.994256 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.994527 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:36.906594 29083 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:36.996665 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:36.997622 22830 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:36.998661 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:37.000922 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:37.005040 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:37.006798 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:37.014976 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:36.983187 29696 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:36.978520 29572 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.983521 29424 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:37.035970 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.994903 29321 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.985806 29634 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:36.976451 29266 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:36.972445 29652 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.047608 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:36.984651 29578 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:09:37.062027 29321 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.093858 28696 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:37.022819 29479 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:37.095017 29321 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:37.096808 29652 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:37.024984 29712 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:37.100296 29712 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.101181 29618 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.101835 29618 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.057196 29536 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:37.102900 29536 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:37.103263 29536 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:37.057758 29572 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.115927 23449 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:37.120347 29618 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.125350 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:37.125661 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 78170.7us
I20260709 13:09:37.126336 22868 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:37.127228 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:37.130301 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 194652us
I20260709 13:09:37.132393 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:37.162246 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:37.163239 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:37.164639 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:37.165079 28696 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:09:37.178822 22719 leader_election.cc:336] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.773s (SENT)
I20260709 13:09:37.179364 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:37.182303 29576 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:37.185755 29648 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:37.186132 29576 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:37.186450 29648 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:37.187139 29648 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:37.182515 29639 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:37.188517 29639 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:37.188889 29639 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:37.186626 29576 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:37.189745 29546 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:37.190150 29546 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:37.190481 29546 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:37.197181 23449 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 22 index: 20. Preceding OpId from leader: term: 25 index: 22. (index mismatch)
W20260709 13:09:37.198709 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:37.183032 29592 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:37.215056 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:37.215636 23444 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:37.203073 29646 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:09:37.222487 23451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38090 (request call id 1785) took 1983 ms (client timeout 1700 ms). Trace:
W20260709 13:09:37.222995 23451 rpcz_store.cc:269] 0709 13:09:35.239290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.239448 (+   158us) service_pool.cc:224] Handling call
0709 13:09:37.222277 (+1982829us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54005e1ad4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:37.222404 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427253632}
I20260709 13:09:37.204684 29572 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.207561 29616 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:37.288595 23449 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:09:37.221683 29658 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.227872 29712 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:37.293726 29572 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.317683 29616 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.305246 29658 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.326043 29321 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.336022 29083 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:37.316708 29572 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.360231 29321 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.384999 22797 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:37.386125 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:37.389078 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 2287) took 2124 ms. Trace:
I20260709 13:09:37.389518 22828 rpcz_store.cc:276] 0709 13:09:35.264517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.264670 (+   153us) service_pool.cc:224] Handling call
0709 13:09:35.265044 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:35.498691 (+233647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.375754 (+1877063us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54003822d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:37.389012 (+ 13258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494068096}
I20260709 13:09:37.390633 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:37.392843 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 2069) took 1165 ms. Trace:
I20260709 13:09:37.393177 22750 rpcz_store.cc:276] 0709 13:09:36.227669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.232651 (+  4982us) service_pool.cc:224] Handling call
0709 13:09:36.244762 (+ 12111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:36.869526 (+624764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.376293 (+506767us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b5400046854. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:37.392795 (+ 16502us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255278336}
W20260709 13:09:37.398774 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:37.413578 22828 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:09:37.415925 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:37.416625 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:37.388659 29713 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:09:37.432741 28696 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:37.436564 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 218830us
I20260709 13:09:37.410316 29605 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:37.440965 23447 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:37.440080 22751 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:09:37.442000 22751 pending_rounds.cc:85] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 11
I20260709 13:09:37.410810 29558 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:37.442231 22751 pending_rounds.cc:107] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:09:37.442584 22751 raft_consensus.cc:2889] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:37.414585 29646 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:37.424256 29630 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.416143 29719 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:37.449273 28696 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.436340 29479 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.441218 29652 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:37.441358 29507 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:37.441135 29605 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:37.446187 29424 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:37.448943 29592 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:37.586685 29592 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.592947 29572 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:37.437913 29061 consensus_queue.cc:1059] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 13:09:37.597752 29061 raft_consensus.cc:3055] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:37.598171 29061 raft_consensus.cc:740] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:37.599097 29061 consensus_queue.cc:260] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.444078 29712 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.132s
I20260709 13:09:37.615533 29321 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:37.616343 29396 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:37.617903 29725 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.491086 29719 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:37.619258 29719 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:37.445783 29558 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:37.438022 29722 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.216s
I20260709 13:09:37.453063 29727 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.206s
I20260709 13:09:37.466859 29646 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.495143 29630 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.618521 29725 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.591637 29658 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.546186 29399 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.638020 29321 raft_consensus.cc:993] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:37.677016 29061 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:37.692185 29558 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:37.560184 29479 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:37.697479 29546 consensus_meta.cc:287] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.507s	user 0.004s	sys 0.007s
I20260709 13:09:37.612877 29648 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.621429 29507 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:37.714278 29536 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.000s
W20260709 13:09:37.713281 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 275309us
I20260709 13:09:37.621764 29652 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:37.717240 29576 consensus_meta.cc:287] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.001s
I20260709 13:09:37.682169 29605 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.714563 29669 raft_consensus.cc:1081] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:09:37.703007 29399 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.742354 29576 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:37.765836 29639 consensus_meta.cc:287] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.000s
I20260709 13:09:37.766378 29639 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.767580 29576 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:37.768746 29639 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:37.736333 29536 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.774174 29399 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.784087 29630 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.785862 29652 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.753901 29605 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:37.790161 29725 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.791617 29479 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.793352 29648 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.794772 29646 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:37.718863 29546 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.810576 29546 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:37.812242 29536 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:37.817606 29507 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:37.848691 29719 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.854805 29719 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:37.916546 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3067) took 1120 ms. Trace:
I20260709 13:09:37.940007 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1397) took 1009 ms. Trace:
I20260709 13:09:37.938766 22755 rpcz_store.cc:276] 0709 13:09:36.795685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.798967 (+  3282us) service_pool.cc:224] Handling call
0709 13:09:36.799292 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:36.799662 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:36.799699 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:36.877370 (+ 77671us) log.cc:844] Serialized 26 byte log entry
0709 13:09:37.293436 (+416066us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:37.293493 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.293585 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:37.915705 (+622120us) raft_consensus.cc:1685] finished
0709 13:09:37.915787 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.916211 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1366,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":12254,"thread_start_us":397970,"threads_started":1,"wal-append.queue_time_us":541414}
W20260709 13:09:37.919075 22824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.015s	sys 0.024s
W20260709 13:09:37.920032 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 486123us
W20260709 13:09:37.941790 22824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.015s	sys 0.024s
I20260709 13:09:37.940627 22830 rpcz_store.cc:276] 0709 13:09:36.933225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.119738 (+186513us) service_pool.cc:224] Handling call
0709 13:09:37.120094 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:37.184518 (+ 64424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:37.184564 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:37.185146 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:09:37.438924 (+253778us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:37.438958 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.439023 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:37.939307 (+500284us) raft_consensus.cc:1685] finished
0709 13:09:37.939354 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.939744 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64030,"raft.queue_time_us":66839,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":67,"thread_start_us":253440,"threads_started":1,"wal-append.queue_time_us":453261}
I20260709 13:09:37.932117 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3194) took 1153 ms. Trace:
I20260709 13:09:37.943497 22826 rpcz_store.cc:276] 0709 13:09:36.779055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.793790 (+ 14735us) service_pool.cc:224] Handling call
0709 13:09:36.794045 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:36.794337 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:36.794362 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:36.794763 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 13:09:36.992410 (+197647us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:37.217408 (+224998us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.221801 (+  4393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.221885 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:37.931468 (+709583us) raft_consensus.cc:1685] finished
0709 13:09:37.931516 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.931885 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4132,"prepare.queue_time_us":362181,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":2828,"raft.queue_time_us":16534,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":43208,"thread_start_us":420617,"threads_started":2,"wal-append.queue_time_us":258248}
W20260709 13:09:37.960991 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:37.962327 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:37.976786 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 459979us
I20260709 13:09:37.986045 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:37.986636 29732 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:37.975927 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:37.987572 29558 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.982939 29635 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:37.993243 29639 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.994268 29635 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.995898 29536 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:37.997047 29536 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:37.987757 29732 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:37.994410 29639 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.013411 22755 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:38.015024 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.015198 29558 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:38.015935 22755 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 10.
I20260709 13:09:38.016795 29635 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:38.018234 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.029505 29507 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.030501 29507 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:38.034287 22719 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.007s	sys 0.006s
W20260709 13:09:38.034736 22719 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.007s	sys 0.006s
W20260709 13:09:38.035220 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 648670us
W20260709 13:09:38.037051 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.039417 29536 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:38.039813 29605 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:38.040225 29725 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.040751 29605 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.041198 29725 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.041378 29660 raft_consensus.cc:3055] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:38.043655 29605 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.044045 29655 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:38.039454 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:38.045948 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:38.046607 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.047019 28696 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:38.047494 28696 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:38.047554 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 71996.4us
I20260709 13:09:38.047838 28696 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:38.048269 29469 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:38.049233 29469 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.052446 29469 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.044246 29660 raft_consensus.cc:740] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:38.056444 29732 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.055510 29507 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.061033 29639 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.043846 29725 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.053448 29722 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
W20260709 13:09:38.010234 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.007s	sys 0.016s
I20260709 13:09:38.023691 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.076993 29645 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:38.077574 29645 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:38.079967 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.081009 22750 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:38.081966 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.083535 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.084522 29639 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:38.084827 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.085291 29639 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.085752 22755 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:38.086217 29646 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:38.086413 29474 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:38.100731 29474 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.057163 29732 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.102874 29732 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.104682 29474 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.088811 29384 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:38.105645 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:38.106235 29384 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:38.106559 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38074 (request call id 1236) took 1233 ms. Trace:
I20260709 13:09:38.106720 29725 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:38.107015 23450 rpcz_store.cc:276] 0709 13:09:36.872557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.872662 (+   105us) service_pool.cc:224] Handling call
0709 13:09:38.106075 (+1233413us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b5400920254. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:38.106315 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148174720}
I20260709 13:09:38.109768 29725 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:09:38.112716 22756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.004s	sys 0.016s
W20260709 13:09:38.113190 22756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.004s	sys 0.016s
I20260709 13:09:38.114069 29536 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 13:09:38.117043 29083 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.117760 29083 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.120106 29083 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:38.124038 22719 leader_election.cc:400] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13.
I20260709 13:09:38.124413 22719 leader_election.cc:403] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:09:38.125689 29469 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 12. Current term is 13.
W20260709 13:09:38.128154 22753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 3054) took 1805 ms (client timeout 2026 ms). Trace:
W20260709 13:09:38.128657 22753 rpcz_store.cc:269] 0709 13:09:36.322772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.322899 (+   127us) service_pool.cc:224] Handling call
0709 13:09:38.127706 (+1804807us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b5400045454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:38.127932 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976375552}
W20260709 13:09:38.090767 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.007s	sys 0.016s
W20260709 13:09:38.096187 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.096119 28992 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.098614 29639 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.099351 29646 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.136399 22752 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 26 because replica has last-logged OpId of term: 24 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 20.
I20260709 13:09:38.113196 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:38.142494 23419 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:38.143591 29669 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:38.143993 29669 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:38.144385 29669 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:38.147053 29646 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.144305 22750 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:38.147689 29646 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:38.148977 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.150485 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.151665 29725 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:38.152190 29725 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:38.152637 29725 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:38.152668 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.153529 29469 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:38.154604 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.155128 29648 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.155503 29546 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:38.155668 29648 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.156162 29660 consensus_queue.cc:260] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.157241 29546 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.157533 28992 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.157953 29660 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:38.158073 29546 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:38.160090 28992 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:38.160549 22751 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:38.162770 22755 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:38.160398 29546 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:38.136623 29732 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.190064 29536 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:38.190824 29536 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:38.192996 29536 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:38.193653 29536 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:38.193815 29643 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.194543 29536 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.195371 29576 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.196019 29576 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.197772 29536 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.198154 29576 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.198364 29536 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:38.199241 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.199513 29536 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.200004 22753 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:38.200960 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.209488 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.202040 29536 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:38.202287 29680 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.746s	user 0.002s	sys 0.018s
I20260709 13:09:38.203903 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:38.204838 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:38.179736 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 742216us
I20260709 13:09:38.216045 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.221601 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.224231 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:38.225872 22825 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:38.226670 29083 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.227286 29083 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.229580 29083 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:38.229593 23446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1801) took 1271 ms. Trace:
I20260709 13:09:38.234444 23446 rpcz_store.cc:276] 0709 13:09:36.949579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.949704 (+   125us) service_pool.cc:224] Handling call
0709 13:09:38.106316 (+1156612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:38.106843 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:38.106869 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:38.106921 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:38.106952 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.107030 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.219425 (+112395us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400920250. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:38.220689 (+  1264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2637512064}
W20260709 13:09:38.222786 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.606s (SENT) [suppressed 7 similar messages]
I20260709 13:09:38.233361 28696 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.268688 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3064) took 1473 ms. Trace:
I20260709 13:09:38.269424 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:38.269296 22756 rpcz_store.cc:276] 0709 13:09:36.794888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.797886 (+  2998us) service_pool.cc:224] Handling call
0709 13:09:36.836753 (+ 38867us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:36.838007 (+  1254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:36.838038 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:37.371472 (+533434us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b2c00027958. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:09:37.412709 (+ 41237us) log.cc:844] Serialized 26 byte log entry
0709 13:09:38.113902 (+701193us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:38.113968 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.114089 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.268264 (+154175us) raft_consensus.cc:1685] finished
0709 13:09:38.268289 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.268613 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":1361,"spinlock_wait_cycles":73280768,"thread_start_us":673854,"threads_started":1,"wal-append.queue_time_us":789339}
I20260709 13:09:38.272099 29083 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:38.272653 29083 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:09:38.269515 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.194626 29643 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:38.299120 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:38.299963 23446 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:38.301000 22750 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:38.302105 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:36.588874 22898 rpcz_store.cc:276] 0709 13:09:35.388033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.418947 (+ 30914us) service_pool.cc:224] Handling call
0709 13:09:35.419233 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:35.419427 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:35.437442 (+ 18015us) log.cc:844] Serialized 14 byte log entry
0709 13:09:36.338665 (+901223us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b18002f9a40. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb74df3060 00007fcb74e05716 00007fcb74e16287 00007fcb74e16208
0709 13:09:36.453304 (+114639us) log.cc:844] Serialized 14 byte log entry
0709 13:09:36.575202 (+121898us) log.cc:844] Serialized 14 byte log entry
0709 13:09:36.583931 (+  8729us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:36.583977 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:36.584020 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:36.584082 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:36.587631 (+  3549us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9195,"prepare.queue_time_us":199863,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":2641,"raft.queue_time_us":911778,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":4895,"spinlock_wait_cycles":13430400,"thread_start_us":234931,"threads_started":2,"wal-append.queue_time_us":134687}
I20260709 13:09:38.317723 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:38.318295 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:38.621245 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.319787 23447 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:09:38.308333 22752 raft_consensus.cc:2410] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
W20260709 13:09:38.663457 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.320914 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.663968 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:38.664244 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:38.444965 22823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.022s	sys 0.026s
I20260709 13:09:38.664630 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.618683 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.318439 23446 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:09:38.665644 22755 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:38.665763 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:38.489095 29669 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.666837 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.313925 29083 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.667336 23451 raft_consensus.cc:1240] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:38.667557 22756 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:38.668429 22828 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:38.668576 29669 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:38.600451 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:38.615489 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:38.310235 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.655268 28696 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.319693 29725 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.319011 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1793) took 2115 ms. Trace:
I20260709 13:09:38.302129 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:38.659576 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:38.666262 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.666108 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:38.531605 29714 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.134s	user 0.005s	sys 0.008s
I20260709 13:09:38.609496 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.666462 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.674723 22755 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 27.
W20260709 13:09:38.665197 22823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.173s	user 0.029s	sys 0.037s
W20260709 13:09:38.655282 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 437525us
I20260709 13:09:38.664707 23447 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:38.596874 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.309998 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:38.670580 22868 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.664983 22752 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:38.327459 29645 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.595679 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.676826 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:38.681586 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:38.682541 23445 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 18: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:38.669378 29656 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:38.683977 29656 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.686594 29656 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:38.687121 29656 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:38.687760 29656 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.689492 28992 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:38.689850 29507 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.690140 28992 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.690505 29507 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.691992 22754 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:38.676666 22753 raft_consensus.cc:2410] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
W20260709 13:09:38.400753 29061 consensus_meta.cc:287] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.685s	user 0.011s	sys 0.009s
I20260709 13:09:38.671708 29646 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:38.699542 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.697966 22748 raft_consensus.cc:3060] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:38.698889 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:38.660179 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:38.695698 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:38.696938 29384 consensus_meta.cc:287] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.003s
I20260709 13:09:38.704728 22824 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:38.694435 29507 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.706506 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:38.703310 22747 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:38.709286 22828 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:38.711218 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.711692 22752 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:38.671280 23453 rpcz_store.cc:276] 0709 13:09:36.203303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.205402 (+  2099us) service_pool.cc:224] Handling call
0709 13:09:36.205669 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:36.225396 (+ 19727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:36.225433 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:36.225913 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 13:09:36.322094 (+ 96181us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:36.322133 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:36.322193 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.080905 (+1758712us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:38.318461 (+237556us) raft_consensus.cc:1685] finished
0709 13:09:38.318482 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.318841 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14465,"raft.queue_time_us":100,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":12836,"spinlock_wait_cycles":14846336,"thread_start_us":94795,"threads_started":1,"wal-append.queue_time_us":189334}
I20260709 13:09:38.698997 29646 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:38.715931 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.716902 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:38.714728 22825 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:38.717698 28992 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:38.721609 22822 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:38.721864 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 2302) took 1851 ms. Trace:
I20260709 13:09:38.717586 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:38.730769 22896 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:38.729600 22747 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:38.707952 29384 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:38.737512 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:38.737712 22901 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:38.738142 22902 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:38.738263 22900 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 18. (term mismatch)
I20260709 13:09:38.738705 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:38.739636 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 296386us
I20260709 13:09:38.740091 22900 pending_rounds.cc:85] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 17
I20260709 13:09:38.732833 29656 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:38.749792 22900 pending_rounds.cc:107] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 13:09:38.750145 22900 raft_consensus.cc:2889] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:38.752034 23449 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:09:38.752209 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:38.753783 28992 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.754279 28992 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.755112 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:38.755761 23451 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:38.731240 22823 rpcz_store.cc:276] 0709 13:09:36.870565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.873930 (+  3365us) service_pool.cc:224] Handling call
0709 13:09:36.874166 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:36.883203 (+  9037us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:09:37.204487 (+321284us) log.cc:844] Serialized 14 byte log entry
0709 13:09:38.704127 (+1499640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:38.710541 (+  6414us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:38.710592 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.717041 (+  6449us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.721810 (+  4769us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48503,"prepare.queue_time_us":427795,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":1308,"raft.queue_time_us":109119,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":60962,"thread_start_us":1739343,"threads_started":3,"wal-append.queue_time_us":247248,"wal-append.run_cpu_time_us":1174,"wal-append.run_wall_time_us":1009250}
I20260709 13:09:38.763749 23445 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:09:38.765900 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.765955 23444 raft_consensus.cc:1275] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:38.775444 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.769526 29669 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.781440 29708 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:38.781869 22899 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:38.782132 29708 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:38.782608 29561 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.783442 29656 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:38.783805 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 2295) took 1989 ms. Trace:
I20260709 13:09:38.783342 29561 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.784166 29656 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.784204 22827 rpcz_store.cc:276] 0709 13:09:36.794782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.795131 (+   349us) service_pool.cc:224] Handling call
0709 13:09:36.795431 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:36.795731 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:36.795758 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:36.796115 (+   357us) log.cc:844] Serialized 26 byte log entry
0709 13:09:36.997039 (+200924us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:36.997078 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:36.997142 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.530423 (+1533281us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:38.778326 (+247903us) raft_consensus.cc:1685] finished
0709 13:09:38.778348 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.783658 (+  5310us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":54,"spinlock_wait_cycles":9519360,"thread_start_us":200660,"threads_started":1,"wal-append.queue_time_us":539421}
I20260709 13:09:38.785910 29561 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:38.787566 29384 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:38.788328 29725 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.780318 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:38.789665 22897 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:38.793354 29758 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.153s	user 0.000s	sys 0.002s
W20260709 13:09:38.793448 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:38.794229 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.775132 29061 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.799095 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:38.801606 23447 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:38.802829 22748 raft_consensus.cc:2468] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:38.803367 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 3074) took 1386 ms. Trace:
I20260709 13:09:38.804332 22748 rpcz_store.cc:276] 0709 13:09:37.416837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.596392 (+1179555us) service_pool.cc:224] Handling call
0709 13:09:38.803305 (+206913us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:38.821867 22899 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:38.795763 29669 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.828725 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:38.803840 29725 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:38.827693 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:38.826375 28696 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:38.831084 22748 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:38.831066 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:38.783250 29708 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:38.826054 29656 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:38.844062 28696 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.832893 28992 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:38.848347 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:38.849997 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:38.843320 29646 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:38.853500 29486 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.857102 28696 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.862371 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:38.849160 29061 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:38.869609 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:38.872032 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:38.874670 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:38.879725 29756 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.225s	user 0.001s	sys 0.001s
I20260709 13:09:38.880234 22904 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:38.855021 29725 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:38.882710 29725 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:38.883415 29645 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:38.884495 22897 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:38.886467 22750 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:09:38.887689 22750 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:38.888197 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:38.888855 22822 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:38.890093 29748 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.257s	user 0.003s	sys 0.000s
W20260709 13:09:38.890450 29752 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:09:38.892835 29269 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.893479 29269 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:38.894241 29669 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:38.894327 29696 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.894909 29384 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.894973 29696 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.895663 29269 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:38.895610 29384 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.896736 22822 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:38.897694 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:38.898521 22750 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:38.882725 22797 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:38.866847 29706 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:38.900655 23447 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:38.870307 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:38.902076 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1398) took 1968 ms. Trace:
I20260709 13:09:38.905997 22754 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
W20260709 13:09:38.906656 22754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38488 (request call id 3073) took 1490 ms (client timeout 1547 ms). Trace:
I20260709 13:09:38.906667 22748 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:09:38.908784 22905 raft_consensus.cc:2410] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:09:38.909503 22749 pending_rounds.cc:85] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 8
I20260709 13:09:38.909825 22749 pending_rounds.cc:107] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:09:38.910069 22749 raft_consensus.cc:2889] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:38.910866 22748 raft_consensus.cc:1240] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:38.865453 29486 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:38.912528 22748 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 24 index: 21. Preceding OpId from leader: term: 25 index: 22. (index mismatch)
I20260709 13:09:38.901894 22750 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:56780
I20260709 13:09:38.913569 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:38.914376 22750 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:38.915442 22750 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:38.847843 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:38.919029 22752 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:39.043541 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:38.907264 22754 rpcz_store.cc:269] 0709 13:09:37.416599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.595130 (+1178531us) service_pool.cc:224] Handling call
0709 13:09:38.906584 (+311454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:39.044875 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.046041 23447 raft_consensus.cc:2410] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:09:38.964996 22750 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.049342 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.051110 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:38.964906 23446 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:39.054237 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:39.051666 22905 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:39.054968 22828 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:39.056398 22823 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:39.052906 22903 raft_consensus.cc:606] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:39.057920 22903 raft_consensus.cc:614] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:39.057168 22747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 2092) took 1144 ms. Trace:
I20260709 13:09:39.049041 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:39.059840 29507 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:39.060588 29507 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.061348 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.062429 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.062722 29507 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:39.063026 22831 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:39.063691 22826 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:39.063839 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.064818 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.064839 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.065642 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.065855 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.066327 22826 raft_consensus.cc:2393] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:39.066471 29696 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:39.066771 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.066865 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.067027 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.067417 22824 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:39.068002 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.068010 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.068676 29251 raft_consensus.cc:2804] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:39.069254 29251 raft_consensus.cc:697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:39.070370 29251 consensus_queue.cc:237] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:39.072800 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:39.073585 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.068547 22750 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:38.902428 29706 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:38.921041 29061 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:39.108341 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.068552 22824 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:39.110004 22754 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 20. Ops: []
I20260709 13:09:38.903899 22829 rpcz_store.cc:276] 0709 13:09:36.933569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.126254 (+192685us) service_pool.cc:224] Handling call
0709 13:09:37.126520 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:37.175241 (+ 48721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:37.175285 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:37.186118 (+ 10833us) log.cc:844] Serialized 26 byte log entry
0709 13:09:38.054191 (+868073us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b18002150e0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b0d1 00007fcb74db6a0f
0709 13:09:38.619866 (+565675us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:38.619912 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.655930 (+ 36018us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.889358 (+233428us) spinlock_profiling.cc:243] Waited 59.5 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:38.899242 (+  9884us) raft_consensus.cc:1685] finished
0709 13:09:38.899268 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.901891 (+  2623us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48342,"raft.queue_time_us":56151,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"spinlock_wait_cycles":157888640,"thread_start_us":381921,"threads_started":1,"wal-append.queue_time_us":1445847}
W20260709 13:09:39.115674 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 56846.7us
I20260709 13:09:39.074723 29083 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.110831 29706 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:39.058780 22747 rpcz_store.cc:276] 0709 13:09:37.912294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.853962 (+941668us) service_pool.cc:224] Handling call
0709 13:09:38.863186 (+  9224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:38.867953 (+  4767us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:09:38.914902 (+ 46949us) log.cc:844] Serialized 14 byte log entry
0709 13:09:39.054731 (+139829us) log.cc:844] Serialized 14 byte log entry
0709 13:09:39.056816 (+  2085us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:39.056892 (+    76us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:09:39.056953 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.057012 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.057117 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87343,"prepare.queue_time_us":40999,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":89543,"thread_start_us":53577,"threads_started":1}
I20260709 13:09:39.165472 29639 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.126053 29384 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:39.198993 29639 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.162348 29486 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:39.164266 29646 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:39.202651 29646 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.204670 29646 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:39.154152 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:39.219395 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.115634 29669 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.243571 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3078) took 1818 ms. Trace:
W20260709 13:09:39.256811 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46008.8us
I20260709 13:09:39.255502 22751 rpcz_store.cc:276] 0709 13:09:37.425621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.443681 (+ 18060us) service_pool.cc:224] Handling call
0709 13:09:38.159950 (+716269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:38.160940 (+   990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.240507 (+1079567us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b5400045450. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfa8ad 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:39.243485 (+  2978us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576609024}
I20260709 13:09:39.274159 29751 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:39.274914 29751 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:39.275345 29751 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:39.281733 22749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3079) took 1329 ms. Trace:
I20260709 13:09:39.282377 22749 rpcz_store.cc:276] 0709 13:09:37.952075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.867076 (+915001us) service_pool.cc:224] Handling call
0709 13:09:38.871870 (+  4794us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:38.910842 (+ 38972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:38.910875 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:38.911454 (+   579us) log.cc:844] Serialized 52 byte log entry
0709 13:09:38.965723 (+ 54269us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:38.965757 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.965816 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:39.281356 (+315540us) raft_consensus.cc:1685] finished
0709 13:09:39.281385 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.281605 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1257,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"thread_start_us":53733,"threads_started":1,"wal-append.queue_time_us":258486}
I20260709 13:09:39.287338 29643 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:39.292366 29755 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.655s	user 0.006s	sys 0.003s
W20260709 13:09:39.299167 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:39.303057 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 61515.2us
I20260709 13:09:39.346258 23446 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:09:39.247543 29639 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:39.161175 29725 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.094s
I20260709 13:09:39.350085 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.352825 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:56780 (request call id 2097) took 1230 ms. Trace:
I20260709 13:09:39.353333 22750 rpcz_store.cc:276] 0709 13:09:38.116655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.899187 (+782532us) service_pool.cc:224] Handling call
0709 13:09:39.346890 (+447703us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280}
I20260709 13:09:39.352644 22903 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:39.363396 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.364234 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.127151 29061 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.229s
W20260709 13:09:39.376067 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 221986us
I20260709 13:09:39.245254 29083 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.379401 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:39.379827 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.385249 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.388753 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:39.389784 29656 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:39.390540 29656 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:39.390765 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.391875 29656 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.385360 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:39.393299 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.395668 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.396585 22905 raft_consensus.cc:2410] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:09:39.397153 29546 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:39.397650 29546 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:39.398036 29546 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:39.398295 29061 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:39.403728 29725 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:39.404692 29725 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:39.405966 29725 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:39.407199 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.407545 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:39.399781 29083 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:39.349208 29669 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.408476 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:39.688738 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 279797us
I20260709 13:09:39.688993 22900 raft_consensus.cc:2376] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18. Re-sending same reply.
I20260709 13:09:39.415391 22828 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
W20260709 13:09:39.588894 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:39.692174 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.437628 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.401278 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:39.414566 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:39.694450 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.696791 22899 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:39.409297 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:39.475764 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.596477 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:39.704048 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.708976 29618 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:09:39.709695 29061 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:39.709110 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:39.597237 29765 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 13:09:39.474896 29753 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.842s	user 0.009s	sys 0.008s
I20260709 13:09:39.469401 29719 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.713232 22831 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:39.712725 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.716004 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:39.714989 22896 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:39.716337 29669 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:39.716589 29772 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:09:39.716022 29662 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:39.722815 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:39.724357 29645 consensus_meta.cc:287] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.840s	user 0.009s	sys 0.004s
W20260709 13:09:39.724712 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 344037us
I20260709 13:09:39.737603 22902 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:39.727895 29546 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:39.716120 29643 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:39.738250 29643 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:39.733065 22905 raft_consensus.cc:2393] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 14: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:09:39.732334 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 273243us
I20260709 13:09:39.741555 22756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3081) took 1787 ms. Trace:
I20260709 13:09:39.602072 29721 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.132s
I20260709 13:09:39.739221 29643 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:39.746848 29643 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:39.747038 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:39.747905 22749 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 17: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:39.742223 22756 rpcz_store.cc:276] 0709 13:09:37.952736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.886355 (+933619us) service_pool.cc:224] Handling call
0709 13:09:38.886697 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:38.887102 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:38.970758 (+ 83656us) log.cc:844] Serialized 14 byte log entry
0709 13:09:39.369784 (+399026us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:39.433652 (+ 63868us) log.cc:844] Serialized 26 byte log entry
0709 13:09:39.433798 (+   146us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:39.433845 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.433934 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:39.739514 (+305580us) raft_consensus.cc:1685] finished
0709 13:09:39.739534 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.739656 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27646,"prepare.queue_time_us":28279,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":22,"raft.queue_time_us":367,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":4479,"thread_start_us":349259,"threads_started":2,"wal-append.queue_time_us":489510}
I20260709 13:09:39.752959 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3086) took 1052 ms. Trace:
I20260709 13:09:39.753944 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:39.755540 22750 raft_consensus.cc:2410] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 25 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:09:39.753628 22902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1658) took 1005 ms. Trace:
I20260709 13:09:39.754390 22748 rpcz_store.cc:276] 0709 13:09:38.700125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.924003 (+223878us) service_pool.cc:224] Handling call
0709 13:09:39.049670 (+125667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:39.106578 (+ 56908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:39.106608 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:39.119328 (+ 12720us) log.cc:844] Serialized 26 byte log entry
0709 13:09:39.449931 (+330603us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:39.449991 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.450083 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:39.752642 (+302559us) raft_consensus.cc:1685] finished
0709 13:09:39.752672 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.752902 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2778,"raft.queue_time_us":4574,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"thread_start_us":330178,"threads_started":1,"wal-append.queue_time_us":628815}
I20260709 13:09:39.756419 22902 rpcz_store.cc:276] 0709 13:09:38.748533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.751210 (+  2677us) service_pool.cc:224] Handling call
0709 13:09:38.775914 (+ 24704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:39.737961 (+962047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.751177 (+ 13216us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b5400206f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:39.753567 (+  2390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2210793344}
I20260709 13:09:39.758031 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:39.759106 29754 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 1.125s	user 0.004s	sys 0.000s
I20260709 13:09:39.760228 29546 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:39.765705 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1729) took 1493 ms. Trace:
I20260709 13:09:39.766122 23452 rpcz_store.cc:276] 0709 13:09:38.272571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.298592 (+ 26021us) service_pool.cc:224] Handling call
0709 13:09:38.299123 (+   531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:38.299651 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:38.299682 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:38.300219 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:09:38.481151 (+180932us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:38.481201 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.481283 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:39.756269 (+1274986us) spinlock_profiling.cc:243] Waited 55.3 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:39.764890 (+  8621us) raft_consensus.cc:1685] finished
0709 13:09:39.764912 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.765530 (+   618us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2803,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"spinlock_wait_cycles":127194624,"thread_start_us":180379,"threads_started":1,"wal-append.queue_time_us":333414}
I20260709 13:09:39.768572 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:39.768949 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.769734 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:39.770344 22750 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:39.764565 29751 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.775148 29648 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:39.775378 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:39.783689 22831 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:39.783505 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:39.789726 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:39.789906 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:39.790100 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:39.790547 29399 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:39.790999 29399 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:39.791385 29399 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:39.793706 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:39.794153 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:39.807607 23452 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:39.808151 23446 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:39.808202 23451 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:39.808236 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:39.810787 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.812479 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:39.813974 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 404718us
I20260709 13:09:39.780910 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:39.818522 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.820225 22750 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:39.819320 29083 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:39.821535 22756 raft_consensus.cc:2410] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:09:39.821940 29083 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.776171 29751 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:39.855309 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:39.859629 22831 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:09:39.752735 29669 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:09:39.848654 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 67847.2us
I20260709 13:09:39.825459 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:39.825785 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.858290 22748 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:09:39.813812 29561 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:39.824234 29706 consensus_meta.cc:287] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.000s
I20260709 13:09:39.866600 22830 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3206) took 1661 ms. Trace:
W20260709 13:09:39.862581 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 173069us
I20260709 13:09:39.867259 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:39.867885 22825 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:09:39.829746 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.871858 22823 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:39.745338 29719 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:09:39.884135 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:39.913206 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.920997 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.898424 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:39.928735 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.872881 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:39.931386 22831 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:39.920315 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:39.931428 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:39.813796 29616 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 13:09:39.934980 29061 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:39.937523 22755 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:39.927603 22830 rpcz_store.cc:276] 0709 13:09:38.204855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.206701 (+  1846us) service_pool.cc:224] Handling call
0709 13:09:38.207057 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:38.207612 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:38.207650 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:38.208166 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:09:38.444737 (+236571us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:38.444794 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.444895 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:39.363837 (+918942us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:39.859065 (+495228us) raft_consensus.cc:1685] finished
0709 13:09:39.859085 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.866540 (+  7455us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1791,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":60469,"spinlock_wait_cycles":92569856,"thread_start_us":236055,"threads_started":1,"wal-append.queue_time_us":424638}
I20260709 13:09:39.867406 29643 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:39.938930 22829 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:39.940255 22828 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:39.925845 29751 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.872956 29546 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:39.957185 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:39.940678 29669 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:39.946728 29561 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:39.920089 29519 consensus_queue.cc:1048] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:09:39.941880 29783 consensus_queue.cc:1059] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:09:39.932334 29706 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.946524 29643 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:39.983224 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 158000us
I20260709 13:09:39.959744 29662 raft_consensus.cc:2764] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:09:39.871301 29708 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 13:09:39.961808 29669 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:39.980389 29561 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:39.955421 29751 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:39.993901 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:39.993542 22827 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:39.998054 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 87742.6us
I20260709 13:09:39.987893 29643 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:39.988550 29083 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:40.004434 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:40.010638 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.997532 29669 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:40.015503 22825 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:40.016021 23452 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:40.016814 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.019169 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:39.990854 29546 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:40.023334 23448 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 23: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:40.012507 29662 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:40.037494 29662 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:40.037724 29725 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:40.037963 29662 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:40.035081 29558 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:09:40.017195 29782 raft_consensus.cc:3055] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:40.046306 29782 raft_consensus.cc:740] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:40.047302 29782 consensus_queue.cc:260] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.048861 29782 raft_consensus.cc:3060] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:40.049904 29719 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:40.050761 29546 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:40.052582 29751 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:40.054979 22823 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:40.059028 29706 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:40.072400 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.082526 22825 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:40.092010 23451 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
W20260709 13:09:40.093739 29787 log.cc:927] Time spent T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.052s	user 0.000s	sys 0.002s
I20260709 13:09:40.125929 29669 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:40.126832 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:40.093972 29788 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.052s	user 0.003s	sys 0.000s
I20260709 13:09:40.094810 29646 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:40.098685 29784 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.090s	user 0.002s	sys 0.000s
W20260709 13:09:40.111460 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 104067us
W20260709 13:09:40.094152 29780 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.275s	user 0.000s	sys 0.003s
I20260709 13:09:40.103873 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:40.096921 29785 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.230s	user 0.003s	sys 0.000s
W20260709 13:09:40.149770 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 214304us
I20260709 13:09:40.150404 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:40.120241 29561 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.123005 29662 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.096376 29399 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.128034 29669 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:40.163973 22871 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:40.164763 22826 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:09:40.097754 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 242481us
I20260709 13:09:40.131182 29061 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:09:40.131651 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 110145us
W20260709 13:09:40.140053 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 137931us
I20260709 13:09:40.140853 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.111927 29643 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.148190 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.221072 23451 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
W20260709 13:09:40.159458 29779 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.246s	user 0.000s	sys 0.002s
I20260709 13:09:40.222040 23452 raft_consensus.cc:2393] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:40.222321 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.161095 29662 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:40.222043 23419 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.225746 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:40.161542 29719 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:09:40.162029 29786 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.143s	user 0.000s	sys 0.003s
W20260709 13:09:40.162928 29789 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.002s
I20260709 13:09:40.163347 29399 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:40.165232 22871 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:40.206403 29751 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:40.208308 23446 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 23: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:40.201460 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1400) took 1314 ms. Trace:
W20260709 13:09:40.202160 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 98360.4us
I20260709 13:09:40.203917 29561 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:40.153291 29646 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:40.156651 29725 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:40.228272 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.237984 22827 rpcz_store.cc:276] 0709 13:09:38.887490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.898585 (+ 11095us) service_pool.cc:224] Handling call
0709 13:09:38.898877 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:40.000739 (+1101862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:40.201142 (+200403us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001f9fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:40.201380 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2517021440}
I20260709 13:09:40.239782 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:40.240477 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.251510 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 24 index: 21 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.252638 23444 raft_consensus.cc:2410] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 26 because replica has last-logged OpId of term: 25 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 21.
I20260709 13:09:40.242252 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.242704 29646 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:40.244345 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 79177.2us
I20260709 13:09:40.245186 22829 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:40.246778 29643 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:40.227375 29546 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:40.228973 29645 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:09:40.227375 29719 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:09:40.251085 23416 leader_election.cc:336] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.541s (SENT)
I20260709 13:09:40.250011 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:40.241395 22826 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:09:40.258677 23416 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:40.255776 23447 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:40.257853 22750 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:40.261260 29061 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:40.250581 29795 raft_consensus.cc:2804] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:40.261183 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.262463 29719 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:40.259300 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:40.270612 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.274575 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.275774 22826 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:40.274189 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.274014 29266 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:40.278272 29266 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:40.278360 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.278690 29266 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:40.279536 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.280438 29662 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:40.281904 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.282194 29797 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:40.282696 29618 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:40.283564 29795 raft_consensus.cc:697] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:40.284667 29795 consensus_queue.cc:237] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.286605 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:40.295805 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44722.1us
W20260709 13:09:40.297139 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 167343us
I20260709 13:09:40.300076 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:40.316210 29061 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:40.316699 29061 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:40.317099 29061 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:40.321269 22826 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:40.324769 29669 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:40.325618 29669 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:40.326015 29669 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:40.326615 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.326864 29669 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.327842 29669 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:09:40.329244 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.330821 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:40.331700 29798 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:40.332103 29798 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:40.332067 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.332629 29798 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:40.313463 29643 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:40.333839 29643 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:40.335191 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.336323 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.341521 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.342319 23452 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:40.344013 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.346474 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.346539 29796 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:40.347146 29719 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:40.347417 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.348277 29083 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:40.348641 29083 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:40.349005 29083 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:40.323571 29782 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:09:40.349689 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 96226.5us
I20260709 13:09:40.351168 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.355376 23446 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:40.351552 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:40.351759 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.357045 23453 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:40.357285 29618 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:40.351943 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.349967 29782 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:40.359095 29782 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:40.352466 29266 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:40.363373 22871 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:40.352576 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.365671 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.366891 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:40.367426 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:40.341565 29662 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:40.363516 29266 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:40.374857 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.374995 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.370795 29083 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.378494 22829 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:40.379562 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:40.380102 23446 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 21
W20260709 13:09:40.381052 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.382428 22749 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:40.395316 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:40.374404 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.396948 22826 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:40.350314 29656 raft_consensus.cc:2764] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:09:40.419549 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.349282 29719 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:40.425230 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.373136 29639 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:40.428110 23448 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:40.428181 29083 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:40.428737 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:40.425455 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.334741 29643 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:40.466746 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.393401 29725 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:40.471175 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:40.474022 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.383565 29800 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:40.417387 29546 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:40.502758 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 135917us
I20260709 13:09:40.504088 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:40.465008 29061 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:40.373025 29646 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 13:09:40.435111 29639 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:40.461103 29751 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:40.514190 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:40.467775 29648 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:40.469360 29486 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:40.512214 29061 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:40.516501 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:40.519008 23446 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
W20260709 13:09:40.511960 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 137020us
I20260709 13:09:40.522588 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:40.523494 22826 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:40.526839 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:40.527863 22897 raft_consensus.cc:2393] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 16: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:40.528548 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:40.533213 22905 raft_consensus.cc:1275] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:09:40.533474 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:40.533872 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:40.533675 22905 pending_rounds.cc:85] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 10
I20260709 13:09:40.534236 22901 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:40.535759 29800 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:09:40.536144 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.539407 22904 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:40.539568 22905 pending_rounds.cc:107] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:09:40.539929 22905 raft_consensus.cc:2889] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:40.519747 29639 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:40.520682 29648 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:40.520247 29751 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:40.508867 29799 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.287s
I20260709 13:09:40.555243 29798 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.549583 29751 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:40.532099 29646 consensus_queue.cc:1243] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:09:40.558816 22829 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:40.559546 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.564801 29817 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:40.566706 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:40.560972 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 98827us
I20260709 13:09:40.514954 29546 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.568548 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.570784 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.571619 22825 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:40.549768 29648 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:40.574419 29824 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:40.574476 29266 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:40.577939 29811 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:09:40.583935 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:40.578886 29807 log.cc:927] Time spent T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:09:40.574435 22827 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:09:40.549685 29618 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:40.578187 22702 placement_policy_util.cc:248] tablet 135e2f7520a34b53aaf4ddb360a8a029: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 13:09:40.552515 22654 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 14 to 15, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 15 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:40.589679 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.584833 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:40.603467 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.604158 22825 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:40.604981 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:40.581987 29646 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:40.612797 23447 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:40.597501 29266 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.617076 22828 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:40.617663 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:40.619630 22797 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:40.572913 29817 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.565518 29725 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.625979 29399 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:40.627895 29266 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:40.629958 29618 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:40.632565 22899 raft_consensus.cc:2393] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 19: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:40.633930 29546 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:40.656054 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:40.658515 22827 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:40.634339 22652 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33010 (request call id 227) took 34294 ms (client timeout 15000 ms). Trace:
I20260709 13:09:40.662793 29817 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:40.659778 22652 rpcz_store.cc:269] 0709 13:09:06.340477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:06.340777 (+   300us) service_pool.cc:224] Handling call
0709 13:09:40.634107 (+34293330us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:36.231451 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:36.232026 (+   575us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:36.232067 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:09:36.232085 (+    18us) tablet.cc:662] Decoding operations
0709 13:09:36.242502 (+ 10417us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:36.248077 (+  5575us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:36.250009 (+  1932us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:09:36.264476 (+ 14467us) tablet.cc:701] Row locks acquired
0709 13:09:36.264497 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:09:36.264651 (+   154us) write_op.cc:270] Start()
0709 13:09:36.264772 (+   121us) write_op.cc:276] Timestamp: P: 1783602576264608 usec, L: 0
0709 13:09:36.264794 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:09:36.265451 (+   657us) log.cc:844] Serialized 913 byte log entry
0709 13:09:36.513593 (+248142us) op_driver.cc:464] REPLICATION: finished
0709 13:09:36.685181 (+171588us) write_op.cc:301] APPLY: starting
0709 13:09:36.685291 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:36.687337 (+  2046us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:36.687468 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:36.689210 (+  1742us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:36.689476 (+   266us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:09:36.689557 (+    81us) write_op.cc:312] APPLY: finished
0709 13:09:36.689882 (+   325us) log.cc:844] Serialized 33 byte log entry
0709 13:09:36.700086 (+ 10204us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:36.701904 (+  1818us) write_op.cc:454] Released schema lock
0709 13:09:36.728137 (+ 26233us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:36.766341 (+ 38204us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:36.768722 (+  2381us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1521526,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":167086,"apply.run_cpu_time_us":5518,"apply.run_wall_time_us":65752,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12487,"num_ops":2,"prepare.queue_time_us":110318,"prepare.run_cpu_time_us":55840,"prepare.run_wall_time_us":117019,"raft.queue_time_us":139088,"raft.run_cpu_time_us":21206,"raft.run_wall_time_us":201859,"replication_time_us":224749,"thread_start_us":343538,"threads_started":4,"wal-append.queue_time_us":83255,"wal-append.run_cpu_time_us":6183,"wal-append.run_wall_time_us":1344435}]]}
I20260709 13:09:40.619062 29646 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:40.667192 29646 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.668728 29646 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:40.638012 29639 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.598099 29824 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:40.639159 29699 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:40.624460 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:40.642693 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44295.3us
I20260709 13:09:40.646175 22826 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:40.673712 29615 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:40.674562 29561 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:09:40.681557 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 366589us
I20260709 13:09:40.681649 22654 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 14 to 18, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 18 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:40.683682 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.639907 29782 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:40.690384 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 66019.8us
I20260709 13:09:40.708261 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:40.729210 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.739392 29061 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:40.741531 29061 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:40.743963 23446 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 22. Current term is 23. Ops: [22.19-22.19]
I20260709 13:09:40.740067 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.746250 23447 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:40.744632 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.747840 23452 raft_consensus.cc:2410] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 19 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 15.
W20260709 13:09:40.739307 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 189506us
W20260709 13:09:40.753237 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:40.738734 29788 log.cc:927] Time spent T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:09:40.755693 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:40.704183 29266 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:40.767072 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.756487 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.775530 29061 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.777772 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:40.779255 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 138498us
I20260709 13:09:40.780171 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:40.786623 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.703868 29790 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:09:40.787453 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.755089 29792 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:40.803715 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:40.804629 23447 raft_consensus.cc:2393] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:40.797995 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:40.814944 29832 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:40.815433 29832 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:40.815786 29832 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:40.836822 29824 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.838896 29782 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:40.741140 29645 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:09:40.842101 23451 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
W20260709 13:09:40.847779 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:40.853236 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 49857.6us
W20260709 13:09:40.867496 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:40.877350 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 175580us
I20260709 13:09:40.881089 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:40.854513 29792 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:40.919986 29795 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:40.920423 29795 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:40.895155 29800 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:09:40.940181 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.877723 29384 consensus_queue.cc:1048] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:09:40.942304 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.945555 22654 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 9 to 12, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 12 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:40.948827 22827 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:40.950912 23450 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:40.967962 29615 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:40.968245 29616 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:09:40.969455 29615 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:40.969525 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.969929 29615 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:40.970544 29699 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:40.970714 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.971542 29699 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:40.970738 29615 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:40.971905 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.972553 29615 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:40.972678 29699 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:40.975558 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:40.976336 29616 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:40.933485 29800 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:40.980010 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 469498us
W20260709 13:09:40.980827 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 88340.5us
I20260709 13:09:40.982595 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:40.946264 29639 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:40.991600 23452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:41.000995 23452 raft_consensus.cc:606] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:41.001344 23452 raft_consensus.cc:614] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:09:41.003247 23416 leader_election.cc:336] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.606s (SENT)
I20260709 13:09:41.003680 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:40.945750 29817 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:40.943264 29830 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:09:40.986430 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.013031 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.009800 29618 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:41.015017 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.017761 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 166260us
I20260709 13:09:41.025068 29648 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.027489 29648 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:41.021222 29792 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.058357 29083 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:40.921242 29795 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:41.063982 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 80610.7us
I20260709 13:09:41.066625 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.012321 29817 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.067453 29817 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:41.011662 29798 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:41.016731 29830 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:41.075124 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 61833.7us
I20260709 13:09:41.083074 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:41.084026 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 80803.9us
I20260709 13:09:41.047981 29639 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
W20260709 13:09:41.129762 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.065681 29699 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:41.132215 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51908.2us
I20260709 13:09:41.132375 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:41.074062 29830 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:41.133546 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.048285 29832 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.168385 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:41.168609 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:41.172040 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:41.173627 29860 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:41.174065 29860 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.174392 29860 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:41.175639 29858 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:41.177829 22750 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:41.171998 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:41.089629 29835 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:09:41.183868 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 117373us
I20260709 13:09:41.033061 29800 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.131628 29795 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:41.167675 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.191793 22756 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:41.131346 29656 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.197816 29849 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
W20260709 13:09:41.199698 29844 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:09:41.202409 29798 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.202865 29798 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:41.204576 29845 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:09:41.190172 29834 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:41.208469 29832 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:41.190375 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:41.209843 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44180.6us
I20260709 13:09:41.210004 29648 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:41.210496 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.211555 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:41.214097 22871 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:41.223666 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.224257 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:41.225502 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57910.7us
I20260709 13:09:41.230824 29061 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:41.231765 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:41.235939 29860 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:41.236855 29834 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:41.237371 29832 raft_consensus.cc:2764] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:09:41.187611 29835 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:41.239480 29083 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:41.210505 29648 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:41.239856 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:41.225805 29669 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:41.180608 29699 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.254686 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:41.268044 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 75976.3us
I20260709 13:09:41.268615 22827 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:09:41.255300 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.257643 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.238615 29800 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:41.251844 29837 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 13:09:41.264159 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53740.5us
I20260709 13:09:41.276402 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.276674 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:41.277001 29856 log.cc:927] Time spent T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:09:41.277287 29825 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.277838 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:41.277812 29825 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:41.279026 29854 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:09:41.279111 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.277446 22823 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:41.280148 29834 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:41.283056 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.285605 29848 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:41.222549 29656 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.253219 29648 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:41.287844 29656 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:41.289077 29699 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:41.294068 29646 consensus_meta.cc:287] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.625s	user 0.007s	sys 0.004s
I20260709 13:09:41.285867 29860 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:41.278306 22828 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21. Ops: [18.13-20.15]
I20260709 13:09:41.280002 29825 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:41.249012 29792 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:41.246639 29835 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:41.304335 22753 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:09:41.301466 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.365878 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:41.366561 22748 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:41.365916 22753 pending_rounds.cc:85] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 13
I20260709 13:09:41.367281 22753 pending_rounds.cc:107] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:09:41.367550 22753 raft_consensus.cc:2889] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:41.302619 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:41.269136 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.340482 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.368993 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.276906 29819 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:41.374889 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 72354.6us
W20260709 13:09:41.375311 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73992.6us
I20260709 13:09:41.381551 22797 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:41.294899 29829 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:41.238129 29828 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:09:41.250500 29810 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.118s
W20260709 13:09:41.386480 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.408069 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.492714 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 1879799f5d0c462c9d2c95bd8a88862e: Illegal state: Replica 3b626c41600d4737a178ebab8a7ebfc2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
W20260709 13:09:41.494645 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.523758 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 244690us
I20260709 13:09:41.379503 29646 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:41.526412 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:41.527225 22827 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:41.387006 29643 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:41.384568 29384 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:41.379808 29819 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:41.385442 29719 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:41.271771 29836 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.267s
W20260709 13:09:41.528448 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.388392 29825 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 13:09:41.557557 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.587322 22749 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 14. Current term is 15. Ops: []
W20260709 13:09:41.589884 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60564.9us
W20260709 13:09:41.590863 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 322158us
I20260709 13:09:41.310992 29832 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.202s
W20260709 13:09:41.594563 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 224177us
I20260709 13:09:41.380898 29817 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.389442 29618 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.192s
I20260709 13:09:41.609884 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.610193 23449 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:09:41.579304 29725 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.030s	sys 0.042s
I20260709 13:09:41.614089 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.619609 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:41.582502 29819 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:41.627012 22748 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:41.636844 29618 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:41.637274 29618 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:41.628557 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.626029 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:41.641395 22749 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
W20260709 13:09:41.618750 29725 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.030s	sys 0.042s
I20260709 13:09:41.602525 29798 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:41.615897 29646 raft_consensus.cc:3055] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:41.605017 29825 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:41.663338 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.638172 29832 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:09:41.687839 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 73825.6us
I20260709 13:09:41.628819 29848 consensus_queue.cc:1059] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:09:41.702634 29616 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:41.703312 29616 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:41.704872 29639 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:41.706584 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:41.710732 29615 consensus_meta.cc:287] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.738s	user 0.006s	sys 0.005s
I20260709 13:09:41.682631 29832 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:41.711498 29648 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:41.711992 29834 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:41.661638 29646 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:41.716079 22797 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:41.669215 29836 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:09:41.731209 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 111668us
I20260709 13:09:41.687445 29825 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:41.737972 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:41.688982 29871 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:41.740607 22797 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:41.714560 29874 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:09:41.717195 29639 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:41.722200 29751 consensus_meta.cc:287] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 1.148s	user 0.011s	sys 0.000s
W20260709 13:09:41.742221 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 6ce771dbb93248d397c8d5b222b27fca: Illegal state: Replica 3b626c41600d4737a178ebab8a7ebfc2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:41.730664 29821 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:09:41.738926 29830 consensus_meta.cc:287] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.000s
I20260709 13:09:41.727339 23416 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:41.736090 29615 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:41.754055 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:41.755353 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:41.756197 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:41.757355 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:41.774706 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:41.726809 29859 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:09:41.777541 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:41.775875 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:41.734015 29790 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:09:41.749553 29857 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:41.746049 29639 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:41.704877 29616 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.788048 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:41.751945 29821 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:41.759397 29561 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:41.792734 29546 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.775471 29615 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:41.733588 29618 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:41.792441 29867 log.cc:927] Time spent T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:09:41.789992 29857 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.748631 29751 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.760864 29830 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.793251 29871 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:41.797807 29821 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:41.761401 29829 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:09:41.814828 29828 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:41.816402 29616 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:41.846434 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.013s	sys 0.005s
W20260709 13:09:41.847882 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.013s	sys 0.005s
W20260709 13:09:41.848170 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 682018us
I20260709 13:09:41.849655 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:41.850369 23452 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:41.852717 22898 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:41.853188 22755 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 20. Current term is 21. Ops: []
W20260709 13:09:41.844620 29866 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:09:41.787617 29646 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:41.843286 29857 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:41.852921 22905 raft_consensus.cc:1240] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:09:41.875033 22903 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:41.816654 29725 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.035s
I20260709 13:09:41.845059 29817 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:41.844146 29878 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.870307 22904 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:41.871749 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:41.881609 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.881882 22826 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:41.813264 29546 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.785145 29832 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:41.878345 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:41.879524 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.886201 23444 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:41.880312 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:41.879619 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:41.880048 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:41.883605 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:41.851401 29871 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:09:41.906306 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.884079 29725 consensus_queue.cc:1243] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 24.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 13:09:41.909981 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:41.905164 22719 leader_election.cc:336] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.556s (SENT)
I20260709 13:09:41.910676 22826 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:41.910701 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:09:41.913533 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.916843 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 155613us
I20260709 13:09:41.916954 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:41.895411 29833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.014s	sys 0.013s
I20260709 13:09:41.917789 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:41.906231 29251 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.934182 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:41.912421 29796 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:41.919330 29873 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:09:41.936753 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:41.920497 29833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.014s	sys 0.013s
I20260709 13:09:41.916594 29832 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:41.939767 23453 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:41.919500 29648 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:41.944200 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:41.948642 22901 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:41.949021 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:41.888574 29878 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:41.923534 29828 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:09:41.970846 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 72408.9us
I20260709 13:09:41.971860 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:41.972640 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.093770 22901 raft_consensus.cc:2410] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:09:41.973330 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:42.013041 29870 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.327s	user 0.002s	sys 0.001s
I20260709 13:09:42.094352 22823 raft_consensus.cc:2376] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17. Re-sending same reply.
W20260709 13:09:41.978067 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 84ea43361e534fbebad704327c9f6367: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:41.915377 29615 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:09:41.986796 29875 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:09:41.987344 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:41.990368 29546 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:41.990545 29800 consensus_meta.cc:287] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.001s
I20260709 13:09:41.991114 29751 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:41.942684 29796 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:41.943876 29251 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:41.994545 29878 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:41.997987 29830 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:41.998690 29871 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.006033 22902 raft_consensus.cc:1217] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Deduplicated request from leader. Original: 22.20->[24.21-24.21]   Dedup: 24.21->[]
I20260709 13:09:41.921556 29884 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:42.010078 29618 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:42.019014 29704 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:42.020684 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:42.022624 29847 log.cc:927] Time spent T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.851s	user 0.000s	sys 0.009s
I20260709 13:09:42.030166 22896 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
W20260709 13:09:42.031391 29876 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:09:42.034111 29639 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.039944 29781 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:09:42.044538 29852 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.871s	user 0.012s	sys 0.009s
W20260709 13:09:42.045310 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.045643 22905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2962) took 1439 ms. Trace:
I20260709 13:09:42.050050 29061 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:42.133831 29061 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:42.052601 29821 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.142765 23445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1827) took 1396 ms. Trace:
I20260709 13:09:42.144678 29821 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.143301 23445 rpcz_store.cc:276] 0709 13:09:40.746413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.779110 (+ 32697us) service_pool.cc:224] Handling call
0709 13:09:40.788373 (+  9263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:40.838860 (+ 50487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:40.838893 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:40.839358 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:09:41.019114 (+179756us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:41.019153 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.019209 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:42.043804 (+1024595us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:42.142391 (+ 98587us) raft_consensus.cc:1685] finished
0709 13:09:42.142421 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.142680 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103207,"raft.queue_time_us":51173,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":9992,"spinlock_wait_cycles":55876864,"thread_start_us":175054,"threads_started":1,"wal-append.queue_time_us":333431}
I20260709 13:09:42.054119 29399 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:42.054744 29836 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.054867 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:42.163445 22827 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:42.164575 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:42.165390 22824 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:42.166534 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:42.055958 23444 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:42.058144 29843 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:42.167325 29843 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:42.059788 29829 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:42.059839 29862 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.889s	user 0.004s	sys 0.005s
I20260709 13:09:42.060606 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.055994 22903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:42.064486 29834 raft_consensus.cc:3055] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:09:42.068281 29863 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.431s	user 0.002s	sys 0.003s
I20260709 13:09:42.068830 29825 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.071310 22755 raft_consensus.cc:1240] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:42.072597 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:42.072763 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:42.077360 29864 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.901s	user 0.000s	sys 0.019s
W20260709 13:09:42.080149 29868 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.443s	user 0.002s	sys 0.000s
I20260709 13:09:42.080696 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1836) took 1104 ms. Trace:
W20260709 13:09:42.085301 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.085812 29083 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:42.088630 23452 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:42.093014 22828 raft_consensus.cc:2410] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 23 because replica has last-logged OpId of term: 22 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 18.
I20260709 13:09:42.100661 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:42.104452 22831 raft_consensus.cc:2368] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21.
I20260709 13:09:42.109745 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:42.116391 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:42.185523 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 85026.6us
I20260709 13:09:42.116750 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:42.186518 22823 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 23: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:42.122285 22902 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 24. Current term is 25. Ops: [24.21-24.21]
W20260709 13:09:42.124352 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.131299 22825 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:42.132557 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:42.136420 29061 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.187685 29061 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:42.140079 29639 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.189160 22823 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:42.127305 29704 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:42.190661 29704 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.191947 29704 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:42.141990 29861 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.972s	user 0.003s	sys 0.009s
I20260709 13:09:42.142490 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:42.149652 23448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1826) took 1403 ms. Trace:
I20260709 13:09:42.194979 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.195228 22827 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:42.151428 29850 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.965s	user 0.000s	sys 0.013s
I20260709 13:09:42.151629 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:42.182471 23450 rpcz_store.cc:276] 0709 13:09:40.977074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.983241 (+  6167us) service_pool.cc:224] Handling call
0709 13:09:41.015186 (+ 31945us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:41.021776 (+  6590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:41.021825 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:41.042351 (+ 20526us) log.cc:844] Serialized 26 byte log entry
0709 13:09:41.269716 (+227365us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:41.269766 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.269847 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:41.949672 (+679825us) raft_consensus.cc:1685] finished
0709 13:09:41.949689 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.080607 (+130918us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5122,"thread_start_us":188267,"threads_started":1,"wal-append.queue_time_us":319023}
I20260709 13:09:42.153609 29251 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.153674 29646 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:42.142803 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.199190 22896 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:42.133775 22905 rpcz_store.cc:276] 0709 13:09:40.606058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.606237 (+   179us) service_pool.cc:224] Handling call
0709 13:09:40.606556 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:41.874506 (+1267950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:42.045334 (+170828us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540034bcd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:42.045546 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2865796864}
W20260709 13:09:42.168845 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 230243us
I20260709 13:09:42.171370 22903 raft_consensus.cc:606] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.116914 29828 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:42.104195 29857 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.093356 29819 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:42.179593 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 107075us
I20260709 13:09:42.205380 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:42.188503 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.193099 29083 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.207504 29083 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.185843 22901 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:42.178668 29891 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:42.208982 22756 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:42.183151 29835 consensus_meta.cc:287] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.795s	user 0.006s	sys 0.005s
I20260709 13:09:42.173234 29834 raft_consensus.cc:740] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:42.216645 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:42.196422 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 104696us
I20260709 13:09:42.197852 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.197780 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.145701 22898 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:42.195128 23448 rpcz_store.cc:276] 0709 13:09:40.746145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.746261 (+   116us) service_pool.cc:224] Handling call
0709 13:09:40.766557 (+ 20296us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:40.789687 (+ 23130us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:40.789716 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:40.813136 (+ 23420us) log.cc:844] Serialized 52 byte log entry
0709 13:09:40.912611 (+ 99475us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:40.912666 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:40.912748 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:42.004078 (+1091330us) spinlock_profiling.cc:243] Waited 75.2 ms on lock 0x7b2c002212d8. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb730ab291 00007fcb730c60ca 00007fcb74df44ad 00007fcb74dfbf8e 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:09:42.021830 (+ 17752us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:42.149391 (+127561us) raft_consensus.cc:1685] finished
0709 13:09:42.149422 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.149586 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54508,"raft.queue_time_us":71771,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":11900,"spinlock_wait_cycles":215827072,"thread_start_us":99046,"threads_started":1,"wal-append.queue_time_us":357560}
I20260709 13:09:42.218866 22755 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:42.202255 22754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3116) took 1031 ms. Trace:
I20260709 13:09:42.203083 29821 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:42.218958 22753 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:42.203552 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43998 (request call id 2968) took 1250 ms. Trace:
I20260709 13:09:42.203742 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 2121) took 1069 ms. Trace:
I20260709 13:09:42.203552 22826 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:42.171861 29836 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.204752 22752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3117) took 1033 ms. Trace:
I20260709 13:09:42.173525 29829 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.205341 29825 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 28 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.182226 29725 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:42.153048 29796 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:42.212152 23416 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.591s (SENT)
I20260709 13:09:42.193629 29399 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:42.219218 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:42.206228 29886 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:09:42.219750 22754 rpcz_store.cc:276] 0709 13:09:41.171072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.172277 (+  1205us) service_pool.cc:224] Handling call
0709 13:09:41.172660 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:41.172875 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:41.172907 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:41.173428 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:09:41.179836 (+  6408us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:41.179888 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.179993 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:42.076614 (+896621us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:42.201977 (+125363us) raft_consensus.cc:1685] finished
0709 13:09:42.202001 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.202190 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13064064,"thread_start_us":5991,"threads_started":1,"wal-append.queue_time_us":2061}
I20260709 13:09:42.224695 22752 rpcz_store.cc:276] 0709 13:09:41.171353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.171488 (+   135us) service_pool.cc:224] Handling call
0709 13:09:41.171784 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:41.171978 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:41.172003 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:41.172420 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 13:09:41.232787 (+ 60367us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:41.232837 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.232927 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:42.067094 (+834167us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:42.204495 (+137401us) raft_consensus.cc:1685] finished
0709 13:09:42.204516 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.204700 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11628416,"thread_start_us":59948,"threads_started":1,"wal-append.queue_time_us":464658}
I20260709 13:09:42.220733 22751 rpcz_store.cc:276] 0709 13:09:41.134671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.134782 (+   111us) service_pool.cc:224] Handling call
0709 13:09:41.182134 (+ 47352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:41.182518 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:41.182556 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:41.221107 (+ 38551us) log.cc:844] Serialized 26 byte log entry
0709 13:09:41.302097 (+ 80990us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:41.302148 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.302220 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:42.079353 (+777133us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:42.203394 (+124041us) raft_consensus.cc:1685] finished
0709 13:09:42.203413 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.203585 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4371,"raft.queue_time_us":106,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":23666,"spinlock_wait_cycles":6288768,"thread_start_us":68140,"threads_started":1,"wal-append.queue_time_us":402771}
I20260709 13:09:42.209214 29891 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:42.219216 22719 leader_election.cc:336] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.600s (SENT)
I20260709 13:09:42.220559 22897 rpcz_store.cc:276] 0709 13:09:40.952682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.952838 (+   156us) service_pool.cc:224] Handling call
0709 13:09:40.985651 (+ 32813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:40.994736 (+  9085us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:40.994785 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:40.995310 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:09:41.135216 (+139906us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:41.135259 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.135336 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:42.059154 (+923818us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:42.203296 (+144142us) raft_consensus.cc:1685] finished
0709 13:09:42.203321 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.203495 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32932,"raft.queue_time_us":25448,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":13477,"spinlock_wait_cycles":43950976,"thread_start_us":75141,"threads_started":1,"wal-append.queue_time_us":148936}
I20260709 13:09:42.228226 23451 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:42.229810 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.168661 29843 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.237920 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.241808 22901 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:42.246748 29083 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:42.247155 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.247773 22902 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 11.
I20260709 13:09:42.249439 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.233908 29399 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:42.257103 29399 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:42.262269 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:42.242010 29796 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:42.270435 29796 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:42.271395 29796 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.250218 22905 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
W20260709 13:09:42.273602 29900 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:09:42.257100 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 24 index: 21 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.274329 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.275439 29819 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:42.278232 29836 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:42.279243 29829 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:42.229176 29725 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.279148 29834 consensus_queue.cc:260] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.279101 29399 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.280017 29725 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:42.280535 29834 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:42.226367 29792 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:42.281527 29399 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.226758 29821 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.284090 29871 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:42.285772 29821 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:42.286062 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:42.289858 23445 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:09:42.290479 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 3583f8fda02d46af9ef0ff2ae6f1b443: Illegal state: Replica 3b626c41600d4737a178ebab8a7ebfc2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:42.206195 29828 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.295943 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.297008 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:42.300720 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:42.302819 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.275465 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:42.303916 23453 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
W20260709 13:09:42.304641 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:42.305289 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57513.7us
I20260709 13:09:42.305433 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.306660 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:42.296980 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:42.312012 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.207661 29266 consensus_queue.cc:1059] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:09:42.331954 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:42.333206 22822 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:42.332194 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:42.461687 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.462713 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.484478 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:42.485692 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.486313 22754 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:42.316937 29546 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.488062 29546 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.490192 29546 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:42.490818 22828 raft_consensus.cc:2410] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:09:42.491870 22901 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 28: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:42.493278 29616 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:42.493453 29704 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:42.494530 29616 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.494946 29616 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:42.493541 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:42.497552 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:42.498270 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 162478us
I20260709 13:09:42.499578 22751 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 20. Current term is 21. Ops: []
W20260709 13:09:42.503646 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:42.503995 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 197383us
W20260709 13:09:42.505246 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.223106 29835 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.257588 29662 consensus_queue.cc:1059] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.207s
I20260709 13:09:42.328492 29792 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.507404 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.508087 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:42.511356 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 292137us
I20260709 13:09:42.514012 22899 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:42.514042 22748 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:42.520855 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.525533 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.524971 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:42.520972 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:42.528856 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.529598 22904 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:42.418604 29821 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.096s
W20260709 13:09:42.529838 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.531500 22901 raft_consensus.cc:2368] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21.
I20260709 13:09:42.508420 29859 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:09:42.520093 22719 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.653s (SENT)
I20260709 13:09:42.536790 22900 raft_consensus.cc:2393] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:42.535364 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.506982 29810 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:09:42.516328 29781 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:42.541235 29083 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:42.543754 29083 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.544080 29083 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:42.541337 22899 raft_consensus.cc:1240] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:09:42.547418 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.534679 29859 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:42.545655 29639 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:09:42.561941 22719 leader_election.cc:336] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.683s (SENT)
I20260709 13:09:42.562207 29859 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:42.582083 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 77318.1us
I20260709 13:09:42.594908 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.540794 29891 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.602435 29891 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:42.607082 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 395188us
W20260709 13:09:42.607753 23416 leader_election.cc:336] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.651s (SENT)
I20260709 13:09:42.520356 29792 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.610260 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.610862 22899 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
W20260709 13:09:42.611954 22719 leader_election.cc:336] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.867s (SENT)
I20260709 13:09:42.617022 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:42.614660 23416 leader_election.cc:336] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.849s (SENT)
I20260709 13:09:42.618712 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:42.634471 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.636009 22897 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:42.547243 29781 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.643867 29083 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.651907 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.653923 22754 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:42.653847 22719 leader_election.cc:336] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 1.929s (SENT)
I20260709 13:09:42.654920 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.635149 29878 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:09:42.658422 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 43791us
I20260709 13:09:42.658604 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.659199 22904 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
W20260709 13:09:42.660745 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.647270 29704 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:42.662268 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.664057 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.664425 22902 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:42.664884 29083 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:42.638226 29725 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.668673 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:42.664812 29639 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:42.672060 29792 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.674674 22903 raft_consensus.cc:2368] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:42.677009 22829 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:42.673995 29725 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:42.678001 29899 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:09:42.681285 23416 leader_election.cc:400] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21.
I20260709 13:09:42.682372 23416 leader_election.cc:403] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:42.680917 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.683213 22904 raft_consensus.cc:2468] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:42.684212 22750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3118) took 1512 ms. Trace:
I20260709 13:09:42.682564 29704 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:42.686867 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.687281 29704 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:42.688539 29704 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:42.680183 29908 log.cc:927] Time spent T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:09:42.705085 29561 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.705597 29561 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:42.708704 29907 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:09:42.712217 29561 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:42.680140 29835 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.684630 22750 rpcz_store.cc:276] 0709 13:09:41.171673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.178667 (+  6994us) service_pool.cc:224] Handling call
0709 13:09:41.178945 (+   278us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:41.179337 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:41.255734 (+ 76397us) log.cc:844] Serialized 14 byte log entry
0709 13:09:41.708790 (+453056us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b18003d4b60. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb749a34bc 00007fcb71cf148b 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b23a
0709 13:09:41.708986 (+   196us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:41.712627 (+  3641us) log.cc:844] Serialized 52 byte log entry
0709 13:09:41.712769 (+   142us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:41.712804 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.712873 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:42.011995 (+299122us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:42.664047 (+652052us) spinlock_profiling.cc:243] Waited 7.87 ms on lock 0x7b2400139bc8. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71b949c4 00007fcb71b92acb 00007fcb71d4ddc9 00007fcb71d4da55 00007fcb71d4cfb0 00007fcb749c3488 00007fcb749c3be8 00007fcb7499b398 00007fcb7499676a 00007fcb74995bb6
0709 13:09:42.683985 (+ 19938us) raft_consensus.cc:1685] finished
0709 13:09:42.684008 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.684147 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99808,"prepare.queue_time_us":93007,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":49792,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":28384,"spinlock_wait_cycles":104034816,"thread_start_us":200797,"threads_started":2,"wal-append.queue_time_us":352599}
W20260709 13:09:42.720178 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.680027 29616 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.723305 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.722499 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.716928 29781 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:42.730644 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 76883.5us
I20260709 13:09:42.731876 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:42.732437 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.726384 29835 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.734316 22896 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:42.734504 22900 raft_consensus.cc:2393] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:42.736636 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:42.737671 22719 leader_election.cc:336] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 2.001s (SENT)
I20260709 13:09:42.739398 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:42.739488 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.743494 29781 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.744005 29781 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.756891 29616 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:42.759001 29871 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:42.731841 29909 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21.
I20260709 13:09:42.776880 22896 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:42.737893 29835 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:42.781533 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 43879us
W20260709 13:09:42.789273 22719 leader_election.cc:336] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 2.193s (SENT)
I20260709 13:09:42.789816 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.791994 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.792909 22755 raft_consensus.cc:2368] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19.
I20260709 13:09:42.782130 29781 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.782917 29836 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:42.793529 29835 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.793836 29836 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:42.794538 22719 leader_election.cc:336] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Timed out: RequestConsensusVote RPC to 127.21.31.67:35797 timed out after 2.195s (SENT)
I20260709 13:09:42.795092 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.794759 29836 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.795746 29834 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:42.796142 29836 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:42.797008 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2337) took 1043 ms. Trace:
I20260709 13:09:42.797139 29871 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:42.797475 22829 rpcz_store.cc:276] 0709 13:09:41.753851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.755147 (+  1296us) service_pool.cc:224] Handling call
0709 13:09:42.796827 (+1041680us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b5400091fd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:42.796947 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917688576}
I20260709 13:09:42.799036 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:42.799731 29871 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:42.800203 29871 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:42.784248 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.801030 29871 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.802544 29858 raft_consensus.cc:3055] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:42.802999 29858 raft_consensus.cc:740] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:42.785049 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.785722 29857 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.786031 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.787037 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.804931 22753 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:42.805171 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:42.805353 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 222455us
I20260709 13:09:42.806596 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.807096 22752 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:42.809293 29912 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.809446 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:42.809810 29912 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.811738 29912 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:42.811964 29910 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.812230 29912 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:42.812454 29910 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.814746 29910 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:42.816068 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.812734 29912 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.818889 29843 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:42.820011 29912 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:42.811764 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.822579 22752 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:42.822988 29618 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.823443 29618 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.824992 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.828702 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.828924 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.782872 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.836195 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:42.836882 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.837110 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:42.829255 22749 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:42.830497 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.838020 22756 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:42.831177 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:42.832291 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.837679 23450 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:42.839366 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:42.839715 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.833277 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:42.834257 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:42.840297 22904 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:42.834522 29618 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.847828 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:42.845681 22896 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:42.846063 23447 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 18 LEARNER]: Advancing to term 21
I20260709 13:09:42.838819 29639 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:42.825073 29849 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.847254 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:42.842715 29858 consensus_queue.cc:260] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.851681 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.861568 22753 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:42.780876 29859 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.859273 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.859506 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.860229 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.833293 29909 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:42.863554 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.864500 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.867962 22752 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:42.875514 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.874260 22756 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:42.876487 22901 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:42.878588 22755 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:09:42.886215 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.886528 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.890520 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.890595 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.891019 22749 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:42.892282 22747 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:42.892438 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.889012 23447 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 21 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:42.893878 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.894291 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.894778 22896 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:42.895712 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:42.896121 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.893441 29836 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.897706 29836 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.899585 29061 raft_consensus.cc:993] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:09:42.899663 29836 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:42.900174 29061 raft_consensus.cc:1081] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Signalling peer 7aad11e70ca94fce8f672db05f1c28c3 to start an election
I20260709 13:09:42.900302 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.901607 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.901850 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:42.874243 29832 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.902562 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:42.901072 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:42.903441 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:42.903955 22825 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 16.
I20260709 13:09:42.904733 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.906040 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.904712 22896 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:09:42.906818 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.907506 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.906004 22826 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:42.906423 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:42.907016 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.911518 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.911700 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.911989 22902 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:09:42.912178 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.915992 22752 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:42.916880 29083 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:42.916985 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.917827 22902 raft_consensus.cc:2410] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:09:42.918506 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:42.920282 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.920419 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.921207 22904 raft_consensus.cc:2410] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 23 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 13:09:42.921710 22871 leader_election.cc:400] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19.
I20260709 13:09:42.922254 22871 leader_election.cc:403] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:42.922226 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.922760 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.928539 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.928846 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.923317 22902 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:42.930475 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:42.931257 22900 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:09:42.931397 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.932261 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.929129 22901 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:09:42.902629 29832 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.934679 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:42.935518 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.935761 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:42.936630 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.937121 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.937108 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:42.938810 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:42.938298 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.939524 22871 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.940080 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:42.940670 22871 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.941119 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:42.941460 29061 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19.
I20260709 13:09:42.903024 29914 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:42.941725 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.941726 23452 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:42.864018 29849 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:42.942536 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:42.945363 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.945937 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.943979 29891 raft_consensus.cc:3055] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:42.933777 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.878093 29639 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.950219 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.951366 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.951973 22904 raft_consensus.cc:2468] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 11.
I20260709 13:09:42.952102 29061 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:42.951987 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.878861 29781 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:09:42.952898 22871 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.946574 22751 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:42.953336 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:42.953778 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:42.956425 22902 raft_consensus.cc:1275] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:42.957512 22871 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.958228 29061 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:42.959312 22871 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.959543 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:42.886004 29909 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:42.961215 22750 raft_consensus.cc:2410] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:09:42.957832 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:42.961470 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.962610 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.962623 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:42.953186 29792 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:42.966218 22751 raft_consensus.cc:2393] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:42.962771 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:42.978145 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:42.967207 22824 raft_consensus.cc:2410] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:09:42.968015 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:42.944344 29860 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:42.965108 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.978935 29061 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:42.979816 29061 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.979671 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.980430 29061 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:42.981189 29083 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:42.952592 29891 raft_consensus.cc:740] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:42.956969 29810 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:42.978694 23448 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:42.959729 29871 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:42.979730 22822 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:42.960335 29546 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:42.963552 29834 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:42.960183 29266 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:42.969650 29825 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:42.985559 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:42.965569 29662 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:42.954399 29914 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.001967 22871 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:42.974089 29615 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.994314 22756 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:09:42.957118 29819 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:09:42.992509 29266 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.992337 29795 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:43.036109 22899 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
W20260709 13:09:43.035705 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 47969.8us
I20260709 13:09:43.035897 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:43.045503 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 43620us
I20260709 13:09:43.035502 22751 raft_consensus.cc:2410] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:09:43.046463 22871 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:43.041944 22749 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:42.966305 29751 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:09:42.984740 29821 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:43.048903 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:43.058993 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 101217us
I20260709 13:09:42.989635 29903 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:09:43.049935 29795 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:43.050570 29926 log.cc:927] Time spent T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:09:43.022764 29662 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.056102 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:43.066041 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.056751 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:43.056972 29909 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:43.022754 29798 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:43.057395 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:43.074344 22830 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:43.059559 22828 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:43.059767 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:43.035997 29878 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:43.059190 22749 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:43.061239 29061 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.058595 29836 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:42.984344 29800 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.030431 29639 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.031924 29832 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:43.067216 22827 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:43.044816 29266 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:43.069543 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:43.132658 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.136044 29828 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:43.034420 29849 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:43.072695 22871 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:43.139465 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66858.2us
I20260709 13:09:43.058197 29751 consensus_queue.cc:1243] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:09:43.140985 22871 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.142161 22755 raft_consensus.cc:1217] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Deduplicated request from leader. Original: 21.19->[22.20-25.22]   Dedup: 22.20->[23.21-25.22]
I20260709 13:09:43.142622 22755 pending_rounds.cc:85] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 20
I20260709 13:09:43.142829 22755 pending_rounds.cc:107] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 24.21
I20260709 13:09:43.143024 22755 raft_consensus.cc:2889] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:43.076817 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 84834.3us
W20260709 13:09:43.078182 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:43.149375 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 165365us
I20260709 13:09:43.078670 22831 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:09:43.150522 22831 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:43.150667 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:43.150902 22831 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:42.985149 29860 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.151348 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.151957 22899 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:43.152516 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.152731 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:43.153468 22899 raft_consensus.cc:2368] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:43.153633 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.153540 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.154053 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 24 index: 21 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.154259 22896 raft_consensus.cc:2376] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19. Re-sending same reply.
I20260709 13:09:43.154718 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.081674 29928 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:43.082908 29871 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:09:43.083927 29859 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:43.166299 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.077668 29725 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:43.086031 29911 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:43.182276 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 99793us
I20260709 13:09:43.094624 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:43.097441 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:43.098847 29925 log.cc:927] Time spent T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:09:43.112167 29878 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:43.069787 29821 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:43.069602 29662 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:43.114262 29924 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:09:43.212332 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.162376 29783 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.120384 22823 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:43.215466 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:43.124157 29061 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.155566 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.162130 22828 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:43.167814 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:43.168103 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.216873 22901 raft_consensus.cc:2393] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 28: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:43.104206 29795 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:43.163638 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:43.217514 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 53920us
I20260709 13:09:43.181805 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:43.218345 22827 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:43.077806 29891 consensus_queue.cc:260] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.183736 22752 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:43.185592 22825 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:43.194465 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 61905us
I20260709 13:09:43.169943 23447 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:09:43.171705 29923 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.211536 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.212087 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.161669 29860 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:43.161480 29800 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:43.212370 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.218889 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:43.223603 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:43.224013 22898 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:43.166996 29751 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:09:43.115701 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:43.182380 29828 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:43.119154 29914 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.216035 22823 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
W20260709 13:09:43.216751 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49045.3us
I20260709 13:09:43.217541 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:43.204984 22822 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:43.216279 22900 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:43.044831 29615 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.219717 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.213079 22904 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:43.261831 22905 raft_consensus.cc:2393] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 11: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:43.263998 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:43.270731 23445 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:09:43.272711 22753 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:43.273250 22751 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
W20260709 13:09:43.273939 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 54285.2us
I20260709 13:09:43.274665 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:43.226140 29859 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:43.225414 29546 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:09:43.278267 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49842.5us
I20260709 13:09:43.278666 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:43.274683 22831 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:43.279693 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48383.7us
I20260709 13:09:43.229723 29399 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:43.269232 22827 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:43.207194 29911 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.282615 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:43.284441 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:43.278002 29546 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.292948 22831 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.294135 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:43.289897 29546 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:43.278954 29819 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:43.229382 29384 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:09:43.303198 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.304369 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:43.304760 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:43.305994 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.278095 29828 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:43.283108 29399 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.279184 29859 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:43.282122 29251 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:43.288022 29795 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:43.314284 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.311376 29399 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:43.301926 29903 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:43.307674 29836 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:43.307104 29704 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:43.311510 29914 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:43.328712 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:43.308439 29796 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:09:43.354485 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:43.335109 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:43.328361 29828 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:43.321853 29836 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.329608 29914 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.365124 23452 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:43.362818 29871 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:09:43.317781 29251 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.374174 29662 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.375782 29251 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.376088 29662 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.345738 29933 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.377532 29836 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:43.374185 29546 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.382515 29615 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.291791 29878 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.385589 23445 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:43.386451 29083 raft_consensus.cc:2764] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:09:43.391301 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 108752us
I20260709 13:09:43.348557 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.403143 23452 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:43.403611 29911 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.407286 22749 pending_rounds.cc:85] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 11
I20260709 13:09:43.407606 22749 pending_rounds.cc:107] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 13:09:43.407829 22749 raft_consensus.cc:2889] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:43.409021 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:43.409634 23452 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:43.421784 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:43.422382 22753 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:43.421629 29546 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.432440 22753 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:43.439280 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:43.439888 22752 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:43.311890 29859 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:43.348770 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.444130 22904 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:43.446429 22831 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:09:43.447486 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.448046 22747 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:43.448923 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:43.448956 22831 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 20. Ops: []
W20260709 13:09:43.449708 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 101214us
I20260709 13:09:43.451855 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.453510 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:43.383322 29914 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:43.425515 29824 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:43.459878 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.460246 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:43.463554 22904 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:43.456578 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:43.461473 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:43.462702 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:43.444779 29828 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:43.466956 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.464829 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:43.468094 23447 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:43.465425 22752 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 24.
I20260709 13:09:43.465457 22900 raft_consensus.cc:3055] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:43.453845 29903 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.469532 22900 raft_consensus.cc:740] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:43.471853 22797 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:43.489135 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:43.495994 23447 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:43.496819 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.485489 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:43.506690 23452 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:43.495316 29824 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:43.510958 29824 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:43.514070 29948 consensus_queue.cc:1048] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:09:43.520572 22899 raft_consensus.cc:1240] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:09:43.507795 29946 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:43.526149 29251 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:43.455134 29546 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:09:43.497296 29939 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:43.498258 29940 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.462554 29933 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:09:43.495738 22900 consensus_queue.cc:260] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.483881 29923 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.541206 22900 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:43.467180 29751 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:09:43.494650 29618 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:43.500204 29810 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:43.472436 29795 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:43.471109 29938 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:43.494943 29821 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:43.559870 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.508816 29725 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:09:43.571264 23416 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:43.552861 29952 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:43.559063 29618 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:43.575080 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 90644us
I20260709 13:09:43.560149 29751 consensus_queue.cc:1243] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:09:43.560913 29938 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.567329 29704 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:43.574034 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:43.570183 29821 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:43.555810 29939 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:09:43.586658 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.603325 29618 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.577768 29909 raft_consensus.cc:2764] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:09:43.578075 29946 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:43.493973 29903 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.546567 29923 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.618918 29923 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.619016 29903 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.619752 29950 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:43.620148 29950 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.618278 29859 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.620448 29950 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:43.620653 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:43.621904 29954 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:43.624284 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.624615 29859 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.626286 29945 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.626647 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.628023 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:43.629068 29945 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:43.628794 29821 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.616745 29836 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.636046 22900 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:43.637642 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.638660 22903 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:43.638865 22900 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:43.631449 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.590655 29938 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:43.655038 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.589612 29835 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:09:43.636780 29945 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.658412 29399 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.660574 29399 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.655409 29704 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:43.662077 29669 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:43.661808 29704 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.662480 29669 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:43.662693 29948 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:43.663446 29948 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:43.663854 29948 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:43.663976 29704 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:43.664717 29948 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.665805 29704 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:43.665974 29948 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:43.666352 29704 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.668344 29704 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:43.668843 29941 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:09:43.665225 29546 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:43.669548 29546 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:43.670069 29955 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:43.670358 29546 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.670593 29955 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.671192 29615 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:43.671554 29546 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:43.671705 29615 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.663376 29669 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.673952 29615 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.641089 29946 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.675720 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.676438 22752 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:43.612164 29561 consensus_queue.cc:1059] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 13:09:43.677349 29561 raft_consensus.cc:3055] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:09:43.677690 29561 raft_consensus.cc:740] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 27 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:43.677701 29954 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:43.678586 29561 consensus_queue.cc:260] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.679754 29561 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:43.675849 29946 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:43.672631 29955 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:43.673950 29266 consensus_meta.cc:287] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.000s
I20260709 13:09:43.689955 22903 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:43.587551 29952 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.695570 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.698045 22719 leader_election.cc:400] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:43.684160 29819 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.699977 22719 leader_election.cc:403] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:09:43.699429 29751 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.642359 29795 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.702020 29828 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:43.687283 29266 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:43.714847 29669 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:43.715349 29266 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:43.716054 22824 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:43.718387 29858 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:43.719321 29860 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.720552 22827 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:43.711163 29914 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:43.733691 29914 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:43.735791 29781 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:43.736541 29656 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:43.736660 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:43.736831 29824 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:43.737597 29824 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:43.738319 29824 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.739357 29824 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:43.740397 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.741482 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.742188 22752 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:43.742537 29639 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:43.742945 29639 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:43.743777 29639 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.744503 22752 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:43.744942 29639 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:43.745597 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.746104 22755 raft_consensus.cc:1240] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:43.749536 29792 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:43.749997 29792 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:43.750844 29792 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.754662 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:43.755352 29751 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.756604 29751 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:43.712106 29945 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:43.757663 29903 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:43.758184 29903 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:43.759055 29903 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:43.760335 29903 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:43.765377 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:43.765936 22750 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:43.767566 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.768505 22748 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:43.769809 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57374.5us
I20260709 13:09:43.770507 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:43.769637 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.772132 22751 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:43.773442 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:43.774513 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.777493 22751 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:43.778132 22747 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 15: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:43.784516 29828 consensus_queue.cc:1059] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:43.785812 29828 raft_consensus.cc:3055] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:43.786168 29828 raft_consensus.cc:740] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:43.779899 29859 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.787124 29828 consensus_queue.cc:260] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:43.787878 29859 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:43.788750 29828 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:43.791425 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:43.782646 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:43.792315 22747 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:43.737020 29656 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:43.777606 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:43.736699 29781 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 13:09:43.902882 29704 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:09:43.867806 29962 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:09:43.780162 29932 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:43.858691 29909 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.776741 29957 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:43.858160 22797 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:43.969826 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 484348us
I20260709 13:09:43.842679 22750 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
W20260709 13:09:43.906325 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:43.974483 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 116379us
W20260709 13:09:43.908739 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:43.856117 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:43.891572 29800 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:43.895051 29836 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:43.901926 29859 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:43.818508 22752 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:09:43.880870 29961 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:09:43.935075 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:43.864591 29832 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:43.865463 29725 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:43.962857 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 185301us
I20260709 13:09:43.849870 29952 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:43.967864 22756 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:09:43.987205 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:43.988817 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:43.996959 22826 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:43.997403 22751 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:43.966817 29871 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:43.850554 29939 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.130s
I20260709 13:09:43.980268 29546 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.979930 29843 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.001291 22794 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:43.983814 29662 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:43.995049 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.010722 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.013202 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:44.013775 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.014546 22899 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:44.014904 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:44.017709 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.019321 29810 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.007270 29859 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:09:43.984905 29800 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:44.013942 23450 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:43.943719 29950 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.023082 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.009179 22797 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:43.985875 29912 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:43.980482 29909 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.033634 22899 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 24.
I20260709 13:09:44.019810 29810 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.043923 22826 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:44.046402 22830 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 20. Ops: []
I20260709 13:09:44.059170 23448 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:09:43.992300 29943 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:44.050087 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:44.062996 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.063622 22901 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:43.999791 29871 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.033231 29922 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:44.032117 29266 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.050369 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.078163 23451 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 23: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:44.078449 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:44.080078 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:44.080818 23453 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:44.061506 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 66540.3us
I20260709 13:09:44.082991 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.064618 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.022199 29662 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.998970 29832 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:09:44.067061 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.001446 29725 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:43.999750 29384 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:09:44.056751 29937 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.061753 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.033491 29546 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.088748 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:44.050566 29957 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:44.075995 29871 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:44.034278 29843 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.090793 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.092196 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.092254 22901 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:44.113768 22900 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:44.087308 29662 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:44.087999 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:44.056593 29910 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:44.097738 23452 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:44.089794 23447 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:44.094712 29834 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:09:44.062141 29912 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:44.096968 29911 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:44.129021 22905 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:09:44.129518 22905 pending_rounds.cc:85] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 13
I20260709 13:09:44.129805 22905 pending_rounds.cc:107] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:09:44.130090 22905 raft_consensus.cc:2889] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:44.099714 29922 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:44.139128 29930 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.865s	user 0.006s	sys 0.002s
I20260709 13:09:44.141698 29810 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.147640 29836 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:44.133776 29910 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:44.150885 22749 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:44.092867 29266 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.125252 29798 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:44.130218 29945 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:44.110648 29937 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:44.168334 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 80378.1us
I20260709 13:09:44.169286 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.173048 29945 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:44.173597 29945 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:44.174041 29945 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:44.175662 29656 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.178867 29891 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:44.182469 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:44.184820 29384 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:44.185462 29384 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:44.186851 29384 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.187660 29546 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.149129 29910 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:44.197510 22747 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:44.163837 29751 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:09:44.208818 29751 consensus_queue.cc:1243] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:09:44.162564 29938 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.161000 29798 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:44.213874 29798 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:44.214098 29909 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:44.217686 29922 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.222549 29725 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:44.225656 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:44.226421 22752 raft_consensus.cc:2410] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 13:09:44.226536 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 57381.9us
I20260709 13:09:44.228148 22899 raft_consensus.cc:2410] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:09:44.229935 23450 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 24.
W20260709 13:09:44.230018 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46507.8us
I20260709 13:09:44.232606 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:44.233431 22827 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:44.233009 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:44.238512 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:44.177044 29656 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:44.241683 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:44.242288 22827 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:44.245872 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:44.247035 22823 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:44.247668 22825 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:09:44.248181 22831 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:09:44.249588 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:44.250380 22822 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 15.
I20260709 13:09:44.250595 23450 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 13:09:44.239084 22749 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:44.227331 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.254261 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:44.255645 22824 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:44.258251 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.262260 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:44.265748 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.265192 22830 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:44.272471 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.155550 29832 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:44.282749 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 55506.2us
I20260709 13:09:44.283442 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.285355 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:44.254338 29946 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:09:44.288139 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:44.289285 29615 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:44.289798 29832 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:44.293584 29859 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:44.293998 29616 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:09:44.294435 29937 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.263147 29950 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:44.296264 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:44.306169 22752 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:44.284291 29957 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:44.295797 22749 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:44.274750 29965 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:44.268673 29918 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:44.284888 29795 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:44.297827 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.327158 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.288228 29932 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:44.307420 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:44.295076 29798 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.303977 29859 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.330976 29859 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:44.331620 29798 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.311529 29561 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:44.332538 29561 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.330480 29878 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:44.335112 29561 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:44.337502 29944 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:44.338681 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:44.310899 29957 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:44.326305 29795 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.350929 22752 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:44.352478 29725 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:44.349248 23453 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 24.
I20260709 13:09:44.338407 29944 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:09:44.342351 29918 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:44.338761 29843 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.351310 29957 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:44.359376 29795 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:44.350574 29950 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:44.386781 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.367970 29918 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.366868 29725 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:44.400282 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 104917us
I20260709 13:09:44.376691 29878 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:44.403182 29918 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:44.407168 29973 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:09:44.408860 29639 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.003s
I20260709 13:09:44.416950 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:44.418462 23445 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:44.421842 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.422267 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.422430 23452 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:44.420568 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.420714 22904 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:09:44.424173 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.425493 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:44.426132 23445 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:09:44.427130 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.428660 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:44.430390 22898 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:44.434722 29945 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.435837 22751 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:44.437011 29945 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.407406 29871 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:44.438140 29828 consensus_meta.cc:287] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.000s
I20260709 13:09:44.438862 29871 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:44.447149 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.447796 29859 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.449659 23444 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:44.448138 29978 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:44.454007 29965 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:44.400835 29950 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.459591 22752 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:44.405150 29725 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:44.425838 29932 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:09:44.465960 29971 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:09:44.468276 29662 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:44.429951 29980 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:09:44.470247 29938 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.471139 29932 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:44.471526 29932 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:44.471827 29615 raft_consensus.cc:1064] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:09:44.472075 22749 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:44.472293 29932 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:44.473296 29878 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.475209 29834 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:09:44.470268 29662 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:44.478250 29615 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.480722 29725 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:44.460080 29945 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:44.484232 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.487671 22898 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:44.483553 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:44.491276 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.484256 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:44.496129 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:44.429261 29639 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:09:44.498654 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:44.500046 29796 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:44.500450 29796 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:44.500815 29796 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:09:44.503857 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.462649 29836 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:09:44.506215 29836 consensus_queue.cc:1243] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:09:44.457520 29984 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
W20260709 13:09:44.515400 29951 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.641s	user 0.009s	sys 0.000s
I20260709 13:09:44.516398 22896 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:09:44.516827 22896 pending_rounds.cc:85] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 14
I20260709 13:09:44.517076 22896 pending_rounds.cc:107] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:09:44.517350 22896 raft_consensus.cc:2889] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:44.517583 22897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1705) took 1002 ms. Trace:
I20260709 13:09:44.515172 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:44.519804 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.524088 22898 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:44.462327 29965 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.525564 22903 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:44.517941 22897 rpcz_store.cc:276] 0709 13:09:43.514857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:43.516655 (+  1798us) service_pool.cc:224] Handling call
0709 13:09:43.517043 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:43.517479 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:43.517510 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:43.518035 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:09:43.572266 (+ 54231us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:43.587409 (+ 15143us) log.cc:844] Serialized 14 byte log entry
0709 13:09:43.588681 (+  1272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:43.588747 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:43.874317 (+285570us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b18002d0ca0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:44.514896 (+640579us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:44.517398 (+  2502us) raft_consensus.cc:1685] finished
0709 13:09:44.517417 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:44.517532 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":952,"prepare.queue_time_us":14917,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":3158,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":16859,"spinlock_wait_cycles":60864256,"thread_start_us":51456,"threads_started":1,"wal-append.queue_time_us":118797}
I20260709 13:09:44.526042 22896 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:44.529031 22904 raft_consensus.cc:1240] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:44.487841 29950 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:44.543526 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:44.544519 29857 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.549521 29857 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.554351 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:44.529182 29656 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:44.573648 23446 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:09:44.574265 23452 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:44.573658 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.552591 29561 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.557435 29662 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:44.544581 29965 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.556572 29859 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.600380 23453 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:09:44.606323 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.607168 23444 raft_consensus.cc:2410] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 13:09:44.608215 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 179663us
I20260709 13:09:44.585733 22794 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:44.610682 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:44.614815 22829 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19. Ops: []
W20260709 13:09:44.616055 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.616319 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:44.616583 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:44.617010 22829 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:44.617606 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:44.618330 23444 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:44.619123 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.561158 29948 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:09:44.577585 29792 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.598584 29561 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.632596 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:44.633991 29561 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:44.577392 29656 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.636034 29656 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:44.639523 29860 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.641952 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.562675 29957 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.656723 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.657898 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:44.658289 22903 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:44.661510 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:44.578795 29725 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:09:44.679498 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 121038us
I20260709 13:09:44.679859 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:44.686760 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.607399 29783 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:09:44.697719 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:44.696483 23445 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 14.
I20260709 13:09:44.663223 23444 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:44.700546 23453 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:09:44.628970 29662 raft_consensus.cc:2804] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:44.702378 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:44.637693 29266 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.709740 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.613835 29940 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:09:44.710759 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:44.672593 29992 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.623827 29824 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:09:44.715759 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:44.712715 23445 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 17: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:44.622510 29821 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.093s
I20260709 13:09:44.632161 29878 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 13:09:44.637655 29792 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.696844 29857 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.721282 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.720577 29083 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:44.726148 29083 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:44.725615 23449 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:44.727250 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:44.729804 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:44.713608 29943 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:44.696499 29725 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:44.730997 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:44.730878 29656 raft_consensus.cc:2764] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:09:44.739452 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:44.740113 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 127706us
I20260709 13:09:44.743616 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.713821 29704 consensus_queue.cc:1059] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:09:44.729964 29952 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:44.761240 29950 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:44.741760 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:44.764257 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:43.321023 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:44.724649 29662 raft_consensus.cc:697] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:44.745822 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:44.750501 29957 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.751330 29546 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:44.742172 29561 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:44.744652 29266 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:09:44.755695 29965 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.757277 29800 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:44.758095 29948 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:44.719841 29992 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.758756 29083 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.742969 29911 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:44.759613 29918 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.759929 29796 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.734064 29909 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:44.767318 22905 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:44.767366 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:44.823093 22899 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:44.769068 29943 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.756785 29795 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.776736 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:44.826200 23449 raft_consensus.cc:2393] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 23: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:44.775007 22903 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:44.780951 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:44.789525 22702 placement_policy_util.cc:248] tablet 135e2f7520a34b53aaf4ddb360a8a029: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 13:09:44.790894 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:44.821516 29083 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:44.777038 29912 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:44.780866 29857 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:44.824211 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:44.776710 29725 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:44.830427 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49555.2us
I20260709 13:09:44.831038 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:44.833914 29950 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:44.836055 29918 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.775077 29859 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:44.781122 29561 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:44.791798 29546 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:44.794486 29948 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:44.780007 29704 raft_consensus.cc:3055] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:44.836704 29943 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.853483 29704 raft_consensus.cc:740] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:44.777711 29800 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:44.850497 29083 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:44.845393 29859 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.849892 29857 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:44.825281 29909 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:44.858086 29083 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:44.858476 29083 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:44.859270 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.838565 29561 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:44.838791 29911 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:09:44.866086 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.855301 29948 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:44.784029 29824 consensus_queue.cc:1175] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7aad11e70ca94fce8f672db05f1c28c3" responder_term: 20 status { last_received { term: 18 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:44.854434 29704 consensus_queue.cc:260] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:44.885993 29986 log.cc:927] Time spent T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:09:44.886477 29704 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:44.890192 29992 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.859225 29859 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:44.897701 29857 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.902172 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.777627 29662 consensus_queue.cc:237] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:44.868304 29911 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.878396 29943 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.868178 29561 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:44.849099 29918 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:09:44.881856 22830 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:44.856182 29546 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.876588 29083 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.859700 29909 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:44.907722 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76721.9us
I20260709 13:09:44.909850 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:44.922456 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:44.923717 22897 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:09:44.925056 29656 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:44.925890 29656 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:44.930199 29909 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:44.931782 29918 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:09:44.906560 29911 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:44.949148 29796 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:44.987916 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:44.993950 22751 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 13:09:44.994787 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:44.995357 22751 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 15.
I20260709 13:09:44.996424 22751 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:09:44.999930 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.000437 22748 raft_consensus.cc:3060] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:45.001629 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.002200 22753 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:45.006495 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.008757 29824 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.009300 29824 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.011569 29824 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.013753 29995 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:09:45.014853 29995 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:45.015205 29995 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:45.008525 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.016166 29995 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.017820 22749 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:45.023413 29795 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.024370 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:45.025105 22826 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:45.026399 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:44.976927 29950 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:45.027122 22752 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:45.027597 23452 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:09:45.029521 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.031109 29932 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:44.974988 30003 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:45.033401 30003 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.034173 29946 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:45.034590 29946 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:45.034825 29083 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.035599 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.035441 29946 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.036902 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:45.037590 22826 raft_consensus.cc:2368] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22.
I20260709 13:09:45.038858 23416 leader_election.cc:400] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22.
I20260709 13:09:45.039126 23416 leader_election.cc:403] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:09:45.039623 29083 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:45.042153 29835 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:45.042527 29835 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:45.043313 29835 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.044404 29835 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:45.036913 29946 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:45.048684 29083 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22.
I20260709 13:09:45.050302 29792 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.052172 29943 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:45.053300 29878 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:45.053956 29878 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:45.055011 29878 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.056957 29878 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:45.060226 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:45.060695 29911 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.064077 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.065166 29943 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:45.066679 29943 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:45.067534 29859 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.067590 29943 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:45.066740 22900 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:09:45.069721 29859 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.075556 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.077239 30003 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.077373 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:45.077795 30003 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.077908 22900 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:45.078244 30003 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.080112 30003 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.081832 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:45.082495 22826 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:45.082775 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:45.085218 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:45.085548 29950 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:45.087396 29950 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:45.090020 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.090445 29796 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:45.090773 29792 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.090998 29796 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.091274 29792 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.098341 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.093287 29792 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.099175 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.093159 29796 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.118573 22751 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:45.119961 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:45.121224 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:45.121878 22822 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:09:45.121708 22651 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 12 to 15, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 15 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:45.123155 22831 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:09:45.121227 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:45.124236 22825 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:09:45.125267 29836 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:45.125650 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:45.125875 29836 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.126639 22825 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 23: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:45.127482 29792 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.045092 29561 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.128028 29792 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.128788 23450 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:45.129776 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.130383 23449 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:45.131273 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:45.132144 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.101570 29995 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:45.132768 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:45.135932 23447 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 21 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:45.123719 22651 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:45.141559 23447 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:09:45.141829 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:45.142212 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.142594 22827 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:45.147650 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:45.148564 23447 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:45.148950 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.143226 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:45.149711 23446 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:45.150216 22826 raft_consensus.cc:2393] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:45.151664 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.155668 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.156952 22750 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:45.158877 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.159428 22754 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:45.160233 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.161958 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.162734 22826 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:45.163834 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.164080 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.164530 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:45.164831 22901 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:45.165648 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.166194 22747 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:45.170022 22748 raft_consensus.cc:2468] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:45.145153 29828 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.174178 22826 raft_consensus.cc:1240] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:45.174746 29828 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.177450 29828 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.179031 29911 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.180519 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.181001 29836 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.181836 29561 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.182049 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.182785 29948 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.183779 23451 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:45.183919 23445 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:09:45.150700 30003 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.187018 30003 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.187911 29994 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:45.178522 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.137934 22651 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: config changed from index 7 to 11, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65), VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66), VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67), VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 15 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:45.189986 23450 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:45.190538 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.191939 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:45.159085 23446 raft_consensus.cc:2410] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for term 29 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:09:45.164448 22899 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 28.
I20260709 13:09:45.152455 29795 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.196025 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:45.197438 22753 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:45.197659 22826 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:45.198766 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:45.173336 29980 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:45.203011 23447 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:45.233534 22754 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:45.204525 22900 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:45.174117 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.205276 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.209712 29948 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.237722 22897 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:45.249173 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:45.255513 22871 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:45.255705 22756 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:45.256309 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:45.257952 22831 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:45.260357 23449 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
W20260709 13:09:45.261277 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.264802 29909 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.266731 22826 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:45.275763 23448 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:45.173483 29992 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:09:45.215487 29824 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:45.294593 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.292866 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:45.296885 22830 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:45.216323 29980 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.302181 22826 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:09:45.288054 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.312531 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.313220 23452 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:45.294704 23416 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:45.198233 29792 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:45.292135 30007 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:09:45.321233 29083 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.325038 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.354267 29083 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.355403 22756 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:45.295145 29948 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:45.325119 29828 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:09:45.305445 29912 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:45.323483 29923 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:45.219202 29795 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.296690 29824 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.364042 29918 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.319125 29980 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:45.366096 29828 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.324925 29704 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:45.324427 29957 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.361340 29800 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.382030 29795 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.322428 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:45.390038 29824 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:45.359953 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:45.363848 29948 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:45.447515 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 125183us
I20260709 13:09:45.390384 29828 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:09:45.448475 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:45.479463 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.486568 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:45.487061 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:45.450914 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 91062.2us
I20260709 13:09:45.490725 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.392513 29878 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:45.493194 22896 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:45.435595 29946 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.379420 29923 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:45.481088 30009 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:09:45.443750 30003 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.437894 29704 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:45.446869 29943 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.442116 29957 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.529554 22755 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:45.536257 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.503695 29946 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.505882 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.543615 22904 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:45.390493 29918 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:45.504940 29923 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:45.441344 29992 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:45.553617 22753 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:09:45.511337 29083 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.477669 29561 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:45.553030 29857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.041s	sys 0.085s
I20260709 13:09:45.514427 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:45.558257 29857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.041s	sys 0.085s
I20260709 13:09:45.559367 22827 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:09:45.491623 29546 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:45.489944 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:45.541918 22781 heartbeater.cc:646] Failed to heartbeat to 127.21.31.126:35001 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.31.126:35001 timed out after 15.000s (SENT)
W20260709 13:09:45.547144 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 180617us
I20260709 13:09:45.538128 29932 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.562521 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:09:45.551074 22903 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.032s	sys 0.053s
I20260709 13:09:45.501684 29948 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:45.509037 29878 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.565135 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:45.445081 29800 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:45.563318 22903 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.032s	sys 0.053s
W20260709 13:09:45.563705 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73856.9us
I20260709 13:09:45.570884 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.575260 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.562166 29796 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:45.580058 22719 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:45.587966 29860 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:09:45.589743 29903 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:45.590590 29946 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.590713 29957 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.572438 29932 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.573817 29857 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:45.592665 29857 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:45.593091 29857 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:45.593278 29932 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.593966 29857 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.568094 29546 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:45.562458 29911 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 13:09:45.596487 30019 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.596985 30019 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:45.597666 29843 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:45.598201 29843 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:45.599172 30019 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.599026 29843 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.599673 30019 consensus_queue.cc:1059] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:45.600436 30019 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:45.601002 30019 raft_consensus.cc:3055] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:45.602821 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:45.604506 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:45.584435 30016 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:45.605468 29810 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:45.606128 30016 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:45.607164 29810 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:45.607795 29810 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:45.608345 30016 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.608995 29810 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:09:45.589942 30010 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:09:45.609874 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:45.543326 29943 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.612916 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:45.557542 29992 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.596943 29878 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.618213 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:45.624388 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.611600 29810 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.629014 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.650398 23449 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:45.610795 30019 raft_consensus.cc:740] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:45.630782 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.632514 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.652035 23444 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:45.648468 22756 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:45.615523 29903 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:45.653847 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.648586 29792 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:09:45.626362 29821 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.650732 29878 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:45.661543 30019 consensus_queue.cc:260] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:45.662261 29878 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.662751 29878 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:45.651305 29810 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:09:45.664501 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.657370 29860 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.650988 29932 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:09:45.656157 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.626670 30020 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:45.695149 23416 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.658484 30018 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:45.611340 29704 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:45.689538 29835 consensus_meta.cc:287] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.645s	user 0.013s	sys 0.001s
W20260709 13:09:45.709465 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 53361.5us
I20260709 13:09:45.696127 30022 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.687888 30019 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:45.711467 29957 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:45.710139 30018 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:45.723037 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 147895us
I20260709 13:09:45.712121 29950 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:45.717520 29792 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:45.729094 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.690447 29821 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:45.741106 30013 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:45.740821 30027 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:09:45.741673 29810 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.742648 30027 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:45.743793 29810 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.709230 29860 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:45.754190 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 132688us
I20260709 13:09:45.730566 29950 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.756922 29950 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:45.720110 29923 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.773061 29923 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.738780 29792 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:45.773878 29792 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:45.735302 29957 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:45.724077 30022 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.776156 29948 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.777452 29957 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.777578 29943 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.777633 30022 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.778656 29932 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:45.779054 29932 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:45.779443 29932 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:45.758654 29251 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:45.781908 29251 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:45.782696 29251 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.784334 29821 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.754590 29800 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.762703 29992 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.730669 30018 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:45.761430 29824 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.758386 29980 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.770294 29878 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.771258 29860 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.784268 29918 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:45.769692 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.786139 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.755409 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.819869 29909 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:45.821564 29909 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.821741 29912 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:45.821939 29909 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:45.822208 29912 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:45.822239 29824 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.824501 29878 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.825534 29824 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:45.825976 29824 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:45.826339 29824 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:45.832527 22651 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 19 to 21, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 21 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
W20260709 13:09:45.833413 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 78115.7us
I20260709 13:09:45.834484 29980 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.833957 22651 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 7 to 17, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 17 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:45.835330 22651 catalog_manager.cc:5697] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 15 to 16, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 16 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:45.837113 29948 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.837443 22651 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 11 to 12, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 12 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:45.838811 29792 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:45.840435 22747 raft_consensus.cc:2368] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16.
I20260709 13:09:45.840812 29792 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.841481 23451 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:09:45.841800 23451 pending_rounds.cc:85] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 15
I20260709 13:09:45.842028 23451 pending_rounds.cc:107] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:09:45.842221 23451 raft_consensus.cc:2889] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:45.795507 29828 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.844774 23419 leader_election.cc:400] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16.
I20260709 13:09:45.845227 23419 leader_election.cc:403] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:09:45.844691 29824 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.845830 29828 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.846407 29828 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 15. Current term is 16.
I20260709 13:09:45.846227 29909 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.846858 29824 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.847003 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:45.847599 23452 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:45.849608 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.850395 22753 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:09:45.853802 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:45.854497 23452 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:45.854193 29909 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.858418 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.861440 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:45.862133 23451 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:45.870193 29932 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:45.889593 22654 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 340) took 32163 ms (client timeout 15000 ms). Trace:
W20260709 13:09:45.890866 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.839228 22651 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 10 to 14, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 14 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:45.899158 22753 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 29.
I20260709 13:09:45.900851 23419 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.908627 23451 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:45.886888 23452 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:45.884846 30018 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.919435 29950 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.920145 29932 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:45.920955 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 308092us
I20260709 13:09:45.921583 29950 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:45.927238 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.927834 22753 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:45.933477 22756 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 24: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:45.934473 30018 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.916185 29995 raft_consensus.cc:2804] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:09:45.937397 29995 raft_consensus.cc:697] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 29 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:45.938123 29950 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:45.938362 29995 consensus_queue.cc:237] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.938692 29950 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.941293 29950 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:45.941391 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:45.942117 22756 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:09:45.944959 30034 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.945437 30034 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:45.945797 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.946789 22753 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:45.946882 29950 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:45.947455 30034 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:45.950637 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.951037 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:45.952363 22651 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:45.954557 23419 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.955031 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:45.962946 22897 raft_consensus.cc:3060] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:09:45.958204 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.964037 22900 raft_consensus.cc:2368] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18.
I20260709 13:09:45.959292 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.964900 22899 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 15.
I20260709 13:09:45.961133 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.962554 22903 raft_consensus.cc:2368] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19.
I20260709 13:09:45.962234 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:45.977604 22905 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:45.978614 23416 leader_election.cc:400] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19.
I20260709 13:09:45.979108 23416 leader_election.cc:403] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:09:45.966784 22651 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:45.986919 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:45.993935 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:45.994958 22904 raft_consensus.cc:3060] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:45.994725 22651 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 17 to 20, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 20 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:46.004925 22897 raft_consensus.cc:2468] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 29.
I20260709 13:09:45.986299 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:45.961483 29950 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.019881 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.020542 22897 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:09:46.020702 23444 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:45.981472 29948 raft_consensus.cc:2804] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:46.028519 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.032410 22904 raft_consensus.cc:2468] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:46.035568 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:46.036072 23448 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:46.036516 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:46.037403 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.038599 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:46.039510 23447 raft_consensus.cc:2368] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18.
W20260709 13:09:46.040087 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 346066us
I20260709 13:09:46.042299 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.043051 22756 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:46.043541 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.044265 22747 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 16.
I20260709 13:09:46.045627 23446 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 16: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:46.045555 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:45.997207 29828 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19.
I20260709 13:09:46.046216 22747 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:46.050249 23448 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:46.052323 22651 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 17 to 25, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 25 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:46.011116 30020 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:09:46.064437 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 335411us
I20260709 13:09:46.039919 29996 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:46.010339 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:46.058116 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:09:46.058367 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:46.070935 23452 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:46.049230 29946 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.017365 29950 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:46.039800 30018 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.025645 30031 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:46.048934 29662 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:46.071851 22904 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:09:46.073547 22797 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:46.073894 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:46.074334 22871 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:45.997586 29824 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:09:46.095839 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.096673 23452 raft_consensus.cc:2410] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
W20260709 13:09:46.075747 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 65515.4us
I20260709 13:09:46.051548 29948 raft_consensus.cc:697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:46.106153 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:46.107872 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.104761 22868 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:46.076333 22905 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:09:46.046725 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:46.109346 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:45.890658 22654 rpcz_store.cc:269] 0709 13:09:13.725696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.725852 (+   156us) service_pool.cc:224] Handling call
0709 13:09:45.889473 (+32163621us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:41.523548 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:41.787454 (+263906us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:41.795563 (+  8109us) write_op.cc:435] Acquired schema lock
0709 13:09:41.799700 (+  4137us) tablet.cc:662] Decoding operations
0709 13:09:41.923704 (+124004us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:41.937644 (+ 13940us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:41.945081 (+  7437us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:09:41.983846 (+ 38765us) tablet.cc:701] Row locks acquired
0709 13:09:41.983889 (+    43us) write_op.cc:260] PREPARE: finished
0709 13:09:41.984055 (+   166us) write_op.cc:270] Start()
0709 13:09:41.984190 (+   135us) write_op.cc:276] Timestamp: P: 1783602581983996 usec, L: 0
0709 13:09:41.984207 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:09:41.984839 (+   632us) log.cc:844] Serialized 1298 byte log entry
0709 13:09:42.657746 (+672907us) op_driver.cc:464] REPLICATION: finished
0709 13:09:42.781906 (+124160us) write_op.cc:301] APPLY: starting
0709 13:09:42.782012 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:42.785780 (+  3768us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:42.785910 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:42.945197 (+159287us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:42.989542 (+ 44345us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:09:43.022931 (+ 33389us) write_op.cc:312] APPLY: finished
0709 13:09:43.051118 (+ 28187us) log.cc:844] Serialized 41 byte log entry
0709 13:09:43.051375 (+   257us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:43.051492 (+   117us) write_op.cc:454] Released schema lock
0709 13:09:43.051558 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:43.052076 (+   518us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:43.052123 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":224734,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":120468,"apply.run_cpu_time_us":7100,"apply.run_wall_time_us":270103,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16667,"num_ops":3,"prepare.queue_time_us":123592,"prepare.run_cpu_time_us":24426,"prepare.run_wall_time_us":797532,"raft.queue_time_us":337262,"raft.run_cpu_time_us":25455,"raft.run_wall_time_us":115292,"replication_time_us":655662,"thread_start_us":493964,"threads_started":4,"wal-append.queue_time_us":239403,"wal-append.run_cpu_time_us":30070,"wal-append.run_wall_time_us":1830796}]]}
I20260709 13:09:46.080978 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:46.082154 23450 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 24: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:46.082717 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:46.083458 22651 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 18 to 21, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 21 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:46.115243 23447 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:46.115700 22651 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 14 to 17, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 17 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:46.084784 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48018.9us
I20260709 13:09:46.086753 22902 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:09:46.086833 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.118017 22719 leader_election.cc:400] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18.
I20260709 13:09:46.118592 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:46.089098 30031 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.094112 30033 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:46.105286 29860 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:46.122323 22651 catalog_manager.cc:5697] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:46.076550 29977 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:09:46.121443 22794 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:46.121819 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.073592 29912 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.106271 29836 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:46.125738 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:46.113456 29810 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:46.089429 30018 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.086175 29946 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.134799 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.153968 22826 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:46.135673 23447 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:46.136721 22827 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:09:46.153039 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.140031 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.140094 22753 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:46.155668 22824 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:46.141426 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.141687 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.156267 22822 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:46.156706 22823 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 23: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:46.149856 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.149911 22756 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:09:46.156263 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.159480 22753 raft_consensus.cc:2410] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:09:46.150108 22651 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:46.157768 22828 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:46.155504 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.120008 29725 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:46.162161 22754 raft_consensus.cc:2410] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 13:09:46.117005 29887 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.014s	sys 0.015s
I20260709 13:09:46.171578 22651 catalog_manager.cc:5697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 9 to 17, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } }
I20260709 13:09:46.175590 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:46.175436 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 49768us
I20260709 13:09:46.175619 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:46.123270 30031 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:46.096083 29662 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.186017 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.234537 22755 raft_consensus.cc:1275] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:09:46.234462 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:46.129009 30039 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:09:46.236310 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60758.9us
I20260709 13:09:46.130786 30022 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:09:46.237382 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56509.8us
I20260709 13:09:46.238750 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1724) took 1078 ms. Trace:
I20260709 13:09:46.240069 22719 leader_election.cc:400] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18.
I20260709 13:09:46.245963 22756 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:46.246203 22719 leader_election.cc:403] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:09:46.247860 22824 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
W20260709 13:09:46.181059 29887 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.014s	sys 0.015s
I20260709 13:09:46.262951 22719 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:46.262786 22651 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:46.246150 22898 rpcz_store.cc:276] 0709 13:09:45.160316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.160425 (+   109us) service_pool.cc:224] Handling call
0709 13:09:45.160694 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:45.196293 (+ 35599us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:09:45.239052 (+ 42759us) log.cc:844] Serialized 14 byte log entry
0709 13:09:45.429240 (+190188us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b18000c4000. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cf0e07 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b23a 00007fcb74e05a2a
0709 13:09:45.770993 (+341753us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b18000c4000. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb787f33c7 00007fcb74e059a7 00007fcb74e150bf 00007fcb74e15038
0709 13:09:46.016472 (+245479us) log.cc:844] Serialized 14 byte log entry
0709 13:09:46.016686 (+   214us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:46.017117 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:09:46.018359 (+  1242us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:46.018394 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:46.018461 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:46.238573 (+220112us) raft_consensus.cc:1685] finished
0709 13:09:46.238592 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:46.238695 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46690,"prepare.queue_time_us":247112,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":46480,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":40769,"spinlock_wait_cycles":22887552,"thread_start_us":360009,"threads_started":2,"wal-append.queue_time_us":412624}
I20260709 13:09:46.131958 30033 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:46.128224 29251 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:46.129531 29836 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.169643 29810 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:46.241675 30006 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:09:46.232004 29725 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:46.285127 22651 catalog_manager.cc:5697] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 13 to 15, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 15 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:46.243381 30039 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.302565 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.250438 29546 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:46.303138 22824 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:46.308882 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.310119 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.311619 22824 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:09:46.271874 29992 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:46.303730 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.280516 30022 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:46.250546 29980 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:46.305416 22651 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 13 to 15, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 15 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:46.238955 29932 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:09:46.330497 29996 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:46.331005 29996 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:46.279608 29810 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.317616 30033 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.263754 29824 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:46.270370 30047 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:09:46.307475 30039 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:46.319983 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.241492 30037 raft_consensus.cc:2764] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:09:46.283962 29835 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.319446 29546 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:46.325547 22747 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:46.293875 29251 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:46.297158 29859 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:46.328866 30041 log.cc:927] Time spent T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
W20260709 13:09:46.340147 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 163042us
I20260709 13:09:46.340237 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.378424 22827 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:46.341081 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.284333 29836 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:46.348927 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.271274 30040 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
W20260709 13:09:46.389809 30046 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:09:46.351224 22651 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:46.351742 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:46.352326 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.353842 22823 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:46.160655 30020 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.327857 29980 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:46.401031 29980 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:46.241964 29948 consensus_queue.cc:237] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.396889 22651 catalog_manager.cc:5697] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:46.397746 30050 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:09:46.397624 29266 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:09:46.398281 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46607.4us
I20260709 13:09:46.421926 29946 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:46.422713 29662 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.423447 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:46.375391 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:46.424497 29662 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:09:46.424947 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49650.6us
I20260709 13:09:46.426004 29912 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.426402 30037 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:46.426785 29251 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.427987 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.428568 22828 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:46.429800 22797 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:46.430213 29751 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:46.430761 29857 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:46.407341 30022 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.425179 30026 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.007s
I20260709 13:09:46.377815 30032 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:09:46.370903 29878 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18.
I20260709 13:09:46.376382 29918 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.435081 30018 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.434962 29918 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.436925 29918 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.348673 29824 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.437685 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.438853 29824 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:46.407130 30040 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:46.438997 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.439769 22826 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:46.441434 30056 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:09:46.442036 30030 log.cc:927] Time spent T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.377s	user 0.003s	sys 0.000s
I20260709 13:09:46.442585 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.443223 22748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 2185) took 1270 ms. Trace:
I20260709 13:09:46.433899 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.443598 22748 rpcz_store.cc:276] 0709 13:09:45.172967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.218691 (+ 45724us) service_pool.cc:224] Handling call
0709 13:09:45.219012 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:45.555857 (+336845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:45.555885 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:45.566690 (+ 10805us) log.cc:844] Serialized 26 byte log entry
0709 13:09:45.792077 (+225387us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:45.792140 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.792281 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:46.441507 (+649226us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:46.442947 (+  1440us) raft_consensus.cc:1685] finished
0709 13:09:46.442967 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:46.443167 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16975,"raft.queue_time_us":44841,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5487,"spinlock_wait_cycles":6848256,"thread_start_us":490411,"threads_started":2,"wal-append.queue_time_us":441351}
I20260709 13:09:46.439950 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.444895 29546 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.446076 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.446987 22823 raft_consensus.cc:2368] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19.
I20260709 13:09:46.448176 22823 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:46.419710 22651 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 15 to 17, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:46.449230 22822 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:46.450191 22651 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 14 to 15, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 15 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:46.418686 29836 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.444749 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.456207 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.377302 29835 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.452385 22651 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 17 to 18, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 18 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:46.459595 29835 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.380415 29725 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.460528 29704 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.461012 29704 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.383462 29859 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.464218 29836 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:46.465610 29859 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.466741 23452 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:46.472422 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.474851 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:46.475833 22756 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:46.477869 22827 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:46.480979 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:46.483464 22822 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 24.
I20260709 13:09:46.489445 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.490451 22905 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:46.493454 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:46.494179 22748 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:09:46.494164 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.447037 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.498513 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.498607 29992 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:46.500042 29992 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.502170 22748 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:46.503350 22904 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:46.505997 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:46.462963 29704 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:46.507470 23446 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:09:46.509812 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.509999 30066 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:09:46.515561 22794 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 84ea43361e534fbebad704327c9f6367. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:46.522759 30039 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.524559 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:46.525384 22752 raft_consensus.cc:1240] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:09:46.525815 22828 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:09:46.529462 30020 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.529608 30066 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.529976 30020 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.530313 29878 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.530896 29878 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.532168 30020 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.532649 30020 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.532766 29878 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:46.534276 22871 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:46.533107 30020 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.536600 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.537693 22899 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:09:46.540048 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.540601 23450 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:46.530105 30066 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.544811 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.546388 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.546407 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.547892 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.548426 22827 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:46.510326 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:46.550884 22750 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 24: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:46.556699 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.568508 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.569113 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:46.569494 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88575.7us
I20260709 13:09:46.570185 29083 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:46.608016 29083 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.608443 29083 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:46.539683 29911 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.524820 30039 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:46.535488 30033 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:46.524883 29950 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.596706 22827 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:46.601794 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:46.608943 22752 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:09:46.622645 22752 pending_rounds.cc:85] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 14
I20260709 13:09:46.622942 22752 pending_rounds.cc:107] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:09:46.622258 29083 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.610591 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:46.610642 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.623687 22898 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:46.610488 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:46.529187 29980 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.617980 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:46.557955 30018 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:46.557762 29923 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:46.623193 22752 raft_consensus.cc:2889] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:46.607434 29878 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.639030 29083 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:46.643575 22898 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:46.558157 29828 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:09:46.608251 30016 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:46.608218 29937 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:46.657780 29083 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:46.658706 23453 raft_consensus.cc:1240] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:46.664152 30064 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:46.664247 30018 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.664836 30064 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.664919 30018 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.666531 29980 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.667061 30064 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.667495 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:46.667666 30018 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:46.668970 30068 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:09:46.669471 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 54752.8us
I20260709 13:09:46.673388 29859 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:46.675984 30027 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:46.678068 30026 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:46.680079 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:46.680513 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.681475 23444 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 24: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:46.681934 30002 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:09:46.682247 30048 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.357s	user 0.000s	sys 0.004s
I20260709 13:09:46.682395 30002 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.682792 30002 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:46.575690 29981 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:09:46.686602 29981 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:46.686936 29981 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:46.687755 29950 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:46.687709 29981 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.688982 29981 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:46.640609 29932 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:46.690605 29932 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.690951 29932 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:46.622357 29911 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.694170 29911 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.694209 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.695645 22719 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:46.697186 30031 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.698218 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.699174 30031 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.700011 29980 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:46.635524 30054 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.700515 29980 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:46.700871 30054 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.702912 30054 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:46.703235 29980 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.646773 30033 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:46.680799 23447 raft_consensus.cc:2368] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:46.668843 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:46.712244 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.712821 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.713600 23444 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:46.651703 29835 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:46.705566 30033 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.661772 29878 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.726220 29266 consensus_queue.cc:1059] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:46.727980 22719 leader_election.cc:400] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:46.728363 22719 leader_election.cc:403] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:09:46.729404 30018 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:46.662407 30039 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:46.733718 30039 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:46.734150 30066 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.735647 30039 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:46.736158 30069 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:09:46.737051 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.740073 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:46.740742 22752 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:09:46.742055 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:46.754287 22897 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 14.
I20260709 13:09:46.754886 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.770069 29266 raft_consensus.cc:3055] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:46.770546 29266 raft_consensus.cc:740] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:46.771260 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 197743us
I20260709 13:09:46.773851 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:46.774407 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:46.774928 23445 raft_consensus.cc:3060] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:46.774993 22750 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 13:09:46.778731 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:46.780026 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.727046 30038 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:46.780568 23447 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:46.771502 29266 consensus_queue.cc:260] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.662743 29937 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.784595 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.719163 29923 consensus_queue.cc:1048] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:09:46.722579 29662 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:09:46.786552 22749 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 24: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:46.789410 23445 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:46.787781 23444 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:09:46.796695 22752 raft_consensus.cc:2468] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
W20260709 13:09:46.799093 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 72316.9us
I20260709 13:09:46.788632 30020 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:46.802831 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:46.803990 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:46.805430 22868 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:46.812733 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:46.815224 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:46.820000 22826 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:46.820572 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.821715 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.822886 22751 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:09:46.823702 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.823769 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.824239 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:46.824728 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.825134 22824 raft_consensus.cc:2393] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:46.826588 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.826995 30020 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.827606 30020 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:46.828167 29992 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.828687 29992 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.830119 30020 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:46.830658 29992 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.834503 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.817800 30031 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:46.840049 30031 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:46.840521 30031 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:46.845209 30043 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.518s	user 0.000s	sys 0.004s
I20260709 13:09:46.833777 30054 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:09:46.861783 30090 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:09:46.869205 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.832710 29792 raft_consensus.cc:2804] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:09:46.874006 29996 consensus_meta.cc:287] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.543s	user 0.001s	sys 0.010s
I20260709 13:09:46.875005 22754 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:09:46.804930 29918 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:09:46.892313 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 116512us
I20260709 13:09:46.873113 29792 raft_consensus.cc:697] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:46.896097 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:46.799049 30076 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.088s
I20260709 13:09:46.905952 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:46.874511 29996 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.887051 30045 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.100s
I20260709 13:09:46.903258 29932 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:46.957952 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 61942.7us
I20260709 13:09:46.958578 29932 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:46.959534 22755 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:09:46.960896 22751 raft_consensus.cc:2955] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:09:46.962466 22749 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:46.963172 22824 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:46.963451 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.964493 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:46.705181 29912 raft_consensus.cc:2955] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 12 to 15, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:46.967852 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:46.944619 30056 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:46.946544 30018 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 18. Current term is 19.
I20260709 13:09:46.916898 29857 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:09:46.928058 29546 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 13:09:46.948917 30057 raft_consensus.cc:2955] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:09:46.951035 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45185.4us
I20260709 13:09:46.951512 29824 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:47.007349 23452 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:47.007540 22830 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:09:46.952111 29918 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:46.952276 30002 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.935770 29792 consensus_queue.cc:237] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:46.954397 30076 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.972774 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:46.974247 23445 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:09:46.974576 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 349513us
I20260709 13:09:46.971010 22898 raft_consensus.cc:2955] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 12 to 15, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:46.969637 23444 raft_consensus.cc:2955] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 12 to 15, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:46.996603 22901 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:47.020191 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:46.992580 22748 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:47.003504 23447 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:09:46.967370 29948 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:09:47.014253 23446 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:47.021240 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:47.013841 29725 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.017009 29836 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.023324 30076 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.045471 29725 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.046031 30076 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:47.047936 29725 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.054025 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:47.055145 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:47.055287 23419 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:47.056787 29859 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:47.057477 29859 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:47.057821 29859 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.058032 29859 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:47.024783 30003 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:47.086865 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 57128.7us
I20260709 13:09:47.089788 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:47.097380 29950 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:47.087996 29909 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:47.099860 29909 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:47.100246 29909 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:47.101656 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:47.102304 23445 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 18.
I20260709 13:09:47.090507 22830 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:09:47.105396 23444 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:47.074293 29836 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.030494 30064 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:09:47.107780 29836 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.025713 29821 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.085s
I20260709 13:09:47.027467 30031 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.112233 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:47.112010 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:47.126767 22830 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:47.121935 30002 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:47.090735 30026 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:09:47.113200 29843 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:09:47.135344 30031 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.136032 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:47.115408 30003 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.113651 29796 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:47.138398 29796 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.138887 30003 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:47.140254 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 334328us
I20260709 13:09:47.140486 29796 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.109504 30098 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:47.141739 23450 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:09:47.066018 30054 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:09:47.142323 22748 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:09:47.146517 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:47.146245 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:47.129103 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:47.129673 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:47.130460 22827 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:09:47.137964 23445 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:09:47.128880 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:47.152601 22822 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:47.150786 23446 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:47.154614 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:47.157626 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:47.158836 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:47.159569 22751 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:47.164566 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:47.166145 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:47.141068 29859 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:47.190852 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44689.7us
I20260709 13:09:47.196046 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:47.191488 22749 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 24: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:47.191836 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48059.1us
I20260709 13:09:47.200342 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:47.201769 22899 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:09:47.202105 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:47.203397 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:47.204089 22896 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:47.114646 29937 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:47.125298 29950 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:47.218482 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:47.219236 22904 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 24.
I20260709 13:09:47.226871 22899 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:47.229746 22904 raft_consensus.cc:1275] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 13:09:47.233366 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:47.277912 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.278934 23452 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
W20260709 13:09:47.279911 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 258711us
I20260709 13:09:47.282836 23416 leader_election.cc:400] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 18. Current term is 19.
I20260709 13:09:47.283463 22828 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:47.286481 23446 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:47.287818 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.295071 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.206446 30040 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:47.195350 30031 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:47.210003 30033 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.210258 29932 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:47.294087 29836 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:47.306936 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.318460 29996 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.312248 30040 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:47.315860 30033 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.319527 29950 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.346164 29836 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.282173 29980 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:09:47.356441 30068 log.cc:927] Time spent T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.054s	user 0.001s	sys 0.001s
I20260709 13:09:47.312481 29948 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:47.302568 30018 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:47.294302 30039 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:09:47.346107 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 148361us
I20260709 13:09:47.296814 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:47.401130 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 104361us
I20260709 13:09:47.401945 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:47.354928 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48068.2us
I20260709 13:09:47.283327 29937 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:09:47.373945 30053 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:47.344696 29859 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.400555 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:47.311690 30002 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 18.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.083s
I20260709 13:09:47.408591 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.368202 30085 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:47.388831 29836 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:47.311894 30031 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:47.419392 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:47.419610 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:47.395612 30018 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:47.421788 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:47.407287 30037 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:09:47.425243 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:47.434037 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 206108us
I20260709 13:09:47.452327 22901 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:47.420637 22903 raft_consensus.cc:2955] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:09:47.415169 29923 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:47.411962 29932 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:47.457942 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49416.1us
I20260709 13:09:47.461233 22871 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:47.490327 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 55473.3us
W20260709 13:09:47.472662 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 61034.1us
I20260709 13:09:47.492527 23416 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.495611 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:47.497639 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:47.500236 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:47.500170 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:47.426414 30018 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:47.501466 22896 raft_consensus.cc:2410] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:09:47.457079 30085 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:47.417109 30108 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.096s
I20260709 13:09:47.535392 30057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:47094 (request call id 20) took 2655 ms. Trace:
I20260709 13:09:47.502511 30002 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:47.499140 30098 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.554013 30057 rpcz_store.cc:276] 0709 13:09:44.879804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.879943 (+   139us) service_pool.cc:224] Handling call
0709 13:09:44.944400 (+ 64457us) log.cc:844] Serialized 499 byte log entry
0709 13:09:45.213580 (+269180us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b18003c8ec0. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71c88c0a 00007fcb7b3c8219 00007fcb71d10a40 00007fcb71d0b4e4 00007fcb71d0b455 00007fcb74d54b0b 00007fcb74dd9b63 00007fcb74e01752 00007fcb79dc1d29
0709 13:09:45.606024 (+392444us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b54003ba220. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d7bb26 00007fcb74d7549a 00007fcb74d74714 00007fcb74d54eaa 00007fcb74d5bd53 00007fcb74d5bcd9 00007fcb74d5bc69 00007fcb74d5bc31 00007fcb74d5adfd 000000000038f354
0709 13:09:45.780176 (+174152us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b4c0044d680. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb730c7a20 00007fcb730d8a29 00007fcb730d913f 00007fcb7309d2ab 00007fcb730caf3c 00007fcb730cb3fe 00007fcb7335850b 00007fcb74d57381 00007fcb74d5526b 00007fcb74d5bd53 00007fcb74d5bcd9
0709 13:09:47.535002 (+1754826us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65697,"raft.queue_time_us":243050,"raft.run_cpu_time_us":6518,"raft.run_wall_time_us":731977,"spinlock_wait_cycles":36820608,"thread_start_us":169981,"threads_started":1,"wal-append.queue_time_us":380675,"wal-append.run_cpu_time_us":2538,"wal-append.run_wall_time_us":1182523}
I20260709 13:09:47.569763 23453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 1899) took 1502 ms. Trace:
I20260709 13:09:47.616608 23453 rpcz_store.cc:276] 0709 13:09:46.066931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:46.067053 (+   122us) service_pool.cc:224] Handling call
0709 13:09:46.067283 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:46.662879 (+595596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.569424 (+906545us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b5400730354. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:47.569680 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1359550336}
I20260709 13:09:47.586053 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:47.499137 30039 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 13:09:47.451973 30040 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:47.636242 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 131201us
W20260709 13:09:47.640115 22830 consensus_meta.cc:287] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.001s
I20260709 13:09:47.640569 22830 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:47.523324 30003 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.105s
I20260709 13:09:47.527032 29909 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.665012 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:47.678391 29995 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:47.621711 30098 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.683584 29995 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:47.680425 29912 consensus_meta.cc:287] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.000s
I20260709 13:09:47.672783 30037 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:47.662271 30002 consensus_queue.cc:1048] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 13:09:47.700353 29995 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.705297 22903 raft_consensus.cc:1275] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:47.707046 29994 consensus_queue.cc:1048] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:47.708896 29995 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:47.711552 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:47.694707 30033 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.712965 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.719550 29995 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:47.717578 30031 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:47.733494 30128 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:47.735603 29995 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:47.789798 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:47.797451 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:47.799870 22830 raft_consensus.cc:2410] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:09:47.803047 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:47.805725 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:47.808301 29704 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:47.808845 29704 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:47.809648 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:47.811071 29266 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:47.812347 30128 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.812813 30128 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.733603 30036 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:09:47.819007 22826 raft_consensus.cc:2410] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:09:47.819682 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:47.820371 22829 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:47.821691 29909 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:47.823398 30098 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:47.827067 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:47.828372 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:47.735569 30037 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.833150 22748 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:09:47.829156 22753 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:47.839529 29932 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:47.840556 29932 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.840958 29932 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:47.829252 29995 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.846949 29995 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:47.847507 30128 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.847675 29878 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:47.839704 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 107876us
I20260709 13:09:47.850937 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:47.837584 22797 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:47.862844 22753 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:09:47.863411 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:47.864157 23449 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 24.
I20260709 13:09:47.866897 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:47.870005 23453 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:47.878515 30045 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:47.858935 29878 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:09:47.881832 29266 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:47.883674 22826 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2371) took 1062 ms. Trace:
I20260709 13:09:47.884011 22826 rpcz_store.cc:276] 0709 13:09:46.821274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:46.821461 (+   187us) service_pool.cc:224] Handling call
0709 13:09:47.883478 (+1062017us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b54000ac8d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:47.883590 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2291275008}
I20260709 13:09:47.885864 30036 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:47.886226 30036 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:47.886425 30034 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:47.886999 30036 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.874346 29932 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.888334 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:47.889040 30085 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:47.889505 30085 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:47.890309 29932 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.891234 30034 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:47.890326 30085 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.891705 30034 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:47.859113 30018 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.892194 30034 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:47.893759 30018 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.896981 30044 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:47.898729 30085 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:47.902863 30122 log.cc:927] Time spent T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.159s	user 0.001s	sys 0.001s
I20260709 13:09:47.869717 30127 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:47.904383 30127 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.904713 30127 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:47.920572 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:47.873821 30117 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:47.941809 22751 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:09:47.829653 29704 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:47.949752 22897 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:09:47.922852 30054 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
W20260709 13:09:47.983832 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 107413us
W20260709 13:09:47.966027 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 95005.9us
I20260709 13:09:47.946004 29828 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:47.987716 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.991393 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.993537 29083 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:47.992702 23416 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:47.995611 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:47.978716 29909 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:47.985271 29704 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:47.967329 30115 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 13:09:48.002959 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 126014us
I20260709 13:09:48.004608 29083 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:48.004724 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:48.002081 30040 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 13:09:48.011886 29980 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:48.020555 29980 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.020910 29980 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:48.021577 30127 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.023663 30127 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.003816 30128 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.023872 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:48.001775 30026 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.024346 30128 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:47.999910 29836 consensus_meta.cc:287] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.000s
I20260709 13:09:48.025046 30026 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.025796 29836 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.027688 30026 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:47.992486 30136 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:48.017866 29083 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:47.991514 29828 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.028818 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:48.027746 29836 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.030200 29828 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:48.030434 22904 raft_consensus.cc:2368] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:48.015460 29909 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.016552 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:48.033663 30127 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:48.034015 30121 log.cc:927] Time spent T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.488s	user 0.001s	sys 0.007s
I20260709 13:09:48.034377 30127 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.036755 30034 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:48.037145 30127 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.039058 30034 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:48.042692 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:48.042802 30128 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.045678 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:48.053903 22755 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:48.067864 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:48.067171 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:48.054814 22748 raft_consensus.cc:1240] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:48.066555 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:48.079155 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:48.079828 23449 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:48.080888 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:48.081501 22898 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:48.049899 22753 raft_consensus.cc:2955] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 12 to 15, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:48.082607 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:48.090458 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:48.093206 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.094635 22826 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:09:48.095006 22826 pending_rounds.cc:85] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 7
I20260709 13:09:48.095383 22826 pending_rounds.cc:107] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:09:48.095568 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:48.095800 22826 raft_consensus.cc:2889] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:48.094827 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:48.097397 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:48.098697 22826 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:09:48.100354 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:48.103000 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:48.103663 22826 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:09:48.072546 30034 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:48.107321 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:48.076179 30039 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:48.110239 22829 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:48.098773 30128 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.118853 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:48.114576 22898 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 25.
I20260709 13:09:48.076524 30016 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:09:48.093581 30147 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:48.135772 22826 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:48.138916 23449 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 25.
I20260709 13:09:48.140767 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:48.141535 30127 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:48.142359 30127 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:48.143384 30127 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.143157 30037 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:48.130501 30147 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.114130 30034 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:48.186298 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66798us
W20260709 13:09:48.238948 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:48.259071 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.130034 30054 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:09:48.154942 29828 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:48.126889 30128 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:48.121130 30039 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.453388 30136 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.454111 30039 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.453881 30136 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.455152 30039 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:48.455932 30136 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.448683 30161 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.178717 30147 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:48.456984 30161 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.457263 29828 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.457792 29948 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.458355 29948 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.458921 30161 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.459401 29828 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:48.134445 30040 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:09:48.457401 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:48.461534 30132 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.274s	user 0.000s	sys 0.000s
I20260709 13:09:48.455634 30039 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.465008 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:48.462404 29948 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.474836 29909 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:48.498974 30037 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:48.501235 30128 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:48.503494 30153 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.367s	user 0.001s	sys 0.000s
I20260709 13:09:48.515450 29909 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.527907 29909 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:48.529980 30156 log.cc:927] Time spent T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:09:48.501055 29980 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:48.515597 30037 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.576416 30054 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:48.580005 30054 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.576807 30039 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:48.573745 30149 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.000s	sys 0.009s
I20260709 13:09:48.582044 30037 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:48.586043 30149 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.001s	sys 0.009s
I20260709 13:09:48.589841 30147 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.591814 30147 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:48.630944 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 331368us
W20260709 13:09:48.643973 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.655114 22828 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3309) took 1561 ms. Trace:
I20260709 13:09:48.655648 22828 rpcz_store.cc:276] 0709 13:09:47.093737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.094104 (+   367us) service_pool.cc:224] Handling call
0709 13:09:47.113928 (+ 19824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:47.283937 (+170009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:48.654551 (+1370614us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54002f0dd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:48.654934 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389344512}
I20260709 13:09:48.667786 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:09:48.678893 22904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43998 (request call id 3085) took 1259 ms (client timeout 1609 ms). Trace:
W20260709 13:09:48.679386 22904 rpcz_store.cc:269] 0709 13:09:47.419277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.419400 (+   123us) service_pool.cc:224] Handling call
0709 13:09:48.678460 (+1259060us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b54003ea154. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:48.678698 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1401312640}
I20260709 13:09:48.682040 22901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2263) took 1482 ms. Trace:
I20260709 13:09:48.682462 22901 rpcz_store.cc:276] 0709 13:09:47.199990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.200102 (+   112us) service_pool.cc:224] Handling call
0709 13:09:48.681597 (+1481495us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540005a5d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:48.681861 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578107264}
W20260709 13:09:48.690315 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.700848 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:48.702399 22753 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:48.704022 22756 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:09:48.707619 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:48.709242 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:48.710997 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:48.713270 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.714148 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:48.715408 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:48.718659 22753 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:48.723901 29266 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:48.725965 29704 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:48.733355 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.733469 29704 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.741459 29704 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.742213 29704 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.743651 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:48.743566 29704 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.744716 22896 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
I20260709 13:09:48.746572 29796 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.747525 29796 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.749555 29810 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:48.750407 29810 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:48.750712 29796 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.751542 29796 consensus_queue.cc:1059] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:48.752532 29796 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.752947 29810 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:48.753104 29796 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.754374 29828 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:48.755506 22753 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:09:48.755139 29828 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.757962 29828 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.758569 29828 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.760938 29843 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.761493 29828 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.762063 29843 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.760927 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:48.765841 29843 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.767325 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:48.768603 29857 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.770849 29857 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:48.772740 29878 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.774048 29857 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.775646 29828 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.769446 29887 raft_consensus.cc:3055] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:48.776439 29887 raft_consensus.cc:740] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:48.777814 29887 consensus_queue.cc:260] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.779770 29887 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:48.776263 29878 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.782373 29878 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.783813 29878 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:48.784358 29828 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.784370 29878 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:48.789251 29878 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.791246 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:48.578569 29980 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.801939 22753 raft_consensus.cc:2410] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:09:48.792769 29903 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:48.803625 29903 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:48.791863 29909 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.785892 29796 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:48.793682 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:48.787103 29828 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.797104 29911 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:09:48.800720 29937 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.005s	sys 0.001s
I20260709 13:09:48.800817 29948 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.793012 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:48.823271 30175 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.806489 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:48.839424 22794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.001s	sys 0.006s
I20260709 13:09:48.841713 29704 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:48.842079 29903 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:48.867405 30076 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:48.874070 29828 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:48.850380 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:48.874517 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:48.844105 29937 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.006s	sys 0.002s
I20260709 13:09:48.847865 22753 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
W20260709 13:09:48.903867 22794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.001s	sys 0.006s
I20260709 13:09:48.868754 29911 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:48.886354 30076 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.873873 30170 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.914460 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:48.915551 30076 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:48.858666 29909 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:48.918443 30157 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.442s	user 0.000s	sys 0.001s
I20260709 13:09:48.851760 30037 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:48.868975 30180 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:48.917095 30152 log.cc:927] Time spent T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.803s	user 0.002s	sys 0.000s
W20260709 13:09:48.917037 22719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.009s	sys 0.005s
I20260709 13:09:48.858902 29704 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:48.872419 30136 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.888430 29878 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.868094 30034 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:48.919795 30111 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 1.369s	user 0.014s	sys 0.000s
I20260709 13:09:48.851222 30039 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.912139 29911 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:48.937640 22719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.009s	sys 0.005s
I20260709 13:09:48.919021 29909 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.842995 29948 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.944191 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:48.914445 30170 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:48.959566 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.940465 30039 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.943883 29878 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:48.949069 29704 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.972738 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:48.978896 22869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.004s	sys 0.044s
W20260709 13:09:48.979553 30115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.009s
I20260709 13:09:48.960678 30036 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:48.965580 30076 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.933408 30175 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:48.985446 30173 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.959893 30136 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.950395 30037 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:48.997777 30173 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.000111 30037 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.000208 30173 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:49.000645 30158 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.841s	user 0.000s	sys 0.002s
I20260709 13:09:48.985298 29911 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:48.959261 30180 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.002154 29911 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:48.984428 30115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.008s	sys 0.009s
I20260709 13:09:49.017197 29948 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.020385 30117 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:49.025906 30117 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:49.030241 29996 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.436s	user 0.009s	sys 0.019s
I20260709 13:09:49.033583 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 3308) took 1923 ms. Trace:
I20260709 13:09:49.036358 30053 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:49.037078 30039 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:49.034196 29909 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.034703 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:49.037726 29996 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.507s	user 0.010s	sys 0.019s
W20260709 13:09:49.079903 30127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.011s	sys 0.003s
I20260709 13:09:49.078878 30115 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:49.094121 30076 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.110505 30115 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:49.111605 30054 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:49.112470 30076 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:49.113071 30115 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:49.113487 30136 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.115017 22900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2270) took 1101 ms. Trace:
I20260709 13:09:49.037436 30053 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:49.117769 23452 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 18: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:09:49.116905 30127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.011s	sys 0.008s
I20260709 13:09:49.132332 30182 raft_consensus.cc:2764] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader election decision vote started in defunct term 24: won
I20260709 13:09:49.132856 30180 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.133258 30126 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.115473 22900 rpcz_store.cc:276] 0709 13:09:48.013687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:48.013885 (+   198us) service_pool.cc:224] Handling call
0709 13:09:48.014250 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:48.014676 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:09:48.014711 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:48.015201 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:09:48.075972 (+ 60771us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:48.076019 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:48.088754 (+ 12735us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:48.999485 (+910731us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:49.114438 (+114953us) raft_consensus.cc:1685] finished
0709 13:09:49.114459 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:49.114770 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1676,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":1485,"spinlock_wait_cycles":703838592,"thread_start_us":34167,"threads_started":1,"wal-append.queue_time_us":63030}
I20260709 13:09:49.036029 22825 rpcz_store.cc:276] 0709 13:09:47.092278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.092442 (+   164us) service_pool.cc:224] Handling call
0709 13:09:47.092780 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:47.121470 (+ 28690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:47.121515 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:47.122059 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 13:09:47.292323 (+170264us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:47.292359 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.292419 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:48.884309 (+1591890us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:49.002614 (+118305us) raft_consensus.cc:1685] finished
0709 13:09:49.002649 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:49.015766 (+ 13117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20757,"raft.queue_time_us":22808,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":67,"spinlock_wait_cycles":59897728,"thread_start_us":169958,"threads_started":1,"wal-append.queue_time_us":329752}
I20260709 13:09:49.145545 29878 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.146625 30170 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.151969 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:49.153359 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 93191.8us
I20260709 13:09:49.154464 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:49.158510 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 213726us
I20260709 13:09:49.160756 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:49.162585 29704 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.171464 23449 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:09:49.174609 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:49.175287 23452 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:49.177759 30175 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:49.179921 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:49.180692 22748 raft_consensus.cc:2368] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21.
I20260709 13:09:49.180262 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.189095 30126 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:49.190603 30076 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:49.191757 30126 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.195315 23419 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:49.196369 30098 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:49.198329 29909 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:49.203166 30180 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:49.204288 23451 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:09:49.203948 30180 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.205500 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.206192 23444 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:49.210819 30180 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:49.209131 29909 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:49.211776 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.212162 29909 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:49.212602 23449 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:49.214648 29909 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.154492 30053 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:49.220058 29909 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:49.223232 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.223611 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.223976 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:49.224140 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63519.2us
I20260709 13:09:49.225853 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:49.226352 29824 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:49.241621 22825 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:09:49.242623 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.243754 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:49.282819 22828 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:49.244060 29860 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.244339 30027 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:09:49.247233 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.247184 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.247990 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.253882 30178 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:49.255139 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.247026 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:49.256489 30117 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.257608 22827 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:09:49.259061 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.286477 22823 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:49.290477 22822 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 17: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:49.291501 23452 raft_consensus.cc:2410] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:09:49.289986 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.296058 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:49.297574 29859 consensus_queue.cc:1059] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:09:49.298626 30113 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:49.288674 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.300630 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:49.289628 22827 pending_rounds.cc:85] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 13
I20260709 13:09:49.297256 22794 leader_election.cc:400] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:49.309011 22794 leader_election.cc:403] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:49.325834 22754 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:49.335208 30117 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:49.335952 29860 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:49.336915 30018 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:49.337478 30018 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.338136 29860 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:49.340201 30018 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:49.342296 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.344961 29859 raft_consensus.cc:3055] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:49.345396 29859 raft_consensus.cc:740] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:49.346267 29859 consensus_queue.cc:260] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 24.20, Last appended by leader: 20, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.301849 22827 pending_rounds.cc:107] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:09:49.349364 22827 raft_consensus.cc:2889] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:49.350049 29946 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.350729 29946 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.302306 29824 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:09:49.355051 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 159830us
I20260709 13:09:49.355851 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:49.357281 29662 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:49.357986 29662 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.358728 30098 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:49.359949 29662 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:49.360770 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.361339 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.358245 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:49.367908 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.368525 22830 raft_consensus.cc:2410] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:09:49.375275 22756 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:49.289095 30076 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:49.381914 23444 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:49.389304 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:49.390009 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.397495 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.355109 29946 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:49.395071 22825 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:49.403496 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.398205 23447 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 24.
I20260709 13:09:49.352636 29824 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:49.410226 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 113227us
I20260709 13:09:49.412612 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.416851 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.417488 23444 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:49.408980 30114 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:09:49.426592 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.440438 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.441135 23453 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:49.442101 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.442656 23453 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:49.450794 30038 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.450878 30031 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:49.451316 30038 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.451731 30031 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.455117 29800 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:49.458719 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.419641 29995 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:49.461273 30038 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:49.461517 29995 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:49.461967 29995 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:49.465685 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.387235 30027 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:49.480904 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:49.484161 23451 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:49.493707 23453 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:49.493052 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.496353 30031 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:49.451261 30114 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:49.501560 30114 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:49.507019 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.507365 30125 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.507899 30125 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:49.508553 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:49.509567 22896 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:49.509807 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:49.510056 30125 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:49.510512 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:49.510998 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:49.407034 30178 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:49.431663 29860 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:49.481081 30018 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:49.494644 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:49.919766 23453 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 22. Current term is 23. Ops: []
W20260709 13:09:49.661664 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 51615.6us
I20260709 13:09:49.779989 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:49.581722 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.584723 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:49.581974 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 46253us
I20260709 13:09:49.639968 22871 leader_election.cc:400] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21.
I20260709 13:09:49.602941 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.704453 23449 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
W20260709 13:09:49.930239 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:49.939092 22828 consensus_meta.cc:287] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.650s	user 0.011s	sys 0.000s
I20260709 13:09:49.931834 22871 leader_election.cc:403] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:09:49.939821 22828 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
W20260709 13:09:49.553961 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 6ce771dbb93248d397c8d5b222b27fca: Illegal state: Replica 3b626c41600d4737a178ebab8a7ebfc2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:49.774771 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:49.553498 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:49.512976 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:49.681429 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.480453 30117 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:49.803910 30197 ts_tablet_manager.cc:933] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:49.971908 30198 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:09:49.976203 22899 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:49.977545 23453 raft_consensus.cc:1275] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:09:49.980006 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:49.925439 29859 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:09:49.986526 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 59619us
I20260709 13:09:49.992537 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:49.639856 30106 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:49.554868 29546 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:49.746577 30038 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:49.989410 23447 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:50.000198 23453 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:49.675755 30114 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:49.922040 29995 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:49.997802 22868 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.68:42017 timed out after 1.595s (SENT) [suppressed 46 similar messages]
I20260709 13:09:49.990011 30098 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:50.018172 22827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.017s	sys 0.020s
W20260709 13:09:50.018709 22827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.018s	sys 0.020s
I20260709 13:09:49.952759 30018 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:50.020037 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 504397us
W20260709 13:09:50.027881 22901 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.010s	sys 0.019s
W20260709 13:09:50.028329 22901 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.011s	sys 0.019s
W20260709 13:09:50.012127 29824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.017s	sys 0.022s
I20260709 13:09:49.982872 30117 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:09:50.034972 29800 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.014s	sys 0.014s
I20260709 13:09:49.577631 23448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:50.053574 23448 raft_consensus.cc:606] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:50.036516 29824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.017s	sys 0.022s
I20260709 13:09:50.052513 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:50.008000 30106 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.058416 23448 raft_consensus.cc:614] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:50.035254 30027 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:50.057229 23447 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:09:50.053335 29800 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.015s	sys 0.014s
I20260709 13:09:50.064893 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.066902 22830 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:50.019044 29546 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.069149 22871 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:50.071002 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.029716 30098 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.075588 29792 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:09:50.076581 30201 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.284s	user 0.003s	sys 0.000s
W20260709 13:09:50.076704 23450 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.010s	sys 0.019s
W20260709 13:09:50.078527 23450 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.010s	sys 0.020s
I20260709 13:09:50.072923 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:50.061822 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.071596 22826 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
W20260709 13:09:50.087545 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.074296 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.086397 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.091209 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.070178 30131 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:50.055001 30038 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:50.099049 23447 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:50.107903 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.109175 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.107412 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.097432 30131 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:50.126744 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.109784 29704 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.132006 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.108649 30027 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.106446 30114 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:50.138684 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 22 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:50.124773 30130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.035s	sys 0.064s
I20260709 13:09:50.135921 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.146999 23452 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:50.141830 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:50.151525 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.153244 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.134503 30178 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:50.154006 22830 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
W20260709 13:09:50.143602 30130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.035s	sys 0.064s
I20260709 13:09:50.176100 22825 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:50.177654 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.180501 22826 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:09:50.180951 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.185915 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:50.189561 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.154906 29251 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:50.193897 29251 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.185685 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.189795 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.192752 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.196576 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.198038 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.174089 29995 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:50.144764 30027 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.142617 29704 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.209619 30018 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:50.246326 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.223544 23448 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:50.224232 22899 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:50.247592 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:50.252051 23451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:50.252734 23451 raft_consensus.cc:606] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:50.253121 23451 raft_consensus.cc:614] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:50.253937 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.258260 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.258427 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 22 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.259181 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.250433 29903 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21.
I20260709 13:09:50.263599 22822 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:50.251287 30018 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:50.271083 29909 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.282013 22830 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:50.286302 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.288533 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:50.290048 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.291011 22756 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:09:50.292189 30203 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:09:50.294698 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.302968 22825 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:50.176924 30131 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.330053 22747 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:50.339571 23419 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:50.341265 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:50.322165 29251 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:50.342242 22830 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:50.343196 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.316881 29995 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.351226 29995 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.352094 30038 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:50.354236 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:50.355379 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.308144 29909 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.356099 22904 raft_consensus.cc:2393] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:50.360495 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:50.361109 23452 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 24.
I20260709 13:09:50.363006 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:50.364214 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:50.364789 23452 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
W20260709 13:09:50.362214 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48624.8us
I20260709 13:09:50.387722 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.346817 30106 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:50.350230 30131 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:50.389889 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.392949 22753 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:50.348951 29546 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:50.361542 30018 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.398977 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:50.426843 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 2395) took 1070 ms. Trace:
I20260709 13:09:50.440688 23453 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:50.446484 23451 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 23: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:50.508091 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:09:50.412274 30202 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
W20260709 13:09:50.400533 30204 log.cc:927] Time spent T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:09:50.393249 30098 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:50.399664 30117 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:50.438206 22827 rpcz_store.cc:276] 0709 13:09:49.356558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.374675 (+ 18117us) service_pool.cc:224] Handling call
0709 13:09:49.397825 (+ 23150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:49.411077 (+ 13252us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:49.433687 (+ 22610us) log.cc:844] Serialized 14 byte log entry
0709 13:09:50.019353 (+585666us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:50.041349 (+ 21996us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.041540 (+   191us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:50.041588 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.041681 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:50.426570 (+384889us) raft_consensus.cc:1685] finished
0709 13:09:50.426600 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:50.426774 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3211,"prepare.queue_time_us":18432,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":7144,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":10865,"thread_start_us":579163,"threads_started":1,"wal-append.queue_time_us":929274}
I20260709 13:09:50.528611 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:50.541358 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.511795 29995 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:50.530347 30098 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:50.464061 30018 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.596715 29704 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:50.602391 30098 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:50.658663 29859 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:50.706341 22824 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.014s	sys 0.046s
W20260709 13:09:50.706879 22824 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.014s	sys 0.046s
I20260709 13:09:50.683849 29546 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:50.683192 30117 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:50.674696 29859 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:50.716693 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:50.717775 29800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.035s	sys 0.068s
W20260709 13:09:50.718501 30170 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.007s	sys 0.016s
W20260709 13:09:50.722147 29800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.035s	sys 0.068s
I20260709 13:09:50.677628 30098 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:50.714653 30197 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.000s	sys 0.024s
I20260709 13:09:50.720621 30117 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:50.719563 29546 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:50.724574 30170 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.007s	sys 0.016s
W20260709 13:09:50.728255 30197 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.001s	sys 0.025s
W20260709 13:09:50.738442 30209 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:09:50.736982 30018 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:50.745857 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 341651us
I20260709 13:09:50.749279 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:50.757158 29835 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.006s	sys 0.020s
I20260709 13:09:50.751333 30197 tablet_copy_client.cc:323] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
W20260709 13:09:50.757647 29835 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.007s	sys 0.020s
W20260709 13:09:50.758486 22829 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.001s	sys 0.024s
W20260709 13:09:50.758878 22829 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.001s	sys 0.024s
I20260709 13:09:50.757453 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.760048 22752 raft_consensus.cc:2393] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 20: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:50.765108 29546 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:50.767124 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:50.769040 30205 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.490s	user 0.003s	sys 0.007s
I20260709 13:09:50.768837 22898 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:50.763240 29251 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.775429 29251 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:50.777913 30208 log.cc:927] Time spent T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.485s	user 0.004s	sys 0.000s
I20260709 13:09:50.778097 29859 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:50.796942 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.799438 23450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38074 (request call id 1325) took 1403 ms. Trace:
W20260709 13:09:50.802881 22868 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.013s	sys 0.021s
W20260709 13:09:50.803369 22868 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.013s	sys 0.021s
W20260709 13:09:50.803746 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 805901us
W20260709 13:09:50.805954 22828 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.002s	sys 0.034s
W20260709 13:09:50.806422 22828 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.002s	sys 0.034s
I20260709 13:09:50.779621 29251 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:50.807632 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 468123us
I20260709 13:09:50.808809 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:50.810111 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:50.808277 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 405927us
I20260709 13:09:50.811535 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:50.799908 23450 rpcz_store.cc:276] 0709 13:09:49.382782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.447873 (+ 65091us) service_pool.cc:224] Handling call
0709 13:09:49.480210 (+ 32337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:49.486414 (+  6204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.486454 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:49.496049 (+  9595us) log.cc:844] Serialized 52 byte log entry
0709 13:09:50.078894 (+582845us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:09:50.078945 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.079001 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:50.777340 (+698339us) spinlock_profiling.cc:243] Waited 57 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:50.779901 (+  2561us) raft_consensus.cc:1685] finished
0709 13:09:50.779921 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:50.786127 (+  6206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9000,"raft.queue_time_us":403,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":7406,"spinlock_wait_cycles":131792256,"thread_start_us":582478,"threads_started":1,"wal-append.queue_time_us":796434}
I20260709 13:09:50.810294 29704 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:50.846462 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.807928 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:50.848824 22719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.017s	sys 0.044s
W20260709 13:09:50.881750 22719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.017s	sys 0.044s
I20260709 13:09:50.886976 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.841040 30098 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:50.887015 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.846185 30227 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:09:50.925272 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:50.928013 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:50.892088 30117 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:50.941398 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 178613us
W20260709 13:09:50.941699 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 127389us
I20260709 13:09:50.935108 22827 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:50.925879 30018 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:50.947314 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 1.51393e+06us
I20260709 13:09:50.943076 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:50.948088 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:50.935775 29704 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:50.932574 30225 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:50.931334 30098 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
W20260709 13:09:50.962704 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.967401 30227 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:50.968096 30227 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:50.968546 30227 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:50.956898 30018 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.972456 30018 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:50.964148 30223 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:50.971957 30227 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.974058 30223 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.957299 29704 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:50.985939 30213 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:09:50.962256 30225 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:50.977644 30217 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:50.999897 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:50.961786 30228 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:51.019794 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 77633.1us
I20260709 13:09:51.004307 29704 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:51.027952 30223 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.028736 30223 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:51.032730 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:09:51.042194 30211 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.749s	user 0.009s	sys 0.005s
I20260709 13:09:51.043735 23444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1839) took 1622 ms. Trace:
I20260709 13:09:51.044111 23444 rpcz_store.cc:276] 0709 13:09:49.421470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.450665 (+ 29195us) service_pool.cc:224] Handling call
0709 13:09:49.472122 (+ 21457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:49.483128 (+ 11006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.483162 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:49.493203 (+ 10041us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.013308 (+520105us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:50.013389 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.013465 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:51.041578 (+1028113us) spinlock_profiling.cc:243] Waited 58.2 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:51.043450 (+  1872us) raft_consensus.cc:1685] finished
0709 13:09:51.043471 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.043681 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4010,"raft.queue_time_us":8367,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5949,"spinlock_wait_cycles":133897216,"thread_start_us":457565,"threads_started":1,"wal-append.queue_time_us":719378}
I20260709 13:09:51.059026 30225 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.073508 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:51.075322 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 132310us
I20260709 13:09:51.076965 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:51.077682 22748 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:09:51.087785 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:51.086333 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 67270.7us
I20260709 13:09:51.086617 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:51.091696 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:51.080583 30126 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:51.093827 22754 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:51.026589 30228 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.090082 30236 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:51.018419 30217 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.082119 30225 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:51.096964 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:51.098042 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72312.7us
I20260709 13:09:51.100319 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.102481 30236 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.099737 29704 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.120894 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:51.122182 23416 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.123864 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.125885 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.119969 30114 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:51.127470 30225 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:51.125279 30243 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:51.128265 29909 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.128412 30228 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.128497 30243 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.129036 30243 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:51.130421 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:51.131086 22904 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:51.134469 30236 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.135025 30234 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:51.136189 29995 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:51.130558 30217 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.129004 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.138089 29977 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:51.138489 29977 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:51.121553 30227 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:51.139986 29977 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:51.140501 29704 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.136829 30243 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.139554 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.143144 29704 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:51.144204 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:51.144440 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 185912us
I20260709 13:09:51.145188 30243 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:51.145263 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:51.146080 29704 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:51.146206 30212 log.cc:927] Time spent T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.821s	user 0.012s	sys 0.004s
I20260709 13:09:51.148167 29977 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.149446 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2282) took 1641 ms. Trace:
I20260709 13:09:51.150460 29977 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.146572 29995 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:51.147289 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.153478 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:51.152968 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.147032 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.149789 22896 rpcz_store.cc:276] 0709 13:09:49.508260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.511604 (+  3344us) service_pool.cc:224] Handling call
0709 13:09:49.512028 (+   424us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:49.522144 (+ 10116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.522190 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:49.640139 (+117949us) log.cc:844] Serialized 52 byte log entry
0709 13:09:50.061876 (+421737us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:50.090501 (+ 28625us) log.cc:844] Serialized 14 byte log entry
0709 13:09:50.091137 (+   636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.091210 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:51.145499 (+1054289us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:51.149143 (+  3644us) raft_consensus.cc:1685] finished
0709 13:09:51.149246 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.149392 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45292,"prepare.queue_time_us":209081,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":47939,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":50699,"spinlock_wait_cycles":62010240,"thread_start_us":421296,"threads_started":1,"wal-append.queue_time_us":683980}
I20260709 13:09:51.153141 30217 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:51.158463 30240 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:51.157707 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:51.159315 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.156665 29835 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:51.160346 29546 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:51.160429 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:51.161461 22754 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:09:51.161103 29546 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.161522 30106 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:51.161007 29835 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.164681 29546 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.152778 29995 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.165243 29995 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:51.165982 29835 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.166957 30131 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:51.162039 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 231125us
I20260709 13:09:51.167682 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.169665 22754 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:09:51.170398 30227 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:51.170475 23419 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:51.170897 30227 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:51.171311 30227 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:51.171344 30236 raft_consensus.cc:2804] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:51.171991 30236 raft_consensus.cc:697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 26 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:51.172835 30236 consensus_queue.cc:237] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.173695 30027 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.174211 30027 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:51.174676 30027 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.175454 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:51.176044 22751 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
W20260709 13:09:51.177405 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.179131 30131 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.180220 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:51.189543 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:51.179610 30227 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.184722 30131 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.162122 30240 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:51.191988 30240 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:51.177204 30027 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.189759 22754 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:51.194146 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.194491 30227 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:51.195501 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.196954 30178 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:51.197726 30178 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:51.199195 22868 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.167573 30243 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:51.154794 29977 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:51.217482 29977 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.217912 29977 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:51.219164 30228 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:51.219345 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:51.219972 30227 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:51.220400 30227 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.220713 30227 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:51.220937 22751 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:09:51.206633 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.219699 30228 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.215471 22829 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43624 (request call id 2391) took 1970 ms. Trace:
I20260709 13:09:51.232323 22797 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:51.202247 30240 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.240140 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:51.238834 22797 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:51.243186 22751 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:51.240043 30225 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:51.246869 22719 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:51.232765 22829 rpcz_store.cc:276] 0709 13:09:49.245416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.245758 (+   342us) service_pool.cc:224] Handling call
0709 13:09:49.246264 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:49.953154 (+706890us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:09:50.759242 (+806088us) log.cc:844] Serialized 14 byte log entry
0709 13:09:51.168641 (+409399us) spinlock_profiling.cc:243] Waited 5.1 ms on lock 0x7b1800243d00. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb749a34bc 00007fcb71cf148b 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb7499520b 00007fcb7499ad23 00007fcb7499b23a
0709 13:09:51.168821 (+   180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:51.168871 (+    50us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:51.168903 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:51.215242 (+ 46339us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400432454. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:51.215298 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.215418 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":858098,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":95,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":666804992,"thread_start_us":873597,"threads_started":2,"wal-append.queue_time_us":204874}
I20260709 13:09:51.236649 30106 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:51.252532 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:51.242400 30098 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.258769 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:51.260490 23453 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:51.260836 23449 raft_consensus.cc:2376] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21. Re-sending same reply.
I20260709 13:09:51.231657 29704 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:09:51.259666 30244 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:51.267777 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:51.268381 23453 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 25.
I20260709 13:09:51.268818 23450 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:51.271029 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.246790 29859 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:51.274121 29859 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.274569 29859 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:51.275692 30240 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.273722 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:51.276643 23453 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:51.278463 30027 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:51.280259 30170 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:51.280732 30170 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.281121 30170 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:51.283612 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.275768 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:51.271857 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 84682.7us
I20260709 13:09:51.248621 30181 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:51.273123 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:51.291589 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.292483 23449 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:51.293606 22829 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:51.294534 30181 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.240517 30038 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:09:51.294816 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.295617 30038 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:51.296030 30038 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:51.294917 30181 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:51.298408 29995 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.298921 22831 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:09:51.299250 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.295634 22824 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:51.286625 23451 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:51.300042 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.303143 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:51.273100 22868 connection.cc:256] RPC call timeout handler was delayed by -1.3094538749999174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
I20260709 13:09:51.313328 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.314579 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.318080 30126 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:51.322229 22828 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:51.247205 30225 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.342500 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.344764 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.350538 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.303105 30234 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.306669 29251 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:51.269870 30098 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.363863 29977 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.364435 30098 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:51.296924 30038 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.373077 30131 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:51.373950 29887 consensus_queue.cc:1175] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3b626c41600d4737a178ebab8a7ebfc2" responder_term: 18 status { last_received { term: 18 index: 14 } last_committed_idx: 14 last_received_current_leader { term: 18 index: 14 } }
I20260709 13:09:51.373822 30131 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:51.373847 30038 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:51.376356 30131 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:51.378398 30178 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:51.379037 30178 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.368688 29977 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.350287 29859 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.369434 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:51.381407 22899 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:51.359244 29251 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.382282 29251 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:51.382472 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:51.371135 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:51.383078 22902 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:51.364991 30114 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:51.383960 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:51.385439 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:51.386034 22903 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:51.386770 29546 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:51.387356 29546 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.389554 29859 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.391769 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:51.380987 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:51.346438 30236 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.394675 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.400274 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.392222 29546 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.398499 30114 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:51.396669 30244 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:51.399410 29977 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:51.410347 30178 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.417250 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:51.417881 22748 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:51.419204 23419 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:51.362106 30234 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.421912 30234 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.422031 30225 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:51.422652 30225 raft_consensus.cc:2764] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:09:51.423369 30234 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:51.423734 30234 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.400044 30236 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.424054 30234 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:51.425837 30236 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:51.425279 30181 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.431869 30131 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:09:51.433696 30227 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.436676 30227 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:51.433694 30244 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:51.440807 30170 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.427956 30181 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:51.577777 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 289679us
I20260709 13:09:51.578576 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:51.579936 30251 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:51.579808 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.580413 30251 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.580888 30251 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:51.581880 30254 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:51.582430 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.583351 30252 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:51.584327 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.584614 30252 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:51.582536 30254 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:09:51.585901 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:51.587464 30254 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:51.588120 30254 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:51.588457 30254 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:51.588825 30254 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:51.599004 30251 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.605096 29995 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:51.631458 29995 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:51.632090 29995 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.635499 29995 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.635485 30254 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.639782 30251 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:51.645040 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 49971.8us
I20260709 13:09:51.645699 30254 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.640744 30234 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.646526 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:51.646468 30264 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 13:09:51.648597 29251 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.649499 30254 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:51.649659 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.652202 30254 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:51.652679 30254 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:51.648211 30234 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:51.653884 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:51.654844 30018 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:51.655196 30264 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:51.655308 30018 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:51.656157 30018 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:51.656867 29251 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.657596 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:51.658419 30251 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:51.658803 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:51.659714 30262 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:51.660174 30262 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:51.658828 30251 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:51.660966 30251 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:51.661096 30262 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.665637 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:51.666226 23449 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:51.669010 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:51.669610 23444 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:51.670857 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:51.670908 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:51.671897 23452 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:51.674873 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:51.676224 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:51.676417 30018 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.678800 30018 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.669173 30251 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.681980 23444 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:09:51.688046 30251 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.692588 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:51.693634 30254 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.698213 30254 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.703281 29251 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.703751 29251 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.704787 30259 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:51.705408 29251 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.705356 30259 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.706384 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:51.707190 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:51.707705 30259 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.708065 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:51.707957 30254 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:51.710493 30254 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.710814 30254 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:51.713225 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:51.714478 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:51.714728 30251 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:51.715193 23449 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:51.715090 30251 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:51.715562 30251 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:51.716022 30018 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:09:51.726998 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 321098us
I20260709 13:09:51.727658 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.728891 30240 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:51.728652 30251 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.732961 30251 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.733837 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:51.734563 29251 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:51.734962 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.742125 30259 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:51.733997 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:51.740054 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:51.735746 30254 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.742859 23449 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:09:51.745388 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 339710us
I20260709 13:09:51.747038 23416 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:51.747841 30225 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:51.747918 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:51.748245 30225 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:51.748704 30260 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:51.748634 30225 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:51.749148 30260 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.749624 30260 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:51.742780 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:51.754305 30254 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.756713 23445 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:51.756861 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:51.757468 23448 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:51.757696 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:51.761474 30254 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:09:51.761994 30254 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:51.762398 30260 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.761240 30170 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:51.749257 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.766613 30234 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:51.766989 30234 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:51.767349 30234 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:51.768875 29251 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:51.763975 30254 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.774628 30260 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.743244 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.780682 30027 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:51.781018 30027 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:51.781385 30027 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:51.790321 30027 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:51.794081 23449 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:51.795544 30027 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.775700 30225 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.796705 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:51.800168 30225 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.801714 30254 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:51.802295 30254 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:51.803109 30254 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.805074 30264 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:51.805518 30264 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:51.806360 30264 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:51.807133 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.809536 30228 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:51.809880 22840 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 84ea43361e534fbebad704327c9f6367 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:09:51.810441 22840 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 84ea43361e534fbebad704327c9f6367 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-84ea43361e534fbebad704327c9f6367
I20260709 13:09:51.810956 30240 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:51.811525 30240 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.277575 22652 catalog_manager.cc:5697] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:51.812798 29859 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.813275 29859 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:51.813640 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.814275 22830 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:09:51.815268 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.815826 22828 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 25.
I20260709 13:09:51.817521 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:51.834947 22823 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:09:51.835647 22823 raft_consensus.cc:606] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:51.836038 22823 raft_consensus.cc:614] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:51.837548 22827 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:51.900095 22840 tablet_copy_source_session.cc:215] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:52.095296 30234 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.097960 30234 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:52.111617 23448 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:52.304741 30264 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:09:52.306207 22719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.310796 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.311384 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.311507 22824 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:09:52.312150 23448 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 20: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:52.312831 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.313426 23447 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 16.
I20260709 13:09:52.313740 22719 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:52.314603 30242 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:52.315235 30242 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:52.309451 30228 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.317747 30228 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.319190 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.319729 22823 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:52.319808 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.320508 22828 raft_consensus.cc:2368] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 22. Current term is 23.
I20260709 13:09:52.320302 30228 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:52.309096 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.321811 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.321888 23451 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:52.322549 22822 raft_consensus.cc:3060] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:52.323195 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 573934us
I20260709 13:09:52.325894 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.326853 22823 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:52.328339 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.331660 22830 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:52.332700 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.333446 23449 raft_consensus.cc:2368] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:52.334270 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:52.335839 23445 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:52.316363 30242 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.337634 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:52.338187 23444 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:52.341668 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.342253 23449 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:52.342343 30228 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:52.342895 30228 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.344096 29859 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:52.344858 30228 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:52.346349 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.346479 23444 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:09:52.347114 22825 raft_consensus.cc:2393] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:52.347965 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.348505 22825 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:52.349593 29859 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.350077 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.351280 22825 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:52.352030 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.352919 22825 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 25: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:09:52.353499 29909 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.007s	sys 0.009s
W20260709 13:09:52.355602 29909 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.008s	sys 0.010s
I20260709 13:09:52.353670 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.359079 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:52.360061 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.360649 23448 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 23: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:52.358120 30263 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:52.358081 30254 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.002s	sys 0.000s
I20260709 13:09:52.363298 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:52.363847 23447 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:52.357195 29859 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.365805 23452 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:52.371361 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:52.371726 22822 raft_consensus.cc:2468] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:09:52.374153 23452 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 23: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
W20260709 13:09:52.364907 30254 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.006s	sys 0.000s
I20260709 13:09:52.376927 30254 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:52.378644 30254 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:52.377806 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:52.379127 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.379766 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:52.380132 23445 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:52.380523 23446 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 25: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:52.381608 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.381814 30254 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:52.377924 23450 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:52.385215 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:52.386197 23448 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:52.363183 30263 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:52.391018 30263 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:52.389086 23447 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:52.394807 30254 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.397617 30254 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:52.399219 29859 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:52.431710 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:52.436884 30274 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:52.437597 23419 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.030s	sys 0.018s
W20260709 13:09:52.438097 23419 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.030s	sys 0.018s
W20260709 13:09:52.438455 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 1.01923e+06us
W20260709 13:09:52.444012 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 84968.4us
I20260709 13:09:52.446790 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.447474 22756 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:52.447986 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.451064 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.451578 22747 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:52.451617 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.452212 22756 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:52.453450 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.456386 30242 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:52.456415 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.456899 30242 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:52.458876 30242 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:52.459841 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.460634 22753 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:52.460956 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.461354 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.462863 22822 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:52.459148 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.463675 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.464578 23448 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:09:52.464481 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.464979 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.460660 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.465260 22825 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 23: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:52.465898 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:52.466966 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.467204 23450 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:52.469264 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.456617 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.476321 22747 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 24 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 13:09:52.477161 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.477406 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.485383 22825 raft_consensus.cc:2368] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 25. Current term is 26.
I20260709 13:09:52.489112 22825 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:09:52.473886 30274 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.472332 30242 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:52.499202 30242 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.505687 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 25 index: 22 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.506381 23450 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 25.
I20260709 13:09:52.513810 30274 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.514031 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:52.515044 30277 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:52.515125 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.515594 30277 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:52.515782 22828 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:52.515884 30274 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:52.516114 30277 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:52.479097 30271 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:09:52.497080 22829 raft_consensus.cc:1240] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:09:52.518872 30240 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:52.497998 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800225b20 after lost signal to thread 21629
I20260709 13:09:52.494808 30263 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
W20260709 13:09:52.520201 22777 debug-util.cc:398] Leaking SignalData structure 0x7b080005f8c0 after lost signal to thread 21632
I20260709 13:09:52.473682 30254 consensus_queue.cc:1059] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:09:52.520503 30263 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:52.520957 30263 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:52.521283 30254 raft_consensus.cc:3055] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:52.521679 30254 raft_consensus.cc:740] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:52.522004 30263 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.522461 30254 consensus_queue.cc:260] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.523173 30263 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:52.523563 30254 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:52.524971 22828 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:52.518291 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.525707 22829 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:52.530560 22756 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:52.534222 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.534920 22828 raft_consensus.cc:2368] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18.
I20260709 13:09:52.535630 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.550416 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.554867 22829 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:52.555790 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.555853 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.559729 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.560446 22828 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:52.563105 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.563179 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.564450 22748 raft_consensus.cc:2410] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:09:52.564796 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.566049 22754 raft_consensus.cc:2410] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
W20260709 13:09:52.565944 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76958.5us
I20260709 13:09:52.539161 30242 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:52.568080 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.568826 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:52.569111 30234 consensus_queue.cc:1059] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:52.570403 30234 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:52.570737 30234 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.571094 30234 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:52.571761 22749 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:52.572522 29909 raft_consensus.cc:3055] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:52.572943 29909 raft_consensus.cc:740] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:52.573508 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:52.577255 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:52.579180 22828 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:52.578465 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:52.581193 22829 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 23: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:52.584915 22755 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:09:52.592505 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.593068 22748 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:52.611351 22719 leader_election.cc:400] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 25. Current term is 26.
I20260709 13:09:52.611812 22719 leader_election.cc:403] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:52.595242 30242 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:52.625211 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.627673 22828 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 24.
I20260709 13:09:52.624894 30242 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:52.612403 29909 consensus_queue.cc:260] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.633953 22823 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:52.631639 30242 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:52.639248 22755 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:52.623270 30271 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 25. Current term is 26.
I20260709 13:09:52.647596 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:52.649716 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.650806 30131 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:52.651206 30131 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:52.652264 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:52.652534 29909 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:52.710338 29704 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:09:52.713581 30278 log.cc:927] Time spent T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:09:52.714184 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 3340) took 1429 ms. Trace:
I20260709 13:09:52.714607 22827 rpcz_store.cc:276] 0709 13:09:51.284956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.299022 (+ 14066us) service_pool.cc:224] Handling call
0709 13:09:52.713982 (+1414960us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400431f54. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:52.714099 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236615936}
I20260709 13:09:52.720593 30098 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:52.633971 30284 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.093s
I20260709 13:09:52.701092 30280 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:52.730449 30280 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.732874 30280 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:52.733386 30279 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:09:52.739840 22777 debug-util.cc:398] Leaking SignalData structure 0x7b08000a5d20 after lost signal to thread 22688
W20260709 13:09:52.740239 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800328dc0 after lost signal to thread 22702
I20260709 13:09:52.737617 30271 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:52.740708 30271 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:52.744194 30272 log.cc:927] Time spent T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:09:52.652032 30131 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.747403 22751 raft_consensus.cc:2393] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 26: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:52.721086 30098 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:52.750056 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:52.749677 30098 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.750739 22903 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:52.751811 22828 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:09:52.754137 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:52.756083 22899 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:09:52.757854 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 1.01909e+06us
W20260709 13:09:52.779296 22777 debug-util.cc:398] Leaking SignalData structure 0x7b080025acc0 after lost signal to thread 22780
I20260709 13:09:52.787904 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:52.789531 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
W20260709 13:09:52.790158 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800274020 after lost signal to thread 22781
W20260709 13:09:52.791253 22777 debug-util.cc:398] Leaking SignalData structure 0x7b08000a3300 after lost signal to thread 22855
I20260709 13:09:52.797919 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:52.799685 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 159696us
I20260709 13:09:52.800546 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:52.810105 22777 debug-util.cc:398] Leaking SignalData structure 0x7b08000aa820 after lost signal to thread 22856
W20260709 13:09:52.811347 22777 debug-util.cc:398] Leaking SignalData structure 0x7b080033c5c0 after lost signal to thread 22929
I20260709 13:09:52.816918 22719 leader_election.cc:400] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 17. Current term is 18.
I20260709 13:09:52.818138 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:52.818770 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:52.819209 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:52.819761 23416 leader_election.cc:400] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 22. Current term is 23.
I20260709 13:09:52.820142 23416 leader_election.cc:403] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:09:52.819893 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:52.820647 22904 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:52.821810 22755 raft_consensus.cc:1240] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
W20260709 13:09:52.836355 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800417020 after lost signal to thread 22930
W20260709 13:09:52.837591 22777 debug-util.cc:398] Leaking SignalData structure 0x7b08003af620 after lost signal to thread 23478
I20260709 13:09:52.729007 30234 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.838258 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:53.045297 23448 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:52.837895 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:53.046558 30280 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:09:53.046689 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 208854us
I20260709 13:09:53.046945 30280 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:53.047303 30280 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:53.047444 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.048565 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:52.852883 30228 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 22. Current term is 23.
I20260709 13:09:53.050053 30228 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:53.050727 30228 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:52.869107 22652 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 17 to 19, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:53.052138 22652 catalog_manager.cc:5697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 17, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:53.053486 22652 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:53.054509 30280 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.055171 22652 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:52.764658 29800 consensus_queue.cc:1048] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:09:53.056599 30280 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:52.881603 22752 raft_consensus.cc:2410] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:09:52.881584 22904 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:52.882763 30178 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:52.819983 30252 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:53.061136 30252 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:53.060978 30178 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.062305 30252 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.063675 30178 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:52.836295 30284 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:53.065614 30284 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:53.066690 30284 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:52.909783 30197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ea43361e534fbebad704327c9f6367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:53.069623 22652 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 19 to 21, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 21 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:52.905529 30271 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:52.937709 30277 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:52.942598 22651 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 337) took 22483 ms (client timeout 15000 ms). Trace:
I20260709 13:09:53.079092 30277 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:52.818158 30263 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:53.081689 30263 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:53.084136 30263 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:52.848129 30131 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:53.033048 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:53.087983 22828 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:53.034497 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:53.088930 22826 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:53.035892 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:53.037278 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:53.038762 22829 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21. Ops: []
W20260709 13:09:53.094025 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 282175us
I20260709 13:09:53.038995 29909 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:53.097553 29977 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:53.097932 29909 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.041584 22824 raft_consensus.cc:1275] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 24 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 13:09:52.871857 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.040110 22831 raft_consensus.cc:1240] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 19. Current term is 20. Ops: [17.14-19.15]
I20260709 13:09:53.071283 23416 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:53.098495 29937 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:09:53.102221 29937 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:53.103350 29937 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.104827 29937 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:09:53.108212 30274 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:53.108661 30274 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:53.109098 30274 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:53.113834 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:53.074120 22651 rpcz_store.cc:269] 0709 13:09:30.459249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.459501 (+   252us) service_pool.cc:224] Handling call
0709 13:09:52.924615 (+22465114us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:46.849254 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:46.849904 (+   650us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:46.849954 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:09:46.849975 (+    21us) tablet.cc:662] Decoding operations
0709 13:09:46.852106 (+  2131us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:46.852169 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:46.852199 (+    30us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:09:46.867355 (+ 15156us) tablet.cc:701] Row locks acquired
0709 13:09:46.867377 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:09:46.867515 (+   138us) write_op.cc:270] Start()
0709 13:09:46.867628 (+   113us) write_op.cc:276] Timestamp: P: 1783602586867473 usec, L: 0
0709 13:09:46.867651 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:09:46.868358 (+   707us) log.cc:844] Serialized 9886 byte log entry
0709 13:09:47.700513 (+832155us) op_driver.cc:464] REPLICATION: finished
0709 13:09:47.877917 (+177404us) write_op.cc:301] APPLY: starting
0709 13:09:47.878005 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:48.430924 (+552919us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:48.450316 (+ 19392us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:49.184908 (+734592us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:49.257907 (+ 72999us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:09:49.257953 (+    46us) write_op.cc:312] APPLY: finished
0709 13:09:49.258586 (+   633us) log.cc:844] Serialized 220 byte log entry
0709 13:09:49.768254 (+509668us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:49.879835 (+111581us) write_op.cc:454] Released schema lock
0709 13:09:49.949657 (+ 69822us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:50.050752 (+101095us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:50.053162 (+  2410us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":936042,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":174125,"apply.run_cpu_time_us":75626,"apply.run_wall_time_us":2134060,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108627,"num_ops":25,"prepare.queue_time_us":230078,"prepare.run_cpu_time_us":35895,"prepare.run_wall_time_us":193698,"raft.queue_time_us":310708,"raft.run_cpu_time_us":20408,"raft.run_wall_time_us":163490,"replication_time_us":832706,"thread_start_us":1171105,"threads_started":5,"wal-append.queue_time_us":1231067,"wal-append.run_cpu_time_us":104080,"wal-append.run_wall_time_us":3075075}]]}
I20260709 13:09:53.136653 30228 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:53.118410 29909 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:53.156076 29251 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:53.157008 29251 consensus_queue.cc:1243] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:09:53.157771 29977 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:53.158250 29977 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:53.159614 30234 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:53.159461 29977 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.160383 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.161886 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.163683 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.165485 29909 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:53.168695 30228 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:53.169621 30228 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:53.178617 29977 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:53.073144 22652 catalog_manager.cc:5697] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } }
I20260709 13:09:53.181605 22652 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 16 to 19, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 19 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:53.005343 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 187439us
I20260709 13:09:53.185442 22719 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.186391 30277 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:53.187023 30277 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:53.188073 30277 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:53.194391 22777 debug-util.cc:398] Leaking SignalData structure 0x7b08002a8c40 after lost signal to thread 23479
W20260709 13:09:53.195325 22777 debug-util.cc:398] Leaking SignalData structure 0x7b080032b3e0 after lost signal to thread 30206
W20260709 13:09:53.195818 22777 debug-util.cc:398] Leaking SignalData structure 0x7b0800242d40 after lost signal to thread 30218
I20260709 13:09:53.178839 22826 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
W20260709 13:09:53.214205 30223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.014s	sys 0.038s
W20260709 13:09:53.214660 30223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.014s	sys 0.038s
W20260709 13:09:53.215538 23451 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.003s	sys 0.008s
W20260709 13:09:53.216068 23451 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.004s	sys 0.008s
W20260709 13:09:53.217391 23450 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.007s	sys 0.006s
W20260709 13:09:53.217912 23450 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.007s	sys 0.006s
I20260709 13:09:53.224037 22752 raft_consensus.cc:1240] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:09:53.228667 23419 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.041s	sys 0.114s
W20260709 13:09:53.229063 23419 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.041s	sys 0.114s
W20260709 13:09:53.229523 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 649156us
W20260709 13:09:53.229892 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44565.1us
I20260709 13:09:53.231014 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:53.231756 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.980074 30242 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.238188 22752 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 25. Current term is 26. Ops: []
W20260709 13:09:53.238852 22825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.000s	sys 0.013s
W20260709 13:09:53.240792 22825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.001s	sys 0.013s
I20260709 13:09:53.233670 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:53.243606 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:53.244351 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:53.245499 22822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.020s	sys 0.045s
W20260709 13:09:53.245951 22822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.020s	sys 0.045s
I20260709 13:09:53.245884 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:53.246876 22823 raft_consensus.cc:2376] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20. Re-sending same reply.
I20260709 13:09:53.254115 22652 catalog_manager.cc:5697] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 19, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:53.258049 23450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:53.258107 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.258945 23450 raft_consensus.cc:606] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.259235 30280 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:53.259418 23450 raft_consensus.cc:614] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:53.246891 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:53.262274 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 451120us
I20260709 13:09:53.264192 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:53.268011 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.263751 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.266930 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:53.271988 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 98731.4us
W20260709 13:09:53.253235 30262 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.008s	sys 0.064s
I20260709 13:09:53.270607 22652 catalog_manager.cc:5697] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:53.253024 30223 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:09:53.261452 30234 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:09:53.262598 30244 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.005s	sys 0.014s
I20260709 13:09:53.271445 30263 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:09:53.283282 30262 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.008s	sys 0.064s
I20260709 13:09:53.277840 30254 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:53.290107 22652 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 18 to 24, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 24 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:53.260994 30280 consensus_queue.cc:1048] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.025s
I20260709 13:09:53.284170 30098 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:53.291939 29251 raft_consensus.cc:2955] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:09:53.289213 30244 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.005s	sys 0.014s
I20260709 13:09:53.261088 30274 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.297408 30254 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:53.302711 30297 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:53.302850 30254 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:53.303248 30297 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:53.293226 30263 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:53.305346 30263 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:53.305581 30178 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.306044 30178 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.282310 30271 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:09:53.287410 30234 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:53.313607 22652 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 22 to 24, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 24 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:53.338496 22905 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
new_leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
 from {username='slave'} at 127.0.0.1:58800
I20260709 13:09:53.339201 22905 raft_consensus.cc:606] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Received request to transfer leadership to TS 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.339625 22905 raft_consensus.cc:614] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:53.291332 30294 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:09:53.349942 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 85233.2us
I20260709 13:09:53.350157 22652 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:53.341673 30234 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:53.373401 23418 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.004s	sys 0.009s
W20260709 13:09:53.379351 23418 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.004s	sys 0.010s
I20260709 13:09:53.373163 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:53.376789 30298 consensus_queue.cc:1059] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:09:53.381664 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 101791us
I20260709 13:09:53.381474 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:53.387836 23450 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:53.387503 22652 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 14 to 17, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:53.387323 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:53.390570 23451 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:53.392290 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 25 index: 22 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:53.392920 23450 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:53.373229 30294 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:53.355530 30295 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:53.396450 23448 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:53.398873 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.382521 30307 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.399978 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:53.382148 30274 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:53.401029 22652 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:53.401649 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.384644 30280 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.402285 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.405568 22896 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:09:53.403451 22904 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 25.
I20260709 13:09:53.402316 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.380945 30297 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:53.404392 22749 raft_consensus.cc:2393] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:53.406466 22902 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:53.407233 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.408324 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.408143 22652 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:53.409495 29251 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:47094 (request call id 21) took 2576 ms. Trace:
I20260709 13:09:53.407183 29937 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.411414 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.411954 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:53.412604 29977 consensus_queue.cc:1059] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:53.398465 29800 raft_consensus.cc:3055] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:53.406744 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:53.442831 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.443229 22755 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:53.407624 30307 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.415305 30262 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:53.415939 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.453692 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.415966 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:53.417798 30289 log.cc:927] Time spent T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:09:53.419800 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.420533 30242 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:53.420789 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.389021 30287 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:53.423830 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.407287 30178 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:53.464259 22749 raft_consensus.cc:2955] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:09:53.424898 29251 rpcz_store.cc:276] 0709 13:09:50.837422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.842673 (+  5251us) service_pool.cc:224] Handling call
0709 13:09:51.020731 (+178058us) log.cc:844] Serialized 499 byte log entry
0709 13:09:51.146906 (+126175us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b54002e0020. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d7bb26 00007fcb74d7549a 00007fcb74d74714 00007fcb74d54eaa 00007fcb74d5bd53 00007fcb74d5bcd9 00007fcb74d5bc69 00007fcb74d5bc31 00007fcb74d5adfd 000000000038f354
0709 13:09:51.178508 (+ 31602us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b180004db20. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74e37d01 00007fcb74d74691 00007fcb74d54eaa 00007fcb74d5bd53 00007fcb74d5bcd9 00007fcb74d5bc69 00007fcb74d5bc31 00007fcb74d5adfd 000000000038f354 000000000038f199
0709 13:09:53.403853 (+2225345us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":993,"raft.queue_time_us":4857,"raft.run_cpu_time_us":5093,"raft.run_wall_time_us":193778,"spinlock_wait_cycles":7821440,"thread_start_us":38556,"threads_started":1,"wal-append.queue_time_us":98443,"wal-append.run_cpu_time_us":11220,"wal-append.run_wall_time_us":1324222}
I20260709 13:09:53.441735 22902 raft_consensus.cc:2468] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:53.443450 22899 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:53.456259 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.454054 22903 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:53.448510 22652 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:53.394259 30298 consensus_queue.cc:1059] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 25.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.050s
W20260709 13:09:53.457360 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.467710 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.472155 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.474555 22652 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 15 to 20, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 20 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:53.472013 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:53.476588 23447 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 18: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:53.477494 29977 raft_consensus.cc:3055] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:53.477865 29977 raft_consensus.cc:740] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:53.478662 29977 consensus_queue.cc:260] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.480187 29977 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:53.481882 22902 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:53.442955 29800 raft_consensus.cc:740] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:53.484246 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:53.489042 29251 raft_consensus.cc:2764] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:09:53.488163 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.492049 22823 raft_consensus.cc:1240] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:53.492223 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:53.407243 30277 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:09:53.453748 30262 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:53.500077 22825 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:09:53.496822 22897 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:53.424054 30280 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:53.521404 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 257690us
I20260709 13:09:53.502291 22652 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 20 to 25. New cstate: current_term: 25 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:53.503738 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:53.504544 30277 raft_consensus.cc:3055] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:53.506111 29800 consensus_queue.cc:260] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.508558 23450 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 26.
I20260709 13:09:53.456645 30264 consensus_queue.cc:1059] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:09:53.467634 30296 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.509544 22831 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:53.510030 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:09:53.516528 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56328.7us
I20260709 13:09:53.502535 30304 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:53.448362 29937 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.454919 30271 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:09:53.524221 22652 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 13 to 18, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 18 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:53.534036 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:53.537909 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 44732.7us
I20260709 13:09:53.557934 22824 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:53.463030 30309 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:09:53.563081 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.569337 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.570050 22902 raft_consensus.cc:2393] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:53.572775 29251 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:53.575687 29251 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:53.523659 30277 raft_consensus.cc:740] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:53.571480 22652 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 21 to 22, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 22 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:53.572376 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.581609 30252 raft_consensus.cc:3055] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:53.582026 30252 raft_consensus.cc:740] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:53.582242 29937 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:53.473517 30287 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.583527 29251 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:53.582866 30252 consensus_queue.cc:260] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.586560 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.589783 22903 raft_consensus.cc:1240] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:53.593895 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:53.594630 22754 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:53.594971 22868 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:53.597357 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 107572us
I20260709 13:09:53.597725 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.592449 22652 catalog_manager.cc:5697] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 18 to 19, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:53.599583 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.589888 30298 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:53.603199 22719 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:53.594329 29800 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:53.608131 30262 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:53.609189 30262 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:53.610055 30314 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:53.610603 30314 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:53.611616 30314 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.613107 30307 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:53.536119 30294 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.615422 30197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38090 (request call id 1934) took 4482 ms. Trace:
I20260709 13:09:53.619006 22896 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:53.600289 30305 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.601842 30252 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:53.601435 30299 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:09:53.626621 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 25 index: 22 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.623584 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.562059 30296 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.637208 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.583144 30313 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:09:53.642813 22897 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:09:53.646540 30287 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:53.647178 30263 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:53.647686 30280 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:53.650844 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.651402 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.652729 22824 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:53.652909 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:53.654605 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.655674 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:53.645231 22755 raft_consensus.cc:2468] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:53.661772 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.662495 22897 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:53.668581 23419 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:53.669782 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 46316.3us
I20260709 13:09:53.670037 29977 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:53.625217 30223 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:53.635797 30178 raft_consensus.cc:3055] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:09:53.673043 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:53.674408 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 25 index: 22 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.678730 30285 raft_consensus.cc:2804] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:53.680828 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.615885 30197 rpcz_store.cc:276] 0709 13:09:49.132610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.154525 (+ 21915us) service_pool.cc:224] Handling call
0709 13:09:49.998772 (+844247us) ts_tablet_manager.cc:934] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
0709 13:09:52.872419 (+2873647us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:53.191031 (+318612us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:53.191164 (+   133us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:53.614607 (+423443us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:53.614943 (+   336us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:53.615260 (+   317us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:53.615328 (+    68us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":11209,"tablet-copy.queue_time_us":642731,"thread_start_us":865656,"threads_started":2}
I20260709 13:09:53.681979 22868 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.682207 30197 tablet_copy_client.cc:806] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:53.682821 30197 tablet_copy_client.cc:670] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:53.682865 30295 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:09:53.683420 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:53.684276 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:53.685639 29903 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:53.685639 29546 raft_consensus.cc:2764] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:09:53.690917 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:53.692050 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.669739 30284 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:53.700335 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.702862 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:53.703500 22831 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:53.707556 23416 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.708963 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:53.676172 30178 raft_consensus.cc:740] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:53.720463 22825 raft_consensus.cc:1240] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:53.685303 30313 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:53.688194 30263 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:53.712651 23448 raft_consensus.cc:2393] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 20: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:53.714095 23450 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:53.714387 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:53.689723 29903 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:53.689888 30285 raft_consensus.cc:697] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:53.727293 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:53.717396 23444 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 25. Current term is 26. Ops: []
I20260709 13:09:53.728087 22826 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:53.717667 22829 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:53.718381 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:53.718662 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:53.718393 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:53.729206 23449 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:53.719761 23452 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:53.712296 22794 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.710983 30274 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:53.665248 30271 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:09:53.690038 30295 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:53.719882 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.686940 30318 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:53.716765 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:53.687927 30223 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:53.727027 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.711994 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:53.720906 30294 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:53.637390 30305 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:53.688807 30264 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:09:53.709648 29909 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:53.737943 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:53.727038 22868 connection.cc:256] RPC call timeout handler was delayed by -1.0063310630000615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 13:09:53.739320 22794 leader_election.cc:400] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19.
I20260709 13:09:53.742200 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.729151 29903 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:53.743860 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.775245 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.778786 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.730885 30313 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:53.733693 30274 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:09:53.791864 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 52827.2us
I20260709 13:09:53.783779 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.785145 30178 consensus_queue.cc:260] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 25.22, Last appended by leader: 22, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.786482 30223 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 25.22, Last appended by leader: 22, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.787436 30285 consensus_queue.cc:237] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.740605 29937 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:53.799425 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.811697 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.801206 23416 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.803092 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.804291 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.731061 30254 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:53.805075 30332 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:53.781723 30131 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:53.782051 30318 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:53.781476 30280 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:53.797456 30274 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:53.829149 23416 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:53.835613 30296 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:53.841922 30296 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:53.843768 30299 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:53.828023 30328 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:09:53.848377 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:53.820377 29977 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:53.789917 29546 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 13:09:53.843779 30332 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:53.810348 30271 consensus_queue.cc:1059] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
W20260709 13:09:53.846745 22794 consensus_peers.cc:597] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e4b3212e80646379ccf77b5952a4363. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:53.823055 30308 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:53.815307 30298 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:09:53.827510 29937 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:53.826539 30331 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:53.856408 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:53.839870 30178 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:53.845770 30287 raft_consensus.cc:2804] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:53.840281 30284 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:53.857882 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:53.896708 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.890297 22755 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:53.852381 30280 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:09:53.896515 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 93522us
I20260709 13:09:53.904122 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:53.898015 30307 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:53.908448 30131 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:53.908150 29903 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.948805 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:53.949451 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:53.911001 30301 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:53.911947 30277 consensus_queue.cc:260] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:53.914490 30326 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:09:53.966542 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:53.967355 22829 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 25.
I20260709 13:09:53.914844 30297 consensus_queue.cc:1059] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:09:53.969940 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 70425.3us
I20260709 13:09:53.939505 22755 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:53.940636 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.943425 22831 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:09:53.943297 22797 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:53.898370 30295 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:53.848129 30234 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.972466 29251 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:53.975237 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.947899 30228 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:53.979378 29251 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:53.980861 23451 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:09:53.952896 30323 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:53.981806 23451 pending_rounds.cc:85] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 13
I20260709 13:09:53.986191 23451 pending_rounds.cc:107] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:09:53.986557 23451 raft_consensus.cc:2889] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:53.899785 30284 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.982652 23453 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:09:53.983624 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 125803us
I20260709 13:09:53.946110 30131 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:53.981575 29251 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:53.965518 30307 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:53.969065 29704 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:53.989358 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:53.995662 23445 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 25.
I20260709 13:09:53.978523 30295 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:53.971325 30340 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:53.981456 30332 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:54.007468 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.978885 30318 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:54.002736 29251 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:53.898247 30313 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:53.988662 30323 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:54.017668 29251 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.003738 29704 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:53.989674 30298 consensus_queue.cc:1059] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:09:54.016855 30280 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.036144 30280 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.038934 30280 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.012100 30277 raft_consensus.cc:3060] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:54.018370 30323 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:54.041314 29251 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:54.017750 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:53.985865 29937 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:54.049698 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:54.021431 29704 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:54.020800 30287 raft_consensus.cc:697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:54.023200 30344 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.034631 30254 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.035367 30305 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.063071 30315 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:54.014626 30332 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.058270 30344 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.057868 30307 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:54.089989 30334 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.081s	user 0.001s	sys 0.004s
I20260709 13:09:54.022156 30318 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:54.091142 30330 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:09:54.060286 30306 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:54.091280 30254 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:54.113005 29251 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:09:54.114363 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.121908 22901 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:54.113976 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:54.144217 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:09:54.129230 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 317601us
I20260709 13:09:54.132977 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:54.145130 22755 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:54.145268 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:54.153527 29903 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:54.145025 30262 consensus_meta.cc:287] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.003s
W20260709 13:09:54.176400 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 203654us
I20260709 13:09:54.145754 30321 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:54.126274 30340 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:09:54.146662 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:54.157169 30287 consensus_queue.cc:237] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:54.232903 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 88780.9us
I20260709 13:09:54.238426 22755 raft_consensus.cc:2468] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
W20260709 13:09:54.241034 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 94459.4us
I20260709 13:09:54.244231 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:54.231758 30263 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.058470 30131 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.249603 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:54.252471 22749 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 17: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:54.232378 30298 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:54.243136 23419 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:54.232512 30309 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:09:54.239651 30252 consensus_meta.cc:287] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.602s	user 0.014s	sys 0.004s
I20260709 13:09:54.254945 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:54.232067 30098 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.258383 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:54.238744 30338 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:09:54.245921 30299 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:54.246150 30321 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:54.255512 30309 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.259783 30234 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.289481 30347 raft_consensus.cc:2804] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:09:54.289582 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.290086 30347 raft_consensus.cc:697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:54.274590 30284 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:54.276612 30178 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:54.256157 30263 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.299399 30347 consensus_queue.cc:237] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.266561 29546 consensus_queue.cc:1059] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:09:54.283533 29903 consensus_queue.cc:1059] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 13:09:54.282444 30309 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:54.284783 30131 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:54.307902 22751 raft_consensus.cc:1240] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 25. Current term is 26. Ops: []
I20260709 13:09:54.260054 30295 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.265471 30098 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:54.311797 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.297897 30178 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.323154 29903 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:54.323585 29903 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:54.324427 29903 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.325743 29903 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:54.330535 30263 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.332774 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:54.338514 30349 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:09:54.309718 30332 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.342720 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.349252 30178 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:54.333806 30318 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.322412 30344 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.355664 22750 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:54.317750 30360 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.358182 30295 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:54.359045 30357 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 13:09:54.361582 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 311896us
I20260709 13:09:54.352424 30321 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.360772 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:54.372519 22749 raft_consensus.cc:1240] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:54.372519 22754 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:54.396463 22900 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 23: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:54.400014 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:09:54.402763 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:54.397143 30274 consensus_meta.cc:287] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.548s	user 0.016s	sys 0.003s
I20260709 13:09:54.419961 22831 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:54.437706 30340 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:09:54.414569 30358 log.cc:927] Time spent T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:09:54.441249 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.441957 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.413836 30312 raft_consensus.cc:3055] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:09:54.439690 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 79008.9us
I20260709 13:09:54.445385 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:54.451061 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:54.378453 29903 consensus_queue.cc:1059] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 13:09:54.408536 30299 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.442284 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:54.402992 30228 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 24 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:54.452678 29546 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:54.452823 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:54.433550 30277 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.466575 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:54.461295 30323 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.441490 30332 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.468367 29251 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:09:54.451201 30331 consensus_meta.cc:287] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.008s
I20260709 13:09:54.468680 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:54.447278 30131 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:54.470777 29546 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:09:54.444747 30263 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.450588 30312 raft_consensus.cc:740] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:54.474494 30131 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.439772 30274 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.519160 29251 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:54.449118 30356 consensus_queue.cc:1059] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:09:54.415910 30360 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:54.537673 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 118907us
I20260709 13:09:54.476318 30331 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:54.518647 30228 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.470276 30277 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.477840 30332 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.467670 30299 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
W20260709 13:09:54.565165 30362 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:09:54.573215 30299 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:54.573565 30098 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:54.568162 29704 consensus_meta.cc:287] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.001s
I20260709 13:09:54.520682 30131 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:54.575548 23449 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 25. Current term is 26. Ops: []
W20260709 13:09:54.580611 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 337520us
W20260709 13:09:54.581173 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 128429us
I20260709 13:09:54.516887 30263 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:54.587311 22903 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 25. Current term is 26. Ops: []
I20260709 13:09:54.568714 30323 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.566715 30274 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.581470 29704 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.602702 30263 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.608410 22751 raft_consensus.cc:1240] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:54.615053 30332 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.615324 30308 raft_consensus.cc:3055] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:54.615658 30332 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:09:54.616394 30332 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:09:54.616719 30332 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:54.623853 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:54.623998 30277 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.624545 30312 consensus_queue.cc:260] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.625494 22829 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:54.625734 30312 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:54.626555 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.627089 22830 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:54.627698 22831 raft_consensus.cc:1240] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:54.627858 29251 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.628345 29251 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.628773 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.630081 30365 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.630731 30365 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.615674 30308 raft_consensus.cc:740] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:54.634289 30308 consensus_queue.cc:260] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.633798 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:54.635533 30308 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:54.635983 29800 raft_consensus.cc:3055] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:54.636674 29800 raft_consensus.cc:740] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:54.637985 22826 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:54.637792 29800 consensus_queue.cc:260] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:54.639137 29800 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:54.641105 30299 consensus_queue.cc:1059] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:54.636107 30360 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.642179 30299 raft_consensus.cc:3055] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:54.642517 30299 raft_consensus.cc:740] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:54.643287 30299 consensus_queue.cc:260] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:54.644644 30299 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:54.645763 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.646292 22902 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 17.
I20260709 13:09:54.601133 29704 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:54.652002 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:54.652925 30277 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:09:54.653637 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.656117 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:54.630084 29251 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.658037 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:54.663069 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:54.664788 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:54.680171 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.680301 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:54.680761 22905 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
W20260709 13:09:54.681618 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 206125us
I20260709 13:09:54.684582 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:54.687187 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.687922 22822 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:54.682479 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:54.703970 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.703999 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.706282 22899 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:54.707810 22903 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 25 index: 22. Preceding OpId from leader: term: 26 index: 23. (index mismatch)
I20260709 13:09:54.708031 29251 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.710196 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.711024 22900 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:09:54.712782 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.712916 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.713687 22828 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 17.
I20260709 13:09:54.713873 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.714380 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:54.714753 22828 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:09:54.708600 29251 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.716061 22826 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:54.666800 30365 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.716847 22823 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:09:54.718267 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:09:54.719727 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.720021 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.720242 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.690227 30263 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:54.723968 29251 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.724380 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:54.729554 22824 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 18: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:54.724458 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.724910 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.724643 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:54.724323 30364 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.731992 22829 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
I20260709 13:09:54.734961 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:54.746644 22824 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:54.746765 22830 raft_consensus.cc:2393] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 25: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:54.750586 22826 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:09:54.746675 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.717154 30242 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:54.758258 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.759821 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.760581 22903 raft_consensus.cc:2393] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 24: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:54.760748 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:54.761260 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:09:54.761611 22869 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.762149 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.763077 22897 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:54.762346 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.767976 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:54.772694 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:54.778981 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:09:54.728454 30314 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.006s	sys 0.023s
I20260709 13:09:54.784317 29251 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:54.781448 22824 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:54.785007 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:54.786553 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:54.781961 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 63759.3us
I20260709 13:09:54.787948 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:54.788954 29546 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:54.758972 30367 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:54.759136 30242 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.802345 30242 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:54.785885 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:54.789886 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:54.789436 30376 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:54.725636 30338 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.064s
I20260709 13:09:54.805243 22751 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:54.707494 30332 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.810469 30332 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:54.805194 30277 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:54.814299 30277 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.814649 30277 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:54.814584 30178 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.815716 22899 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:54.817340 30178 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.821012 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:54.822232 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:54.823377 30295 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:54.777168 30298 raft_consensus.cc:3055] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:54.824117 30298 raft_consensus.cc:740] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:54.825114 30298 consensus_queue.cc:260] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:54.826143 30298 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:54.827339 30131 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.828588 22904 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:54.831493 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:54.831651 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:54.832979 22826 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:54.833607 30310 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:09:54.792459 30304 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:54.822755 30178 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:54.833901 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44099.2us
W20260709 13:09:54.784633 30314 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.006s	sys 0.023s
I20260709 13:09:54.837488 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.838692 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.839038 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.837126 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.850311 23448 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:54.846248 23450 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 25 index: 22. Preceding OpId from leader: term: 26 index: 23. (index mismatch)
I20260709 13:09:54.758538 30364 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:54.860992 22824 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:54.850358 23446 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:54.856426 22897 raft_consensus.cc:2955] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:09:54.834643 30334 log.cc:927] Time spent T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:09:54.849681 23451 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:54.866421 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.834062 30310 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.868130 23444 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:54.834033 30263 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:54.841073 30351 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:54.879874 30131 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:54.880834 30244 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 13:09:54.883368 22755 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:09:54.883522 30309 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.000s
I20260709 13:09:54.884246 29704 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.859139 30297 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:54.884747 29704 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.885288 30297 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.876710 30310 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:54.867081 30356 raft_consensus.cc:3055] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:54.887073 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.897696 23453 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:09:54.897500 30356 raft_consensus.cc:740] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:54.888512 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.888509 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:54.884002 30309 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.890096 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:54.891525 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.858940 30295 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.834089 30277 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.889159 30351 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.875909 30263 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.903724 23452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.907431 23452 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:54.903090 29903 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:54.910473 30263 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:54.925884 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43996 (request call id 2303) took 1192 ms. Trace:
I20260709 13:09:54.929852 22896 rpcz_store.cc:276] 0709 13:09:53.733029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.733411 (+   382us) service_pool.cc:224] Handling call
0709 13:09:54.925629 (+1192218us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b540034cbd4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:09:54.925810 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788539264}
I20260709 13:09:54.908535 30351 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:54.935097 23446 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:09:54.936259 22870 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.938282 22749 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:54.941229 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:54.942137 23444 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:54.907800 30295 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:54.948119 30394 ts_tablet_manager.cc:933] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:54.925969 30356 consensus_queue.cc:260] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.954783 30356 raft_consensus.cc:3060] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:54.955928 22868 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:54.956826 30367 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:54.957291 30367 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:54.945493 30375 consensus_queue.cc:1059] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:09:54.958213 30367 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.959017 30375 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:54.959373 30375 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:54.960153 30375 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:54.982235 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:54.975138 29704 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:54.967388 30178 consensus_queue.cc:1175] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6a5191aaff76469bbf6c4735dc9e6509" responder_term: 19 status { last_received { term: 19 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 19 index: 11 } }
I20260709 13:09:54.998102 22900 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:54.972774 30308 consensus_queue.cc:1059] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:09:55.014298 30308 raft_consensus.cc:3055] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:55.014739 30308 raft_consensus.cc:740] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:55.015771 30308 consensus_queue.cc:260] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:54.965466 29909 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:09:55.000016 30309 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:55.017685 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 79091.8us
I20260709 13:09:55.019343 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:55.020005 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:55.020557 22754 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:09:55.020764 23447 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38064 (request call id 1867) took 1306 ms. Trace:
I20260709 13:09:55.005555 30295 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:54.967232 30304 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:54.909090 30131 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:09:55.021130 23447 rpcz_store.cc:276] 0709 13:09:53.714632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.714741 (+   109us) service_pool.cc:224] Handling call
0709 13:09:53.731401 (+ 16660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:53.948435 (+217034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:53.948482 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:53.949014 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:09:53.997937 (+ 48923us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:54.365186 (+367249us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b1800167320. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb749a34bc 00007fcb71cf148b 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0b455 00007fcb787f33c7 00007fcb74e059a7 00007fcb74e150bf
0709 13:09:54.365377 (+   191us) log.cc:844] Serialized 14 byte log entry
0709 13:09:54.635128 (+269751us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b1800167320. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e522 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:54.887206 (+252078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.906838 (+ 19632us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b1800167320. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e557 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:54.994535 (+ 87697us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54006329d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dfbe01 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2 00007fcb7313648c 00007fcb731358c6
0709 13:09:54.994584 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:55.020558 (+ 25974us) raft_consensus.cc:1685] finished
0709 13:09:55.020574 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:55.020714 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":555806,"prepare.queue_time_us":9373,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":3267,"raft.queue_time_us":596989,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":17763,"spinlock_wait_cycles":441602816,"thread_start_us":56834,"threads_started":2,"wal-append.queue_time_us":40589}
I20260709 13:09:55.029842 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:55.009719 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:55.035918 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:55.009967 30294 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.968106 23446 raft_consensus.cc:1240] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:09:54.971202 23416 leader_election.cc:336] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.635s (SENT)
I20260709 13:09:54.961297 30375 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:55.046525 23453 raft_consensus.cc:1240] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:55.004156 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.004626 30364 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:55.020365 30308 raft_consensus.cc:3060] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:55.007252 30297 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.024809 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:55.024978 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:55.007302 30277 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.023154 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:55.073972 22751 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:55.047606 30393 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:55.074565 30393 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:55.074939 30393 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:55.075397 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:55.079821 30299 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:55.080286 30299 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.080687 30299 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:55.050756 30223 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:55.086860 30263 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.037842 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:55.038359 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.095244 23448 raft_consensus.cc:2393] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:09:55.044684 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73508.9us
I20260709 13:09:55.046132 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:55.011672 23450 raft_consensus.cc:2393] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 26: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:55.053443 29251 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:55.054260 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:55.055346 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.049078 30309 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:55.096513 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:55.040825 30304 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:55.060284 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:55.060654 30339 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:55.013074 30242 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:55.065611 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:55.066242 22755 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:09:55.053107 30295 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:55.050486 30294 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.097761 23447 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:55.056301 22756 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:09:55.098537 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44340us
I20260709 13:09:55.102268 22827 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 21: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
W20260709 13:09:55.102368 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 64592us
I20260709 13:09:55.115592 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:55.117820 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.118378 23448 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:55.120232 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:55.120740 22897 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:09:55.108736 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:55.111379 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 51162.4us
I20260709 13:09:55.122584 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:55.123102 22824 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:55.123833 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:55.123937 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:55.124940 22831 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:55.125176 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.125797 23444 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:55.126956 29251 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:55.128145 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:55.129560 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:55.131471 22900 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:09:55.132637 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:55.132900 22896 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:09:55.129561 22904 raft_consensus.cc:1275] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:09:55.134392 22904 pending_rounds.cc:85] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 14
I20260709 13:09:55.134675 22904 pending_rounds.cc:107] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:09:55.134945 22904 raft_consensus.cc:2889] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:55.111418 30223 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:55.107806 30305 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:09:55.110519 30378 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:09:55.148746 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet dbd3d6688dfb4621b8ecfd11278d5b26: Illegal state: Replica 3b626c41600d4737a178ebab8a7ebfc2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:09:55.141211 30274 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 13:09:55.113281 30295 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:55.155318 22868 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:55.157128 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:55.158890 22897 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:55.149840 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:55.157977 29251 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:55.162134 23448 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 26.
I20260709 13:09:55.163229 22830 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:55.157248 22831 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:55.165009 23444 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:55.116744 30297 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:09:55.169603 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:55.177022 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:55.177942 29251 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:55.178284 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:55.180524 22824 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 26.
I20260709 13:09:55.161731 30371 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:09:55.148418 30378 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:55.145038 30223 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.193809 30277 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:55.148201 30393 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.195185 30295 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.167163 30178 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:55.171115 30131 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:09:55.175396 30370 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:55.175881 29909 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:09:55.185253 30263 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.184057 30367 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:09:55.114496 30309 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:55.141299 30287 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 13:09:55.193877 30347 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:55.197248 30374 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:55.117195 30339 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.132958 30310 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.205178 30223 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:55.198204 30371 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.196095 30342 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 13:09:55.190994 29977 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:55.201413 30378 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:55.248765 30371 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:55.280383 30223 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.188500 30228 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 13:09:55.253206 30242 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:55.254597 30294 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:55.255771 29977 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:55.295373 29977 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.296793 29977 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:55.257751 30309 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:55.259907 30310 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.211829 29909 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:55.297963 29909 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:09:55.298244 30405 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:09:55.300771 22754 raft_consensus.cc:1240] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:55.303153 30308 consensus_queue.cc:1059] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:09:55.304142 30308 raft_consensus.cc:3055] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:55.262347 30295 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:55.251688 30197 tablet_copy_client.cc:538] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:55.264895 22830 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:55.264050 30304 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.308540 30223 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.265887 30299 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:55.270711 30393 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.272316 30339 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:55.310521 30404 log.cc:927] Time spent T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:09:55.311000 30299 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:55.283253 30351 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.312628 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:55.313191 22750 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:55.313645 30351 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.285645 23451 raft_consensus.cc:1240] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:55.318413 22828 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:55.319211 30304 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.320304 30299 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:55.290726 30397 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.242s	user 0.005s	sys 0.000s
I20260709 13:09:55.321426 22750 raft_consensus.cc:2468] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:09:55.324851 22871 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:55.325902 30298 raft_consensus.cc:2804] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:55.326362 30298 raft_consensus.cc:697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:55.327193 30375 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.327324 30298 consensus_queue.cc:237] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:55.327734 30375 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.329591 30308 raft_consensus.cc:740] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:55.329703 30375 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.330485 30308 consensus_queue.cc:260] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.331712 30308 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:55.227404 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:55.333997 30274 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:09:55.334117 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 106820us
I20260709 13:09:55.334865 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:55.335541 30339 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:55.335991 30339 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:55.336833 30339 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.235625 30394 tablet_copy_client.cc:323] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
I20260709 13:09:55.336709 29909 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.343910 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:55.344880 22756 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:55.345062 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:55.367729 22755 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:55.345834 22749 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:55.369617 23419 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:55.371354 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:55.372573 22756 raft_consensus.cc:2410] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:09:55.373664 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:55.320793 30299 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.359022 22897 raft_consensus.cc:1275] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:09:55.361742 22904 raft_consensus.cc:1275] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:09:55.365978 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:55.378558 22897 pending_rounds.cc:85] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 15
I20260709 13:09:55.378854 22897 pending_rounds.cc:107] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:09:55.379145 22897 raft_consensus.cc:2889] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:55.245775 30374 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.386242 22748 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:55.387554 22899 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:09:55.389441 30374 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:55.390668 29909 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:55.375169 30364 consensus_queue.cc:1059] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:09:55.375301 30274 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:55.395341 30356 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:55.395884 30356 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:55.396694 30356 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.397876 30356 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:55.399825 30371 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.401010 23447 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:09:55.404287 30371 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:55.407107 30370 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:09:55.407223 30197 tablet_bootstrap.cc:492] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:09:55.365520 30375 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:55.411561 30374 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.413987 30343 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:09:55.415470 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 49632.4us
I20260709 13:09:55.415360 29251 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:09:55.420732 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 243745us
I20260709 13:09:55.427385 22755 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:55.419847 30295 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:55.433200 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.433955 23451 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:55.434767 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:55.435468 29251 consensus_queue.cc:1243] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 17.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:09:55.435941 29251 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:09:55.436223 29251 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:55.437539 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:55.437537 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.442509 22797 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:55.436903 29251 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.443897 29251 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:55.444480 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:55.445348 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:55.446349 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:55.448211 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:55.449115 22755 raft_consensus.cc:2368] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18.
I20260709 13:09:55.452546 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:55.453150 22823 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:09:55.454694 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:55.450143 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:55.420418 30304 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:09:55.455569 22719 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:55.454769 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:55.457840 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:55.458015 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:55.461164 22830 raft_consensus.cc:2410] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 22 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:09:55.458917 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.462321 23449 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:55.459332 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:55.459769 22831 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 13:09:55.470619 22840 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 4e4b3212e80646379ccf77b5952a4363 from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:09:55.471206 22840 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 4e4b3212e80646379ccf77b5952a4363 from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-4e4b3212e80646379ccf77b5952a4363
I20260709 13:09:55.471908 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.472505 23444 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:55.474424 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:55.474938 22902 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:55.476958 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.482515 23419 leader_election.cc:400] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18.
I20260709 13:09:55.482828 23419 leader_election.cc:403] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:09:55.434437 30310 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:09:55.428445 30375 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:55.486846 30363 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 13:09:55.515021 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:55.516336 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.492993 23449 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:55.493602 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:55.488780 22719 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:55.459936 30301 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:09:55.502990 22826 raft_consensus.cc:2410] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:09:55.461858 30277 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.529153 22900 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:55.507992 23444 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:55.477092 30244 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:55.475944 30314 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:55.524809 22902 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:55.528065 23450 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:55.479485 30340 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:09:55.541190 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 52460us
I20260709 13:09:55.541867 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:55.542424 22902 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:55.543618 22899 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:55.460979 30339 consensus_queue.cc:1048] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:09:55.551836 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:55.555485 30331 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:55.560842 30309 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:55.561390 30309 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:55.561453 22902 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:09:55.562422 30309 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.562872 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:55.564028 30331 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:09:55.564675 30331 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:55.565702 30331 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.568948 30299 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.525673 30343 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 26 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:55.569768 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:55.573050 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:55.569520 30244 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:55.580042 30244 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.580895 22794 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:55.581920 22831 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:55.582185 30340 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:55.531561 30287 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 17. Current term is 18.
I20260709 13:09:55.540159 30314 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.542884 30301 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:55.588197 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:55.586280 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:55.589620 23451 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:09:55.508728 30370 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
I20260709 13:09:55.598716 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:55.539773 30228 consensus_queue.cc:1048] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 13:09:55.571406 30306 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:55.583024 30340 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.548416 30098 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 13:09:55.588864 30314 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:55.548902 30342 consensus_queue.cc:1048] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:09:55.539898 30277 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.611502 29977 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:55.631784 22902 raft_consensus.cc:1240] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 13:09:55.585999 30315 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.591866 30321 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:55.718812 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:55.726423 30345 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:55.628437 30296 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:09:55.612258 30408 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:09:55.745457 30345 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.747571 30345 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:55.751343 30411 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:09:55.752622 30374 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.753298 30343 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.705444 30363 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:55.764927 30363 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.765220 30301 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.766988 30363 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:55.769819 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 153219us
I20260709 13:09:55.785398 22902 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:09:55.727252 30254 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.725792 22897 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:55.639462 30310 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.141s
W20260709 13:09:55.805686 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.806128 30304 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.806919 30304 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.810107 30375 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:55.641638 30306 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:55.812587 30306 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:55.814337 30378 consensus_meta.cc:287] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 13:09:55.814865 30378 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.817013 23448 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:55.735641 30308 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.821157 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:55.831785 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:55.832249 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:55.832381 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:55.836685 23446 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:55.834460 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:55.837643 22903 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
W20260709 13:09:55.839164 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 227494us
I20260709 13:09:55.842811 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:55.785657 30376 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.797284 30321 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:55.864041 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:55.864334 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:55.817497 30254 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.859120 30310 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 13:09:55.739807 30315 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.839864 30308 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.862159 30416 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.883181 30416 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:55.885641 30416 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:55.885143 30321 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.878410 30374 consensus_queue.cc:1059] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:55.886750 30374 consensus_queue.cc:1243] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer 3b626c41600d4737a178ebab8a7ebfc2 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:09:55.887079 30308 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.887925 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:55.879496 30277 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.880663 30254 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.888695 22754 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:55.889374 30315 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:55.890471 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:55.882613 30407 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:55.870702 30262 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:55.892197 30407 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:55.892395 30262 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:55.892761 30407 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:55.860656 30376 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.894209 30308 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:55.873116 30378 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:55.894410 30262 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:55.896632 30308 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.939913 23446 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
W20260709 13:09:55.956564 30390 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.477s	user 0.004s	sys 0.000s
I20260709 13:09:55.969950 22747 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:55.973693 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:55.974666 22753 raft_consensus.cc:2393] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 20: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:55.976188 30419 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:09:55.976728 30419 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:55.976591 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:55.979408 22749 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:55.917884 30308 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:09:55.986233 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 113069us
I20260709 13:09:56.005208 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.005831 22902 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:56.007781 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:56.008960 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.009732 22896 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:56.003537 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.011010 23450 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:56.015636 22752 raft_consensus.cc:2410] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:09:56.019210 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.019987 23450 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:55.997789 29977 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:56.026754 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:09:56.037658 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 173415us
I20260709 13:09:56.014475 30314 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.034914 30416 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:56.024685 30374 consensus_queue.cc:1059] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:09:56.044803 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:56.055249 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:56.047279 30416 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.082782 30416 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:56.061560 29977 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.065235 30314 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.016575 30419 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:56.087990 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44087.6us
I20260709 13:09:56.092057 22719 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:56.094556 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.098567 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:56.101589 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 113153us
W20260709 13:09:56.089486 30417 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.195s	user 0.000s	sys 0.003s
I20260709 13:09:56.084231 30332 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:56.105598 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.106863 22897 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:56.108763 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.109377 23448 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:56.114637 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.115527 22897 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 25.
I20260709 13:09:56.109151 30376 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.107303 30296 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:56.122346 30419 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:09:56.106485 30332 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.128577 30416 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.133543 30424 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:56.135147 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.136572 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.156764 22826 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:56.099256 30375 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.099997 30277 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:56.137836 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:56.157454 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 53459.9us
I20260709 13:09:56.130810 30407 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.151984 22823 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:56.156071 22831 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:56.155138 30332 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:56.154908 30296 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:56.188974 22828 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:56.168071 30277 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.189477 22794 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:56.167588 30308 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.191478 30277 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:56.173743 30375 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.166850 30424 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.196010 30433 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:56.223109 30424 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:56.215905 30433 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:56.215682 30365 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.010s
I20260709 13:09:56.226363 30374 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:56.226886 30374 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:56.260157 30314 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.260766 30314 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.262873 30314 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:56.264557 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:56.265210 22752 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 25.
I20260709 13:09:56.266639 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:56.227857 30374 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.268927 30308 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.269830 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:56.270375 22752 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:56.271288 30375 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:56.273448 30308 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.273986 30308 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.276376 30407 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:56.282637 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 71860.3us
I20260709 13:09:56.283217 22749 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:09:56.285125 22755 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:09:56.285619 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:56.291605 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:56.292645 22755 raft_consensus.cc:2368] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 20. Current term is 21.
I20260709 13:09:56.293520 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:56.294687 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.295491 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:56.296029 22824 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:56.296908 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.297508 22830 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:09:56.299551 22749 raft_consensus.cc:2393] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 25: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:56.303296 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:56.305166 22826 raft_consensus.cc:3055] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:09:56.305529 22826 raft_consensus.cc:740] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:56.304046 22748 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:56.308347 22824 raft_consensus.cc:2468] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 25.
I20260709 13:09:56.276410 30308 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.287142 30314 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:56.306394 22826 consensus_queue.cc:260] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.313476 22826 raft_consensus.cc:3060] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:56.315059 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:56.315721 22830 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:56.317687 22797 leader_election.cc:400] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 20. Current term is 21.
I20260709 13:09:56.318094 22797 leader_election.cc:403] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:09:56.298429 30295 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:56.296108 30444 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 13:09:56.206889 30296 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:56.321794 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 126984us
I20260709 13:09:56.322477 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:56.325752 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:56.334959 22752 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:56.336045 22871 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:56.313328 30314 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:56.338402 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 221687us
I20260709 13:09:56.340355 30407 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.340528 23450 raft_consensus.cc:3055] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:09:56.340883 30407 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.341706 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.342334 23448 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 25.
I20260709 13:09:56.343194 30407 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:56.225172 30433 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.347223 30277 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.340919 23450 raft_consensus.cc:740] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:56.352705 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:56.354121 22752 raft_consensus.cc:1240] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:09:56.368403 22826 raft_consensus.cc:1275] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:09:56.356350 22830 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:56.383407 22826 pending_rounds.cc:85] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 12
I20260709 13:09:56.383754 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:56.356884 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.356743 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:56.388819 23453 raft_consensus.cc:3060] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:56.388404 22826 pending_rounds.cc:107] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:09:56.363821 23448 raft_consensus.cc:1240] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:09:56.389451 22826 raft_consensus.cc:2889] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:56.389120 22748 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:56.321483 30308 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:09:56.394057 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:56.339684 30310 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:56.397187 30310 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.339644 30314 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:56.408362 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 122802us
I20260709 13:09:56.408725 30458 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:56.409153 30458 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.409770 30458 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:56.389667 23450 consensus_queue.cc:260] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.413762 23450 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:09:56.402640 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 66678us
I20260709 13:09:56.321394 30295 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.421989 30459 raft_consensus.cc:2804] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:56.422541 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:56.348261 30416 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:56.423206 22896 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:56.427637 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:56.441141 23453 raft_consensus.cc:2468] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:56.442054 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:56.441202 22719 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:56.427675 30310 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:56.404460 30445 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213) with higher term. Message: Invalid argument: T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 20. Current term is 21.
I20260709 13:09:56.446228 22826 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:56.448181 22896 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:56.461517 22751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56780 (request call id 2294) took 1354 ms. Trace:
I20260709 13:09:56.461896 22751 rpcz_store.cc:276] 0709 13:09:55.107458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.107581 (+   123us) service_pool.cc:224] Handling call
0709 13:09:55.110582 (+  3001us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:55.113382 (+  2800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:55.113419 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:55.148785 (+ 35366us) log.cc:844] Serialized 52 byte log entry
0709 13:09:55.417059 (+268274us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:55.429654 (+ 12595us) log.cc:844] Serialized 14 byte log entry
0709 13:09:55.442050 (+ 12396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:55.442129 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:55.582137 (+140008us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b18002f6020. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e522 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:56.442519 (+860382us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b18002f6020. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71d0e3f3 00007fcb71d107c1 00007fcb71d10779 00007fcb71d10709 00007fcb71d106d1 00007fcb71d0f9cd 000000000038f354 000000000038f199 00007fcb71cf183a
0709 13:09:56.461277 (+ 18758us) raft_consensus.cc:1685] finished
0709 13:09:56.461298 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:56.461443 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17534,"prepare.queue_time_us":12377,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":5457,"raft.queue_time_us":17318,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":22305,"spinlock_wait_cycles":15070848,"thread_start_us":267892,"threads_started":1,"wal-append.queue_time_us":327806}
I20260709 13:09:56.435298 30455 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:56.440104 30433 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:56.469130 22868 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:56.470822 30455 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:56.471969 30455 raft_consensus.cc:2804] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:56.472520 30455 raft_consensus.cc:697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:56.473582 30455 consensus_queue.cc:237] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.435840 30375 consensus_queue.cc:1059] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:09:56.476240 30375 raft_consensus.cc:3055] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:09:56.476619 30375 raft_consensus.cc:740] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:56.452209 30295 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.479688 30375 consensus_queue.cc:260] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:56.480762 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.481042 30375 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:56.486390 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.486999 22826 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:09:56.488778 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:56.492906 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.494393 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.467183 30444 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:56.498204 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.469754 30433 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.468665 30459 raft_consensus.cc:697] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:56.502279 23450 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 18. (term mismatch)
I20260709 13:09:56.503010 23450 pending_rounds.cc:85] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 17
I20260709 13:09:56.477087 30332 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.504393 30325 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:09:56.504778 30332 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.506238 30277 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.484169 30416 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 13:09:56.509299 30436 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.314s	user 0.002s	sys 0.000s
I20260709 13:09:56.502743 30444 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:56.510440 30444 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.513655 30458 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.516005 30458 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:56.516779 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.519562 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:56.520344 22826 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:56.503242 23450 pending_rounds.cc:107] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 20.18
I20260709 13:09:56.522027 23450 raft_consensus.cc:2889] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:56.522269 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:56.523661 23444 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:56.525996 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:56.526748 22751 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:09:56.501858 30433 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:56.538224 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:56.538832 23450 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:56.529871 22868 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:56.498401 30295 consensus_queue.cc:1048] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:09:56.536726 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:56.528914 22826 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:56.541013 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:56.504803 30325 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:56.547959 22896 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:56.528880 30463 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.549787 22827 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:09:56.528959 30296 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.559370 22896 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:56.549081 30459 consensus_queue.cc:237] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.562139 22840 tablet_copy_source_session.cc:215] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:56.563076 23450 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:09:56.565117 22719 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:56.553392 23444 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
I20260709 13:09:56.570254 30464 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:56.551638 30458 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:56.539769 30429 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 13:09:56.571699 22719 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:56.563356 22840 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:47110 (request call id 1542) took 1101 ms. Trace:
I20260709 13:09:56.567279 29704 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:56.581024 29704 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.583225 29704 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:56.566917 30277 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:56.583971 30277 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.584755 30340 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:09:56.585332 30340 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:56.585960 30277 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:56.586130 30340 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.586956 30325 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.588114 30325 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:09:56.566062 30296 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.592031 30296 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.595819 22797 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:56.597295 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.598948 22897 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:56.598161 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.600565 22900 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 26.
I20260709 13:09:56.602806 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.563823 30463 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.606472 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:56.607290 30463 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.573443 30464 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:56.609251 23453 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:09:56.608989 30464 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.610263 30449 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:56.610653 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.610865 30449 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.611413 22826 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:56.613981 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:56.617272 22822 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:09:56.617738 30304 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.611370 30449 raft_consensus.cc:3060] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:56.620070 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.622889 23453 raft_consensus.cc:2393] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:56.624119 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:56.625674 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.625782 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:56.626340 23444 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 19.
I20260709 13:09:56.627257 30242 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:09:56.628010 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.629927 22822 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:56.629689 30242 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:56.631238 30242 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:56.630399 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:56.575904 22840 rpcz_store.cc:276] 0709 13:09:55.461783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.470443 (+  8660us) service_pool.cc:224] Handling call
0709 13:09:56.561708 (+1091265us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b54002e0020. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74d72dc0 00007fcb74e054a5 00007fcb74e053ee 00007fcb79d9b036 00007fcb79da4a3e 00007fcb7166827b 00007fcb79da49ba 00007fcb79d9facc 00007fcb79d9ac7b 00007fcb79d8f8d1
0709 13:09:56.563264 (+  1556us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28131072}
I20260709 13:09:56.628685 30427 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:56.632856 22751 raft_consensus.cc:2468] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 25.
I20260709 13:09:56.635124 22871 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:56.639734 23453 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:09:56.640422 30263 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:56.641571 30314 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.642488 30263 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:09:56.643751 30314 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:56.639783 30223 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:09:56.637353 30295 raft_consensus.cc:2804] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:09:56.634306 30463 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:56.648697 30295 raft_consensus.cc:697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:56.645810 30223 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.652896 22897 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:56.656668 30374 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:56.656450 30295 consensus_queue.cc:237] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.658587 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:56.659231 22751 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:56.666989 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:56.667688 22822 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:09:56.679035 30461 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:56.679615 30461 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.682173 30461 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.682300 30433 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.691130 30310 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.693286 22830 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 19: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:09:56.701279 30305 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 13:09:56.705446 23448 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:09:56.658407 30463 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.709717 30463 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:56.711159 30298 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:56.658602 30223 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:09:56.676962 22822 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 26.
I20260709 13:09:56.658597 29937 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.726438 23448 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:56.678681 22751 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 26.
I20260709 13:09:56.740880 22871 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:56.742182 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.698262 30433 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.744597 30461 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:56.720809 30298 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.735261 30393 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 13:09:56.737372 29937 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.749033 30310 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.746827 30296 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:09:56.780553 29937 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.784312 30298 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.787679 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.772267 30461 raft_consensus.cc:697] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:56.788834 23448 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 25: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:56.792353 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.767371 30464 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
W20260709 13:09:56.796518 30420 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.790s	user 0.004s	sys 0.000s
I20260709 13:09:56.798656 30314 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.800177 30314 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:56.804207 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.805927 22825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1539) took 1347 ms. Trace:
I20260709 13:09:56.806429 22825 rpcz_store.cc:276] 0709 13:09:55.458011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.458191 (+   180us) service_pool.cc:224] Handling call
0709 13:09:55.458475 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:55.569826 (+111351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:55.569872 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:55.585107 (+ 15235us) log.cc:844] Serialized 26 byte log entry
0709 13:09:55.896730 (+311623us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:09:55.896763 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:55.896826 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:56.795836 (+899010us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:56.805579 (+  9743us) raft_consensus.cc:1685] finished
0709 13:09:56.805608 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:56.805861 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22198,"raft.queue_time_us":34802,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":14825216,"thread_start_us":357311,"threads_started":2,"wal-append.queue_time_us":353609}
I20260709 13:09:56.809669 30314 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:56.813656 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.811993 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.835480 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:56.835789 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.824851 30314 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:56.838949 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:56.814862 22899 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:09:56.812481 22896 raft_consensus.cc:1217] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Deduplicated request from leader. Original: 20.12->[21.13-21.13]   Dedup: 21.13->[]
I20260709 13:09:56.806916 30461 consensus_queue.cc:237] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.840190 22896 raft_consensus.cc:1240] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:09:56.830631 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:56.842479 23448 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:56.831929 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:56.833614 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:56.843195 23453 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:56.834223 30242 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.817672 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:56.844909 22871 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:56.851967 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:56.839197 30314 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:56.869037 22898 raft_consensus.cc:2368] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 20. Current term is 21.
I20260709 13:09:56.882690 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:56.870936 30424 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:56.915026 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:56.933277 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.966389 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:56.967114 22825 raft_consensus.cc:2393] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:56.967051 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:56.967916 30449 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:56.969026 22824 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 25: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:56.969511 22828 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:56.965724 30242 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:57.006422 30478 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:57.052556 22828 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 24. Current term is 25. Ops: []
I20260709 13:09:57.053112 30424 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:57.053596 30274 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:57.054121 30274 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.055238 30424 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:57.056231 30314 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:09:57.056588 30274 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:57.057181 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:57.058717 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:09:57.033895 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 162907us
I20260709 13:09:57.036895 30376 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.967085 22905 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:57.038247 30356 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:57.038551 30223 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.039193 22831 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 25.
I20260709 13:09:57.039825 30446 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:09:57.040933 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.042548 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:57.112366 22898 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:09:57.046569 22902 raft_consensus.cc:1240] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 13:09:57.047292 22871 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:57.048497 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:57.048802 23453 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 26.
I20260709 13:09:57.047557 30325 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:57.081609 23450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:57.082262 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:57.085245 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:57.089426 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:57.091536 22825 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:09:57.091696 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:57.119961 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 72721us
I20260709 13:09:57.121608 22901 raft_consensus.cc:2393] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:57.036628 30463 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.113512 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:57.116117 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:57.155042 22902 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:57.116907 23450 raft_consensus.cc:606] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:57.117710 22825 pending_rounds.cc:85] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 15
I20260709 13:09:57.156466 23450 raft_consensus.cc:614] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:57.156571 22825 pending_rounds.cc:107] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:09:57.157081 22825 raft_consensus.cc:2889] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:57.117429 22752 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:09:57.126240 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:57.127375 30298 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:09:57.137593 23448 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:09:57.140353 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58173.1us
I20260709 13:09:57.104112 30451 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:57.143963 30485 log.cc:927] Time spent T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:09:57.098953 30455 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:57.152168 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 60778.8us
I20260709 13:09:57.187469 22903 raft_consensus.cc:2368] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21.
I20260709 13:09:57.163614 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:57.164885 22902 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:09:57.157696 30339 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:57.191488 22898 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 26.
I20260709 13:09:57.171140 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:57.173141 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:57.169591 30416 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:09:57.142835 30223 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:57.180864 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 76131us
I20260709 13:09:57.190155 23448 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:09:57.189090 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:57.117879 30461 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.196877 22905 raft_consensus.cc:2393] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 19: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:57.220552 22898 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:09:57.221953 22898 pending_rounds.cc:85] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 14
I20260709 13:09:57.222304 22898 pending_rounds.cc:107] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:09:57.222554 22898 raft_consensus.cc:2889] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:57.200961 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:57.222961 23450 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:09:57.227205 22898 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:57.105489 30376 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.185793 30242 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:09:57.245707 30242 consensus_queue.cc:1243] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:09:57.246212 23446 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:57.137501 30446 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:57.112483 30356 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:57.251011 23451 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:57.272743 22748 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 26. Current term is 27. Ops: []
I20260709 13:09:57.221534 30449 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:57.161191 30325 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.253773 30461 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.265038 30446 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:57.194001 30378 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:57.274533 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:57.194561 30434 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.273466 22794 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:57.294023 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:57.221302 30298 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:57.282945 22902 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:09:57.285297 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:57.345263 22904 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:09:57.271139 30440 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:09:57.292771 22755 raft_consensus.cc:2393] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 25: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:57.296113 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:57.297104 22822 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:09:57.187378 30451 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.354493 22822 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:57.297621 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:57.347482 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:57.356343 22905 raft_consensus.cc:2393] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 22: Already voted for candidate 6a5191aaff76469bbf6c4735dc9e6509 in this term.
I20260709 13:09:57.246443 30274 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:09:57.194383 30455 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.295761 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:57.250952 30287 consensus_queue.cc:1059] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 13:09:57.293007 30444 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:09:57.293279 30492 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:09:57.317147 30295 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:57.318500 30356 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:57.320441 30325 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.321602 30463 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.324208 22831 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:09:57.325824 30376 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.328815 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:57.329772 30394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4b3212e80646379ccf77b5952a4363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:57.333827 22901 raft_consensus.cc:2410] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for term 22 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:09:57.334766 30461 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:57.339552 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.340430 22794 leader_election.cc:400] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 20. Current term is 21.
W20260709 13:09:57.413759 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 99881us
I20260709 13:09:57.415493 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:57.415921 30378 raft_consensus.cc:3055] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:09:57.416282 22794 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:57.416985 29977 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:09:57.297616 22826 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:09:57.417485 29977 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:57.357350 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:57.357915 22829 raft_consensus.cc:1217] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 15.13->[16.14-19.16]   Dedup: 16.14->[17.15-19.16]
I20260709 13:09:57.418617 22829 pending_rounds.cc:85] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 14
I20260709 13:09:57.418887 22829 pending_rounds.cc:107] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:09:57.418428 29977 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.419121 22829 raft_consensus.cc:2889] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:57.420053 30374 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:57.358173 30424 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:57.357368 22823 raft_consensus.cc:2393] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:57.339617 30315 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:09:57.421051 30424 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.391260 22825 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:57.285897 30242 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 13:09:57.397452 30451 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.392299 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:57.398020 22750 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:09:57.416293 30378 raft_consensus.cc:740] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:57.383745 30449 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 13:09:57.400233 22749 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:09:57.400923 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:57.382884 30393 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:09:57.433862 30378 consensus_queue.cc:260] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.434521 30393 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:57.435016 30378 raft_consensus.cc:3060] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:57.435400 30393 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:57.404238 30263 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:09:57.425453 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:57.362609 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:57.427392 22905 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:09:57.428525 22904 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 27.
I20260709 13:09:57.356649 30434 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.439735 30461 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:57.440371 30458 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:57.361197 30298 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.444367 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:57.438349 30277 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:09:57.431870 30435 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:09:57.463616 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:57.464383 22904 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:57.467442 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:57.468487 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:57.469048 23444 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:57.442955 30263 consensus_queue.cc:1243] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer 7aad11e70ca94fce8f672db05f1c28c3 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:09:57.477147 30298 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.524551 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:57.529058 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:57.560861 30371 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.561434 30371 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.477679 30418 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:09:57.481731 30427 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:09:57.645339 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 81709.5us
I20260709 13:09:57.646724 23416 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:57.651870 30339 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:09:57.659129 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:57.664266 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:57.674659 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 380679us
I20260709 13:09:57.675555 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:57.685724 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.697705 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:57.662019 30418 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:09:57.707842 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:57.709115 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:57.710021 23448 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:09:57.693135 30287 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:57.714733 22905 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:09:57.700116 30461 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:09:57.697341 30492 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:57.717793 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:57.724360 23447 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:57.735585 23448 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
W20260709 13:09:57.741263 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 192645us
I20260709 13:09:57.712414 30455 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.699855 30298 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.024s
I20260709 13:09:57.718473 30461 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:57.753654 22825 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
W20260709 13:09:57.765853 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 283134us
I20260709 13:09:57.760807 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:57.771121 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 78801.2us
I20260709 13:09:57.769418 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:57.726653 30494 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:09:57.780364 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:57.737290 30493 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 13:09:57.738332 30492 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.760746 30455 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.757268 30461 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:09:57.795855 29977 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.796027 30287 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:57.796401 29977 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.796785 30287 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:09:57.798491 29977 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.798694 30424 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.789968 30371 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:57.792387 30304 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:09:57.817997 22900 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.005s	sys 0.011s
W20260709 13:09:57.818338 22900 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.005s	sys 0.012s
I20260709 13:09:57.821599 30375 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:57.792917 30455 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.760246 30418 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.820674 30304 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.856715 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:57.858335 30498 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:09:57.861644 30304 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:57.866811 30455 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.869521 30492 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.880458 23453 raft_consensus.cc:1275] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 17. (term mismatch)
I20260709 13:09:57.880833 23453 pending_rounds.cc:85] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Aborting all ops after (but not including) 16
I20260709 13:09:57.881038 23453 pending_rounds.cc:107] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 13:09:57.881248 23453 raft_consensus.cc:2889] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:57.874150 30495 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:09:57.887584 23446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:40298
I20260709 13:09:57.888302 23446 raft_consensus.cc:606] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:57.888631 23446 raft_consensus.cc:614] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:09:57.888801 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 128081us
I20260709 13:09:57.889519 22794 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:57.897209 23446 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:57.899022 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:57.884464 30511 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:09:57.903398 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:57.893038 30376 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:57.910466 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:57.911947 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:57.895661 30419 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:57.890877 30378 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:09:57.914603 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:57.917737 22831 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:09:57.915658 22823 raft_consensus.cc:1275] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:09:57.910880 30503 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:57.937623 30508 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:09:57.954483 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:57.955201 22823 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
W20260709 13:09:57.955915 30500 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.153s	user 0.002s	sys 0.000s
I20260709 13:09:57.958706 22828 pending_rounds.cc:85] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting all ops after (but not including) 13
I20260709 13:09:57.959098 22828 pending_rounds.cc:107] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:09:57.959399 22828 raft_consensus.cc:2889] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:57.927357 30503 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:57.962181 30479 log.cc:927] Time spent T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.815s	user 0.000s	sys 0.006s
I20260709 13:09:57.993808 30304 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:57.995015 30503 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:57.996431 22825 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 24: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:09:57.997897 30512 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:09:57.997812 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.004467 30394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38090 (request call id 1982) took 3120 ms. Trace:
I20260709 13:09:58.006246 30375 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.006824 30304 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.006733 30375 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.014578 30393 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.015004 30375 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.015070 30393 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:58.009760 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:58.016542 22899 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:58.017084 30393 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:58.017486 22898 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:58.018445 30263 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:58.018960 30263 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:58.019248 30416 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:09:58.019496 30408 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:09:58.020916 30263 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:58.029811 23416 leader_election.cc:400] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21.
I20260709 13:09:58.030234 23416 leader_election.cc:403] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:58.036887 22902 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 26. Current term is 27. Ops: [26.20-26.20]
I20260709 13:09:58.046044 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:58.048077 22753 raft_consensus.cc:1275] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:58.057659 30378 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:09:58.058761 30378 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:09:58.059185 30378 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 26 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:58.060197 30378 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.062009 30378 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:09:58.070200 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44805.2us
I20260709 13:09:58.040272 30435 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.044837 30371 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797) with higher term. Message: Invalid argument: T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21.
I20260709 13:09:58.079488 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:58.084265 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:58.084703 22868 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:58.086622 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:58.099534 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:58.109519 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:58.122166 30434 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.129771 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:58.135371 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.147397 30378 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:58.148066 30378 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.044914 30438 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
W20260709 13:09:58.177285 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 76017.6us
I20260709 13:09:58.181128 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.190158 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.085040 30435 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:58.187546 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:58.111874 30298 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:58.113309 30455 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:58.185042 30295 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.179019 30418 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:58.097725 30375 consensus_queue.cc:1048] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.101s
I20260709 13:09:58.211190 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:58.190706 30314 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:58.111346 30393 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.093s
I20260709 13:09:58.004894 30394 rpcz_store.cc:276] 0709 13:09:54.883641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:54.885501 (+  1860us) service_pool.cc:224] Handling call
0709 13:09:54.948585 (+ 63084us) ts_tablet_manager.cc:934] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
0709 13:09:57.199190 (+2250605us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:57.864561 (+665371us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:57.864676 (+   115us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:58.003743 (+139067us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:58.003996 (+   253us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:58.004335 (+   339us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:58.004391 (+    56us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":60164,"tablet-copy.queue_time_us":61929,"thread_start_us":156683,"threads_started":2}
I20260709 13:09:58.207177 30514 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.205370 30455 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.205224 30491 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:58.218837 30378 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:58.195549 30458 consensus_queue.cc:1048] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.029s
W20260709 13:09:58.236320 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 138693us
I20260709 13:09:58.231545 30314 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:09:58.242565 30314 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.194706 30438 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.218232 30463 consensus_queue.cc:1048] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:09:58.237406 30394 tablet_copy_client.cc:806] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:58.254594 22823 raft_consensus.cc:1275] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:09:58.232039 30455 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:58.250608 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:58.277987 23446 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:58.218602 30295 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.222844 30418 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.223740 30461 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.291667 22752 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
I20260709 13:09:58.300388 22898 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:09:58.305195 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:58.306015 22756 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 18.
I20260709 13:09:58.310707 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:58.234220 30514 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:58.318626 30491 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:58.319185 30491 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:58.326495 22898 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:09:58.331560 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:58.332170 22901 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:09:58.309294 30394 tablet_copy_client.cc:670] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
W20260709 13:09:58.385708 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 183334us
I20260709 13:09:58.386590 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:58.396019 30496 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:58.396528 30496 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:58.396973 30496 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:58.406098 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:09:58.412096 30496 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.437897 30496 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.438660 30496 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:58.443974 30496 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:58.444443 30496 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:58.321259 30491 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:58.484584 23419 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:09:58.512132 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 296931us
I20260709 13:09:58.360128 30298 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:58.469033 30375 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:58.491879 30489 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:09:58.515525 22794 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:58.526196 30345 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:58.524857 30298 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:58.608461 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:58.618263 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:58.574265 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:58.563450 22794 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:58.611323 30461 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.617017 30298 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:58.531396 30533 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.133s
I20260709 13:09:58.532311 30375 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:58.569661 30526 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:09:58.661983 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:58.593606 30524 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.367s	user 0.001s	sys 0.000s
I20260709 13:09:58.554284 30376 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:09:58.608589 30458 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:58.673070 30458 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:58.675365 30458 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:58.610481 30418 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:58.574930 30521 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:09:58.598470 30511 consensus_meta.cc:287] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.003s
I20260709 13:09:58.678126 30511 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:58.678813 23444 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:09:58.618522 30491 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:58.680011 30491 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:58.676052 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:58.681438 22748 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:09:58.682036 30491 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:58.589922 30514 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:09:58.600744 30528 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:09:58.682994 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:58.684056 30491 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:58.606076 30438 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.684566 30491 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:58.685289 30491 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:58.686246 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:58.695237 30491 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:09:58.696961 22652 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 344) took 14694 ms (client timeout 15000 ms). Trace:
W20260709 13:09:58.707242 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 144328us
W20260709 13:09:58.708992 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 433373us
I20260709 13:09:58.710143 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:09:58.527434 30263 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.009s	sys 0.023s
I20260709 13:09:58.588428 30345 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.607103 30295 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:58.703912 22652 rpcz_store.cc:269] 0709 13:09:44.002141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.020799 (+ 18658us) service_pool.cc:224] Handling call
0709 13:09:58.696882 (+14676083us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:54.271198 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:54.271803 (+   605us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:54.271852 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:09:54.271871 (+    19us) tablet.cc:662] Decoding operations
0709 13:09:54.425482 (+153611us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:54.442693 (+ 17211us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:54.445109 (+  2416us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:09:54.631647 (+186538us) tablet.cc:701] Row locks acquired
0709 13:09:54.631698 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:09:54.631873 (+   175us) write_op.cc:270] Start()
0709 13:09:54.632018 (+   145us) write_op.cc:276] Timestamp: P: 1783602594631818 usec, L: 0
0709 13:09:54.632040 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:09:54.632751 (+   711us) log.cc:844] Serialized 8561 byte log entry
0709 13:09:55.250896 (+618145us) op_driver.cc:464] REPLICATION: finished
0709 13:09:55.314714 (+ 63818us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb71cf10ac 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb78831ef0
0709 13:09:55.470989 (+156275us) write_op.cc:301] APPLY: starting
0709 13:09:55.471085 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:56.108862 (+637777us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:56.135522 (+ 26660us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:56.510243 (+374721us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:56.510565 (+   322us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:09:56.510610 (+    45us) write_op.cc:312] APPLY: finished
0709 13:09:56.511005 (+   395us) log.cc:844] Serialized 196 byte log entry
0709 13:09:56.543095 (+ 32090us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:56.543367 (+   272us) write_op.cc:454] Released schema lock
0709 13:09:56.543456 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:56.544017 (+   561us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:56.544050 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1519203,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":214635,"apply.run_cpu_time_us":38844,"apply.run_wall_time_us":1072922,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96087,"num_ops":22,"prepare.queue_time_us":246140,"prepare.run_cpu_time_us":39525,"prepare.run_wall_time_us":579942,"raft.queue_time_us":214053,"raft.run_cpu_time_us":41627,"raft.run_wall_time_us":226542,"replication_time_us":580538,"spinlock_wait_cycles":72919424,"thread_start_us":635141,"threads_started":5,"wal-append.queue_time_us":309962,"wal-append.run_cpu_time_us":34680,"wal-append.run_wall_time_us":2281447}]]}
I20260709 13:09:58.742238 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:58.748266 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:58.748445 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:58.748909 22898 raft_consensus.cc:1240] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:58.760681 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:58.760959 22903 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:09:58.761409 23449 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:09:58.762117 22797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.008s	sys 0.032s
W20260709 13:09:58.762440 22797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.008s	sys 0.032s
W20260709 13:09:58.762657 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 525795us
I20260709 13:09:58.763361 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:58.638981 30435 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:58.735466 30393 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:58.734942 30463 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:58.740746 30345 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:58.773636 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:58.787487 23451 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:09:58.799983 22753 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:09:58.801831 22898 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:09:58.791617 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:58.808284 22750 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 18.
I20260709 13:09:58.760591 30301 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:09:58.820891 22794 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:58.824625 22829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:09:58.825366 22829 raft_consensus.cc:606] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:09:58.760881 30375 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.757972 30453 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:58.798401 30416 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:58.799322 30491 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:58.814049 30519 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:58.761708 30445 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:09:58.846413 30453 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.853880 30453 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:58.854848 30455 consensus_meta.cc:287] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.000s
I20260709 13:09:58.824460 30301 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:58.855402 30455 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.856154 30301 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:58.856787 22899 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 24: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:09:58.844455 30321 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:09:58.872617 30435 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:58.874229 30491 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:58.815443 30463 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.874755 30491 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.876158 30463 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.850397 30416 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.878716 30519 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:58.893181 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 365627us
I20260709 13:09:58.826977 23449 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 24.
I20260709 13:09:58.899873 30343 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:58.829453 30374 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:09:58.877245 30491 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.908933 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:58.893718 30504 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.910763 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.911789 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.916648 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.914477 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:58.912065 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.913134 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.919526 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:58.919804 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:58.920238 22825 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 18.
I20260709 13:09:58.900727 30343 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.922833 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:09:58.923096 22794 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:58.924155 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:58.927177 22794 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:58.933262 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:58.933799 22902 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:58.935977 23449 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:09:58.931162 30515 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:09:58.946746 22902 raft_consensus.cc:2468] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:09:58.907976 30496 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.930085 30514 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
W20260709 13:09:58.947652 30492 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.034s	sys 0.052s
I20260709 13:09:58.923717 30504 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:58.943786 30305 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.987677 30197 tablet_bootstrap.cc:492] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=2 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:09:58.988999 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 459776us
I20260709 13:09:58.975225 30455 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:58.974761 30515 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:58.995608 30305 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:59.002250 30301 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.021898 30416 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:59.026546 30549 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:09:59.038827 30519 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.029110 30515 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:59.080420 22754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.010s	sys 0.010s
W20260709 13:09:59.081899 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 278602us
W20260709 13:09:59.080476 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 136929us
I20260709 13:09:59.089721 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.092696 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38478 (request call id 3771) took 1059 ms. Trace:
I20260709 13:09:59.093230 22755 rpcz_store.cc:276] 0709 13:09:58.032840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.033031 (+   191us) service_pool.cc:224] Handling call
0709 13:09:58.033384 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:58.304578 (+271194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:58.304619 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:58.646791 (+342172us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b2c00027958. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:09:58.647165 (+   374us) log.cc:844] Serialized 26 byte log entry
0709 13:09:58.941012 (+293847us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:09:58.941065 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:58.941146 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:59.091896 (+150750us) raft_consensus.cc:1685] finished
0709 13:09:59.091952 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:59.092437 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50981,"raft.queue_time_us":90495,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":25382272,"thread_start_us":512574,"threads_started":2,"wal-append.queue_time_us":363902}
W20260709 13:09:59.101497 22754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.011s	sys 0.010s
W20260709 13:09:59.105891 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 157952us
I20260709 13:09:58.975776 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:59.115368 22831 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:09:59.108678 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:09:59.138695 29251 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:59.143052 30504 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.138901 30304 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.159870 30378 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.156268 30445 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 13:09:59.162382 30378 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.164057 30305 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:59.145251 30301 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.145606 30343 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:59.155469 30435 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:09:59.178267 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:59.183799 30197 tablet_bootstrap.cc:492] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:09:59.187074 30378 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:59.194839 30510 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.198875 30558 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.152642 30496 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:59.204634 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 121548us
I20260709 13:09:59.205122 30455 consensus_queue.cc:1048] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:09:59.204061 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:59.222501 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:59.224457 29251 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.225265 29251 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.234328 29251 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.155874 30503 consensus_queue.cc:1059] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.082s
W20260709 13:09:59.241838 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 63785.3us
I20260709 13:09:59.252617 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.258433 30301 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:59.258913 30301 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:59.259294 30301 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:59.259634 30503 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:59.260118 30503 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:09:59.265308 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 61685.9us
I20260709 13:09:59.266222 30435 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.272831 30435 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.275709 30435 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:09:59.278496 30571 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.145s	user 0.001s	sys 0.002s
I20260709 13:09:59.281105 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:59.281515 30510 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:59.282171 30576 log.cc:927] Time spent T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.152s	user 0.002s	sys 0.000s
I20260709 13:09:59.285466 30458 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:09:59.286330 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:59.287693 30557 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:09:59.287734 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:59.288913 23445 raft_consensus.cc:2393] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 25: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:09:59.287765 22871 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: c45e1805f049467697a7358fb566de2f
W20260709 13:09:59.290585 30573 log.cc:927] Time spent T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.162s	user 0.002s	sys 0.000s
I20260709 13:09:59.291272 30458 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:59.291656 30458 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:59.292078 30458 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:59.293488 22794 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:09:59.295146 30305 raft_consensus.cc:2804] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:09:59.295722 30305 raft_consensus.cc:697] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:59.297051 30305 consensus_queue.cc:237] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:59.298297 30197 ts_tablet_manager.cc:1403] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 3.891s	user 0.049s	sys 0.055s
I20260709 13:09:59.302876 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.302650 30558 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.261509 30503 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:59.344774 22822 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 25.
W20260709 13:09:59.304836 30569 log.cc:927] Time spent T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.234s	user 0.000s	sys 0.002s
I20260709 13:09:59.316313 30510 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:59.322886 30558 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:59.330332 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:59.321132 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.334398 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.335659 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:59.337589 30562 log.cc:927] Time spent T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.296s	user 0.000s	sys 0.003s
W20260709 13:09:59.339790 30572 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.289s	user 0.001s	sys 0.002s
I20260709 13:09:59.339648 30197 raft_consensus.cc:359] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:59.340332 30517 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:59.340075 22653 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 15 to 16, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 16 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.342222 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.267467 30304 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:59.365289 30564 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.328s	user 0.002s	sys 0.000s
I20260709 13:09:59.366056 30304 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.367211 22827 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47110 (request call id 1552) took 1186 ms. Trace:
I20260709 13:09:59.367295 22653 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:09:59.367712 22827 rpcz_store.cc:276] 0709 13:09:58.180192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.180305 (+   113us) service_pool.cc:224] Handling call
0709 13:09:58.180596 (+   291us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:58.203672 (+ 23076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:58.203715 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:58.293164 (+ 89449us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b1800495460. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb71cdc2a6 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a
0709 13:09:58.760492 (+467328us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b2c00000e98. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb736f4a11 00007fcb73704372 00007fcb74e36a01 00007fcb74dfb43a 00007fcb74dfa83e 00007fcb79dbf419 00007fcb73336bd4 00007fcb73336b62 00007fcb73336a91 00007fcb73336a0c 00007fcb73335cc2
0709 13:09:58.814968 (+ 54476us) log.cc:844] Serialized 52 byte log entry
0709 13:09:59.114050 (+299082us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:09:59.114174 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:59.114302 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:59.364433 (+250131us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:09:59.366819 (+  2386us) raft_consensus.cc:1685] finished
0709 13:09:59.366840 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:59.366993 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31452,"raft.queue_time_us":24276,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":8114,"spinlock_wait_cycles":304009088,"thread_start_us":265488,"threads_started":1,"wal-append.queue_time_us":190173}
I20260709 13:09:59.376259 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.376325 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.362624 30197 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:09:59.387902 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.394945 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:59.402187 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54886.9us
W20260709 13:09:59.415397 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 66668.4us
I20260709 13:09:59.405628 30339 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:59.415956 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 128467us
I20260709 13:09:59.421672 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:59.418977 30339 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.438309 30496 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:09:59.459712 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44384.4us
I20260709 13:09:59.414125 30197 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:09:59.441100 30339 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:59.464846 29251 raft_consensus.cc:993] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:09:59.466696 29251 raft_consensus.cc:1081] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Signalling peer 3b626c41600d4737a178ebab8a7ebfc2 to start an election
I20260709 13:09:59.429472 30301 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:59.464684 30345 consensus_meta.cc:287] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.596s	user 0.000s	sys 0.023s
I20260709 13:09:59.483390 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:59.480394 30345 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.487864 23453 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:09:59.488889 30503 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:59.489001 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:59.490959 22868 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:59.491917 30492 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:59.492594 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:59.493084 30492 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:09:59.493978 30491 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:09:59.488883 30345 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:59.499122 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.462277 22653 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 14 to 17, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 17 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:09:59.493928 30582 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:09:59.516898 30584 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:59.518823 22903 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:09:59.521014 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.521592 29251 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.522312 29251 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.522944 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.524911 29251 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.526317 30491 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:59.527052 30491 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:59.527925 30565 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:59.528880 30565 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.529340 30424 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:59.531227 30565 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.531586 22899 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:09:59.531895 30374 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.532356 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:09:59.532615 30374 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:09:59.532934 30304 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.533697 30304 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.535022 30374 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:59.535543 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.536021 30304 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.536258 22896 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 25.
W20260709 13:09:59.537307 30583 log.cc:927] Time spent T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:09:59.540227 30491 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:09:59.553717 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 140054us
I20260709 13:09:59.555835 22653 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 17 to 19, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 19 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.558162 30517 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:09:59.558859 30517 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.558907 22653 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 20 to 21, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 21 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.560659 30301 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:59.561527 30394 tablet_copy_client.cc:538] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:59.564514 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.566824 30197 ts_tablet_manager.cc:1434] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.268s	user 0.002s	sys 0.008s
W20260709 13:09:59.567721 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 75303.3us
I20260709 13:09:59.568331 30371 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.569672 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.570673 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.561030 30517 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:59.578712 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.580615 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:09:59.580768 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 154381us
I20260709 13:09:59.580917 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.581384 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:59.581835 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:59.586112 22748 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:59.582417 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.584892 23444 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 25.
I20260709 13:09:59.568935 30371 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.599589 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.607259 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:59.607946 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:09:59.616084 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.621552 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.622519 22822 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: []
I20260709 13:09:59.629132 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.630321 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 26 index: 23 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.609941 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:59.633016 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.633747 22899 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 23. Current term is 24. Ops: []
I20260709 13:09:59.604050 22653 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 13 to 18, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 18 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.645983 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:59.647673 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.624424 30455 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:59.601186 30458 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.666494 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.668625 30444 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:59.669593 30439 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:09:59.673442 30464 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:09:59.674425 30371 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:59.675047 30371 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:09:59.675315 30515 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.675951 30515 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.678094 30515 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.678241 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:59.679809 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:09:59.669049 22824 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:59.675402 30371 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.684866 30371 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:59.698410 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:59.701642 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.702616 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 26 index: 23 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.702900 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.704110 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.704861 22898 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 27.
I20260709 13:09:59.705641 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.707019 30461 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:59.707762 30461 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.708372 30458 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.709179 30455 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:09:59.711513 30455 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
W20260709 13:09:59.712375 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 120999us
I20260709 13:09:59.710160 30371 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.719401 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:09:59.720016 30371 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:09:59.721998 22897 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:43998
I20260709 13:09:59.726699 22899 raft_consensus.cc:1240] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:59.712385 22896 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22. Ops: []
I20260709 13:09:59.737159 22748 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:09:59.678799 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:59.747138 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:09:59.744414 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.761636 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:09:59.669065 30444 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:59.789655 22797 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
W20260709 13:09:59.745657 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 136377us
I20260709 13:09:59.800504 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:59.802138 23447 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 24: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:09:59.793848 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:09:59.807161 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.814046 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:09:59.816588 30563 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:09:59.816984 30563 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:09:59.817409 30563 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:09:59.826416 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:09:59.827378 30563 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.791589 30444 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:09:59.836517 30323 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 13:09:59.838074 30563 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.843039 23445 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:09:59.845541 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:09:59.803617 30304 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:09:59.831936 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.854715 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:09:59.841667 22653 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.872870 22897 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:59.846902 30371 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.847412 30455 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:09:59.873289 22897 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:09:59.883975 29251 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:09:59.884357 29251 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:59.884662 29251 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:09:59.913098 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 107802us
W20260709 13:09:59.915673 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 70207.7us
I20260709 13:09:59.874238 30565 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.876689 30461 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.933615 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
W20260709 13:09:59.942009 30589 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:09:59.943408 30590 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:09:59.946123 30565 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.946894 30444 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:09:59.948839 30598 log.cc:927] Time spent T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:09:59.949338 30581 consensus_queue.cc:1059] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.102s
I20260709 13:09:59.950165 30581 raft_consensus.cc:3055] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:09:59.950505 30581 raft_consensus.cc:740] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:09:59.950721 30444 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:09:59.951436 30581 consensus_queue.cc:260] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.952880 30581 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:09:59.963538 29251 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:09:59.967070 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 189636us
I20260709 13:09:59.967785 30511 consensus_queue.cc:1059] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:09:59.967903 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:09:59.968407 30558 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:09:59.873165 30301 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:09:59.970060 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:59.964679 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:09:59.886132 30574 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:09:59.978225 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 161410us
I20260709 13:09:59.983747 29251 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:09:59.985114 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:09:59.969024 30558 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.999281 30304 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:09:59.918220 30371 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.981226 30461 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:09:59.935904 30455 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:09:59.989431 30517 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:10:00.058104 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.069201 30496 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:00.073086 30517 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:00.124590 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 205677us
I20260709 13:10:00.147962 30619 consensus_queue.cc:1059] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:10:00.148993 30314 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:00.148730 30304 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:00.056042 30574 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.127228 30517 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:10:00.091465 30458 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:00.177384 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 259763us
I20260709 13:10:00.154175 30343 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:00.183828 30588 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
W20260709 13:10:00.195559 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 215952us
I20260709 13:10:00.195647 30604 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:10:00.196769 30592 log.cc:927] Time spent T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:10:00.202782 30560 log.cc:927] Time spent T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:10:00.207088 30603 log.cc:927] Time spent T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260709 13:10:00.212219 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 85839.8us
I20260709 13:10:00.218760 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 27 candidate_status { last_received { term: 26 index: 23 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.247285 30605 raft_consensus.cc:3055] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:00.247748 30605 raft_consensus.cc:740] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:00.248940 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:00.177634 30496 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.229518 30510 raft_consensus.cc:3055] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:00.248610 30605 consensus_queue.cc:260] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.189829 30314 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:00.249653 30565 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:00.241905 30618 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:00.272130 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:00.256170 30510 raft_consensus.cc:740] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:00.271308 22897 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.260211 30464 raft_consensus.cc:2764] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:10:00.312487 22898 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:00.272176 30619 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:00.250765 30604 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.329391 22899 raft_consensus.cc:1240] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 21. Current term is 22. Ops: []
W20260709 13:10:00.322376 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46924us
I20260709 13:10:00.303262 30371 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:00.327098 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 76330.5us
I20260709 13:10:00.305388 30605 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:00.309582 30614 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:00.347311 30624 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.347867 30624 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.350441 30624 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:00.352123 22749 raft_consensus.cc:3055] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:00.352541 22749 raft_consensus.cc:740] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:00.353457 22749 consensus_queue.cc:260] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:00.354430 22749 raft_consensus.cc:3060] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:00.361935 30574 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:00.363014 30619 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.368076 30394 tablet_bootstrap.cc:492] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
I20260709 13:10:00.309885 30565 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:00.370339 30496 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:00.370096 30565 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.312475 30491 raft_consensus.cc:2764] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:10:00.375294 30619 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:00.311959 30617 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.389197 30617 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.391265 30617 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:00.391471 30503 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:00.391909 30503 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:00.391937 22653 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:00.392951 30503 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.394300 30503 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:00.394430 22653 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 16 to 18, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 18 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:10:00.400285 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.402994 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:10:00.385293 22905 raft_consensus.cc:1240] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:10:00.320986 30618 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.412062 30618 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:00.311344 30343 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:00.412744 30343 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.414587 22897 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:00.414659 30343 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:00.416433 22749 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
W20260709 13:10:00.416813 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73821.6us
I20260709 13:10:00.417718 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.419926 30599 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:00.416826 22749 pending_rounds.cc:85] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 11
I20260709 13:10:00.421100 22749 pending_rounds.cc:107] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:10:00.421397 22749 raft_consensus.cc:2889] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:00.423148 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:00.423997 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:00.424396 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:00.425907 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.426805 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.428390 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.428846 30565 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:00.388590 30604 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:00.431473 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.433074 30458 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:00.433480 30461 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:00.433732 30622 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:10:00.440523 30622 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:00.441614 30622 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.434791 30491 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:00.446138 30491 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:00.448259 30491 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:00.435832 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.437781 30510 consensus_queue.cc:260] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.449887 30510 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:00.380736 30517 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.455024 30517 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 28 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:00.456868 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:00.457448 22902 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:10:00.472698 22898 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 22.
W20260709 13:10:00.477605 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49290.9us
I20260709 13:10:00.481035 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.483369 22750 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:00.485414 22749 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:00.487567 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.492100 30615 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:00.479655 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.502480 30314 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:00.508690 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:10:00.511336 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 253730us
I20260709 13:10:00.511837 23449 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 27. Current term is 28. Ops: []
I20260709 13:10:00.519141 23445 raft_consensus.cc:2393] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 19: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:10:00.512296 22794 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.522140 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.523098 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.523461 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.527866 22868 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.528194 22902 raft_consensus.cc:2468] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 28.
I20260709 13:10:00.537464 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.533802 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.510371 30339 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:00.528748 30604 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:00.576676 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48904.4us
I20260709 13:10:00.577939 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.540001 30305 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:00.539921 30314 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:10:00.593357 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.594169 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:00.597254 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 76364.7us
I20260709 13:10:00.599974 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.603467 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.604836 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:00.604619 30616 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.114s
I20260709 13:10:00.610998 30532 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:00.612191 30627 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:00.612469 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:00.613540 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.614593 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.615621 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.591418 30464 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:00.623978 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.625694 22828 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:10:00.626535 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.627173 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:10:00.629691 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.630482 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:00.630750 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.631546 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.631829 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.632545 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.633822 22840 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-84ea43361e534fbebad704327c9f6367 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:10:00.634310 22840 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-84ea43361e534fbebad704327c9f6367 on tablet 84ea43361e534fbebad704327c9f6367 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.635447 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:00.594656 30491 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:10:00.640014 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.649366 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:00.605630 30314 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:00.654112 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.643887 22829 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:10:00.653924 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.587147 30339 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:00.652871 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.656147 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:00.655838 23453 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:10:00.649267 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:00.670228 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.670948 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:00.672254 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.599030 30461 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 13:10:00.680281 23419 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:00.687105 22828 raft_consensus.cc:1240] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 23. Current term is 24. Ops: []
I20260709 13:10:00.678897 30628 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 13:10:00.703866 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:00.679662 30517 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:00.650359 30627 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:10:00.719411 22903 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:10:00.719856 30628 consensus_queue.cc:1243] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer c45e1805f049467697a7358fb566de2f log is divergent from this leader: its last log entry 26.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 13:10:00.705196 30511 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:10:00.701982 30626 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:10:00.721213 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.725886 22898 raft_consensus.cc:1275] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:10:00.727048 22905 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:00.728196 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:00.742776 22719 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:00.746575 23446 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:10:00.746759 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:00.747587 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.752419 22902 raft_consensus.cc:1240] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 13:10:00.754769 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
W20260709 13:10:00.755853 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.759721 22903 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:10:00.761758 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.776886 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 27 candidate_status { last_received { term: 26 index: 23 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.777880 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.781971 22749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.778595 23451 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 26.
I20260709 13:10:00.784417 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:00.785974 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:00.775688 30444 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:00.780859 30371 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:00.808598 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:00.809136 23451 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:10:00.819989 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 91863.3us
I20260709 13:10:00.816865 22822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:10:00.822952 22822 raft_consensus.cc:606] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Received request to transfer leadership to TS 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:00.825913 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.827147 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:00.827628 23447 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:10:00.829913 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.779762 30626 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.779139 30517 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.832475 30517 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:00.838958 23416 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:00.840026 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:00.841177 30371 raft_consensus.cc:2804] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:10:00.841675 30371 raft_consensus.cc:697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 28 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:00.842567 30371 consensus_queue.cc:237] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.845688 30605 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:00.852284 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.852860 23453 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:10:00.853672 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.854166 23453 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:10:00.788031 30563 consensus_queue.cc:1059] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.026s
I20260709 13:10:00.864286 30339 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:00.846923 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:00.861390 22823 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:10:00.871315 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.810699 30444 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
W20260709 13:10:00.891201 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44359.7us
I20260709 13:10:00.893885 30339 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:00.894423 22794 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:00.894440 30339 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.896716 30634 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:00.897512 30339 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:00.898162 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.899962 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:00.870958 30624 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:00.868280 30345 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:00.888008 30461 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:00.879067 30305 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:10:00.877063 30511 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:00.909688 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:00.905807 22868 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.910131 22754 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:10:00.911046 22824 raft_consensus.cc:3055] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:00.912096 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:00.879812 30455 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:00.900914 30574 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:00.921900 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:00.943216 23444 raft_consensus.cc:2468] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 15.
I20260709 13:10:00.944507 22868 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.924890 22824 raft_consensus.cc:740] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:00.909946 30622 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:10:00.891731 30314 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:10:00.954437 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:00.957445 29251 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:00.922484 30558 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:00.963219 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.964545 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:10:00.965056 30644 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:00.964579 22824 consensus_queue.cc:260] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.969906 23451 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:10:00.950937 30565 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:00.949633 30621 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:00.987488 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:00.919371 30345 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.989761 22824 raft_consensus.cc:3060] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:00.952955 30616 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:00.909488 30624 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:00.991314 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:00.995908 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.003739 30644 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:01.009213 23416 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.31.66:39003 timed out after 1.761s (SENT) [suppressed 5 similar messages]
I20260709 13:10:01.002045 30455 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:00.993829 30491 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:01.011189 30323 consensus_queue.cc:1059] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:01.011476 30491 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:01.012844 30491 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:01.013938 23448 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:01.025130 22824 raft_consensus.cc:2468] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:10:01.029670 30491 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.036545 30416 raft_consensus.cc:3055] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:10:01.036751 22901 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:10:01.037007 30416 raft_consensus.cc:740] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:01.038059 30416 consensus_queue.cc:260] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.039557 30416 raft_consensus.cc:3060] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:10:01.039631 23447 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 26.
I20260709 13:10:00.959306 30511 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.049719 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:01.053164 23419 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:00.984833 30633 consensus_queue.cc:1048] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:10:01.006364 30563 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:01.021166 30345 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:01.064459 30634 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:01.065214 23447 raft_consensus.cc:1240] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:01.083420 30511 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:01.022491 30616 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.087173 30616 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:01.089135 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:01.090914 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:01.092262 30563 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:01.095192 30563 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:01.005880 30444 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:01.102619 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:01.104676 22829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.054693 30339 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:01.054963 30605 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:01.111387 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.114562 22822 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 24.
I20260709 13:10:01.113675 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.105926 30620 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:10:01.113547 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:01.118912 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:01.119890 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:01.125841 22903 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 24: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:10:01.131433 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:01.140268 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:01.118803 30345 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.121479 30620 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:01.158362 22794 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:01.129720 30625 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:10:01.133680 30635 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:10:01.203838 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 194489us
I20260709 13:10:01.122246 30633 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 13:10:01.160789 30305 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:01.221562 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.227226 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:01.227756 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.228127 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:01.226054 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 230219us
I20260709 13:10:01.229198 22868 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:01.233655 23416 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.174191 30620 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:01.187407 30323 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:10:01.238391 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:01.211961 30496 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:01.185595 30345 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.185905 30625 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:01.248003 22904 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:10:01.219362 30634 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:01.253299 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.244207 22868 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:01.222565 30635 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:01.257705 22868 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:01.258878 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.265890 22825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18"
dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
mode: GRACEFUL
new_leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
 from {username='slave'} at 127.0.0.1:57774
I20260709 13:10:01.266003 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:01.266499 22825 raft_consensus.cc:606] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Received request to transfer leadership to TS 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:01.266877 22825 raft_consensus.cc:614] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:01.267139 30461 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:01.267827 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.268258 30617 raft_consensus.cc:3055] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:01.268625 30617 raft_consensus.cc:740] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:01.269483 30617 consensus_queue.cc:260] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:01.270728 30617 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:01.275013 30634 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:01.275499 30634 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:01.277001 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.280028 30491 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:01.283082 23450 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 25. Current term is 26. Ops: []
W20260709 13:10:01.286564 30647 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.121s	user 0.002s	sys 0.000s
I20260709 13:10:01.286716 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:01.292043 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:01.291916 30634 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.296223 30629 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:10:01.221383 30371 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:01.251549 30644 raft_consensus.cc:2804] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:01.231729 30558 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:01.231318 30618 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:01.256652 30625 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:10:01.237107 30631 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:01.308722 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:01.252323 30343 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:10:01.298620 30458 raft_consensus.cc:2804] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:01.234236 30627 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.072s
I20260709 13:10:01.263439 30635 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:10:01.331650 30651 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:10:01.332281 30626 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:01.316629 30371 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.300176 30605 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:01.360842 30605 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.303489 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:01.340091 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48910.4us
I20260709 13:10:01.342155 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:01.261476 30496 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:01.352123 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:01.323339 30624 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:01.328585 30565 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:01.302132 30323 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:10:01.362115 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58706.8us
I20260709 13:10:01.361258 30605 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:01.363131 30626 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:01.363646 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:01.337177 30644 raft_consensus.cc:697] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:10:01.369150 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 53916.6us
I20260709 13:10:01.603173 23419 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:01.507324 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:10:01.577284 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 228994us
I20260709 13:10:01.496655 30634 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:01.359938 30621 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:01.578096 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:01.346004 30314 consensus_queue.cc:1059] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.115s
I20260709 13:10:01.347544 30623 consensus_queue.cc:1059] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.135s
I20260709 13:10:01.359664 30458 raft_consensus.cc:697] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
W20260709 13:10:01.501892 30651 log.cc:927] Time spent T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:10:01.503957 30491 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:10:01.587685 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 235637us
I20260709 13:10:01.620676 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:01.609478 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:01.617528 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 275500us
I20260709 13:10:01.587817 30628 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:01.618503 30517 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:01.618964 30345 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:01.600525 30565 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:01.614950 30621 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:01.652298 30458 consensus_queue.cc:237] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.653640 30670 raft_consensus.cc:3055] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 13:10:01.640774 30662 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.267s	user 0.000s	sys 0.000s
W20260709 13:10:01.639958 30661 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:10:01.652604 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.654515 30496 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:01.631563 30658 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.637130 30510 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:01.693653 30371 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:01.689788 30658 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:01.662451 30464 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:01.697849 30464 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.668385 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:01.672529 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:01.660784 30670 raft_consensus.cc:740] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:01.674068 22823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43624 (request call id 2516) took 1072 ms. Trace:
I20260709 13:10:01.705216 30658 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:01.715204 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46884.4us
I20260709 13:10:01.715443 30616 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:10:01.715937 30616 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:01.716303 30616 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:01.709161 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:01.678699 30565 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:01.659685 30323 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.681715 30644 consensus_queue.cc:237] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.684923 30621 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:01.688305 30620 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:01.699939 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47403.4us
I20260709 13:10:01.704051 30464 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:01.718945 22823 rpcz_store.cc:276] 0709 13:10:00.601883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:00.602041 (+   158us) service_pool.cc:224] Handling call
0709 13:10:01.673886 (+1071845us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400432954. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74dff2fa 00007fcb79dc0b17 00007fcb7333b6d4 00007fcb7333b662 00007fcb7333b591 00007fcb7333b50c 00007fcb7333a7c2 00007fcb7313648c 00007fcb731358c6 00007fcb7313526f
0709 13:10:01.673985 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555904896}
I20260709 13:10:01.724753 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:01.750927 22900 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:10:01.726655 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:01.729485 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.731045 22902 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 13:10:01.731340 30670 consensus_queue.cc:260] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.731765 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:01.735461 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:01.736651 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:01.739398 22750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:01.740899 22898 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Rejecting Update request from peer c45e1805f049467697a7358fb566de2f for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:01.741931 30621 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:01.709440 30624 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:01.746877 30620 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:01.711493 30615 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:01.763859 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.764030 30615 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.764618 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.765643 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.766064 30615 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:01.755411 22899 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:10:01.753875 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.767750 22831 raft_consensus.cc:2410] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f for term 26 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 13:10:01.761938 30416 consensus_meta.cc:287] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Time spent flushing consensus metadata: real 0.722s	user 0.013s	sys 0.000s
I20260709 13:10:01.749419 30532 raft_consensus.cc:3055] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:01.770646 30464 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:10:01.771116 30666 log.cc:927] Time spent T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.184s	user 0.003s	sys 0.000s
I20260709 13:10:01.771323 30670 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:10:01.764576 30621 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.770855 30532 raft_consensus.cc:740] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:01.775980 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:01.775719 30532 consensus_queue.cc:260] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.776973 30621 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:01.777019 30532 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:01.762086 30503 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:10:01.769112 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:01.760545 30667 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:01.786697 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:01.789991 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:01.790555 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:01.771765 30464 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:01.697227 30605 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.793718 30511 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 20.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.007s
I20260709 13:10:01.801388 30439 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:01.791863 30624 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:01.808526 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:01.809476 22827 raft_consensus.cc:1240] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 LEADER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 25. Current term is 26. Ops: []
I20260709 13:10:01.798915 30616 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:01.791675 30343 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.018s
I20260709 13:10:01.810583 22750 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:10:01.812624 30616 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:01.805545 30511 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:01.813470 30511 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:01.814309 30511 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.794659 30376 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:10:01.820189 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:01.822229 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.822556 23451 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:10:01.823473 22903 raft_consensus.cc:2393] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 24: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
W20260709 13:10:01.824177 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.824697 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:01.826061 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:10:01.849421 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:01.853519 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.854856 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.857623 30517 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:01.858532 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:10:01.791855 30635 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.880012 23416 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.882838 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:01.893347 22830 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:10:01.849602 30624 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.852733 30439 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:01.853503 30581 raft_consensus.cc:2749] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:01.903815 22831 raft_consensus.cc:1240] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 23. Current term is 24. Ops: []
I20260709 13:10:01.912179 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:01.914844 23419 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:01.924443 30635 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:01.925571 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:01.930809 30464 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:01.931439 30439 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:01.932344 30439 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:01.935032 30657 log.cc:927] Time spent T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.625s	user 0.021s	sys 0.029s
I20260709 13:10:01.849603 30625 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:01.936360 23452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38090 (request call id 2063) took 1209 ms. Trace:
W20260709 13:10:01.936604 30682 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:10:01.898312 30629 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:01.936728 23452 rpcz_store.cc:276] 0709 13:10:00.726939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:00.805624 (+ 78685us) service_pool.cc:224] Handling call
0709 13:10:00.861963 (+ 56339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:00.876443 (+ 14480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:00.876475 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:00.904584 (+ 28109us) log.cc:844] Serialized 26 byte log entry
0709 13:10:01.131309 (+226725us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:10:01.131348 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:01.131404 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:01.934436 (+803032us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb7499686e 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221 00007fcb7499f1d9 00007fcb7499f169
0709 13:10:01.935952 (+  1516us) raft_consensus.cc:1685] finished
0709 13:10:01.935970 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:01.936183 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7734,"raft.queue_time_us":7875,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":7509,"spinlock_wait_cycles":28635904,"thread_start_us":220291,"threads_started":1,"wal-append.queue_time_us":311498}
I20260709 13:10:01.938820 30605 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:01.942165 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:01.949628 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.951507 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.951825 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:01.950906 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:01.937065 30625 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 27 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:01.953349 30667 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.992646 22747 raft_consensus.cc:1240] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:01.948371 30558 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:01.995643 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.997525 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:01.910035 30624 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.012240 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:02.023988 30511 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:02.060204 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:02.070626 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:01.998488 30667 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.069641 30371 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:10:02.094241 30669 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:10:02.098153 30680 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:10:02.099216 22871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.099808 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:10:02.100508 30621 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:10:02.100880 30621 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.101212 30621 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:02.112123 30621 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:02.112929 30667 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:02.112124 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:02.115972 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:02.117661 23453 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:10:02.122984 23444 raft_consensus.cc:1240] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:02.119450 22750 raft_consensus.cc:1275] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 13:10:02.123879 30455 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:10:02.126875 30510 consensus_queue.cc:1048] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:10:02.127457 30510 consensus_queue.cc:1243] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [LEADER]: Peer 6a5191aaff76469bbf6c4735dc9e6509 log is divergent from this leader: its last log entry 27.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 13:10:02.129897 30323 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:10:02.131567 22750 pending_rounds.cc:85] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Aborting all ops after (but not including) 19
I20260709 13:10:02.131839 22750 pending_rounds.cc:107] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Aborting uncommitted NO_OP operation due to leader change: 27.20
I20260709 13:10:02.132120 22750 raft_consensus.cc:2889] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:02.144716 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:02.146004 22753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:10:02.146566 22753 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:02.146054 22901 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:10:02.153133 30624 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:02.158728 30621 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:02.165098 22905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:02.165854 22905 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:10:02.172873 22904 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 20. (term mismatch)
I20260709 13:10:02.173437 22904 pending_rounds.cc:85] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: Aborting all ops after (but not including) 19
I20260709 13:10:02.173800 22904 pending_rounds.cc:107] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: Aborting uncommitted NO_OP operation due to leader change: 25.20
I20260709 13:10:02.174072 22904 raft_consensus.cc:2889] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:02.162514 22753 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 25.
I20260709 13:10:02.163817 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:02.181947 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:02.183048 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:02.181025 22871 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
W20260709 13:10:02.163757 30394 log.cc:927] Time spent T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.056s	user 0.002s	sys 0.001s
I20260709 13:10:02.193876 22830 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:02.205289 30581 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:10:02.205791 30581 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.205502 30323 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:10:02.206176 30581 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:02.210520 30630 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:10:02.211107 30458 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:10:02.211745 30458 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:02.215451 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:02.216042 22830 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:10:02.221388 30624 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:02.221797 30624 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.222143 30624 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:02.212720 30458 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:02.289228 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:10:02.290449 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
W20260709 13:10:02.292292 22720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.275492 30599 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:02.302860 22797 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:02.309010 30503 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:10:02.364130 30630 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:02.368117 30630 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:02.363468 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:10:02.369513 30683 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.063s	user 0.002s	sys 0.000s
I20260709 13:10:02.383836 22797 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:02.391794 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:02.400758 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:02.386451 30304 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:02.386600 30611 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:10:02.433573 22830 raft_consensus.cc:1240] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:02.323992 30599 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:02.452755 30669 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:02.494553 22755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38488 (request call id 3502) took 1196 ms. Trace:
I20260709 13:10:02.495122 22755 rpcz_store.cc:276] 0709 13:10:01.297980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.298672 (+   692us) service_pool.cc:224] Handling call
0709 13:10:01.782518 (+483846us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5800003398. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71cd6b71 00007fcb71cd6a22 00007fcb7abefc09 00007fcb79d49ee0 00007fcb79e130ab 00007fcb79e13131 00007fcb79dd4fc0 00007fcb79dbf929 00007fcb79dbf3d2 00007fcb73336bd4
0709 13:10:01.782676 (+   158us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:01.792735 (+ 10059us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:01.792763 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:01.800066 (+  7303us) log.cc:844] Serialized 26 byte log entry
0709 13:10:01.992071 (+192005us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:01.992125 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:01.992216 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:02.494236 (+502020us) raft_consensus.cc:1685] finished
0709 13:10:02.494263 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:02.494484 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5361,"raft.queue_time_us":6830,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":5086,"spinlock_wait_cycles":3553152,"thread_start_us":191623,"threads_started":1,"wal-append.queue_time_us":475184}
I20260709 13:10:02.514390 30599 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:02.515686 22825 consensus_meta.cc:287] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.000s
I20260709 13:10:02.516171 22825 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 24.
W20260709 13:10:02.523159 22898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.026s	sys 0.042s
W20260709 13:10:02.523463 22898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.026s	sys 0.042s
W20260709 13:10:02.532563 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 216916us
I20260709 13:10:02.535529 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:02.539139 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:02.540814 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:02.539341 30670 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:02.547657 30670 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:02.554705 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:02.521632 30669 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.544265 30510 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.555747 30670 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:02.557346 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 292302us
I20260709 13:10:02.585011 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:02.603492 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:02.622061 22755 raft_consensus.cc:3060] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:02.622371 22825 raft_consensus.cc:1240] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 27 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 26. Current term is 27. Ops: []
W20260709 13:10:02.582083 30439 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.012s	sys 0.023s
I20260709 13:10:02.569351 30510 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.570290 30644 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.033s
W20260709 13:10:02.610918 30692 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:10:02.604209 30699 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:02.632752 30439 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.012s	sys 0.028s
W20260709 13:10:02.644274 30695 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:10:02.644979 30581 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.653918 30669 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:02.609984 30624 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:02.658368 30694 log.cc:927] Time spent T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:10:02.663781 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:02.662065 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:10:02.676460 30689 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:10:02.676795 30624 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:02.677848 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:02.678545 23449 raft_consensus.cc:2368] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 22. Current term is 23.
I20260709 13:10:02.683444 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:02.674758 30704 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:02.684423 30510 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:02.684964 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:02.687126 22903 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 22: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:10:02.693025 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:02.695242 22755 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:10:02.742044 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:02.691670 30444 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:02.745995 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 166757us
I20260709 13:10:02.749221 23453 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:10:02.749747 23449 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:10:02.750005 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:02.756716 22823 raft_consensus.cc:1275] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:10:02.684439 30704 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.746672 30629 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:02.740835 30644 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:10:02.777842 22794 leader_election.cc:400] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 22. Current term is 23.
I20260709 13:10:02.782372 22794 leader_election.cc:403] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:02.788203 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:02.789451 22823 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:02.806109 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:02.806876 23449 raft_consensus.cc:1275] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:10:02.809798 30707 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
W20260709 13:10:02.810699 30703 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:10:02.791077 30323 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 23
W20260709 13:10:02.801810 30700 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:10:02.817579 30464 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:02.823215 30704 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:02.817028 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:02.784689 30630 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.799930 30444 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:10:02.842980 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:02.843505 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:02.843755 22825 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:02.847532 23448 raft_consensus.cc:1240] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 24. Current term is 25. Ops: []
I20260709 13:10:02.813637 30698 consensus_queue.cc:1059] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:10:02.825276 30702 consensus_queue.cc:1048] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.025s
W20260709 13:10:02.850627 30701 log.cc:927] Time spent T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.188s	user 0.002s	sys 0.008s
I20260709 13:10:02.851876 22898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43996 (request call id 2383) took 1034 ms. Trace:
I20260709 13:10:02.835682 30697 raft_consensus.cc:2764] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader election decision vote started in defunct term 24: won
I20260709 13:10:02.855268 23453 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 22: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:10:02.852247 22898 rpcz_store.cc:276] 0709 13:10:01.817901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.818041 (+   140us) service_pool.cc:224] Handling call
0709 13:10:01.824612 (+  6571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:01.824907 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:01.824933 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:01.825345 (+   412us) log.cc:844] Serialized 26 byte log entry
0709 13:10:02.523786 (+698441us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:02.523826 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.523885 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:02.850365 (+326480us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b240023db78. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71b949c4 00007fcb71b92acb 00007fcb71d4ddc9 00007fcb71d4da55 00007fcb71d4d09d 00007fcb7497f10e 00007fcb7499688c 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221
0709 13:10:02.851622 (+  1257us) raft_consensus.cc:1685] finished
0709 13:10:02.851640 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:02.851825 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":593,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2230,"spinlock_wait_cycles":8139520,"thread_start_us":621496,"threads_started":1,"wal-append.queue_time_us":783231}
I20260709 13:10:02.864566 30628 raft_consensus.cc:3055] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:10:02.864987 30628 raft_consensus.cc:740] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:02.865849 30628 consensus_queue.cc:260] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.866997 30628 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:02.873638 30676 consensus_queue.cc:1059] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:10:02.875017 30676 consensus_queue.cc:1059] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:10:02.876003 30676 raft_consensus.cc:3055] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:02.876327 30676 raft_consensus.cc:740] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:02.818347 30707 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:10:02.877507 30416 raft_consensus.cc:3055] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:10:02.877938 30416 raft_consensus.cc:740] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:02.878674 30416 consensus_queue.cc:260] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.875079 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:02.856062 22825 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 25.
I20260709 13:10:02.909621 30416 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:10:02.922449 30532 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:02.934252 30676 consensus_queue.cc:260] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:02.984282 30676 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:10:03.019011 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 92789.3us
I20260709 13:10:03.021814 23451 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 23 LEARNER]: Advancing to term 25
I20260709 13:10:03.036638 30304 raft_consensus.cc:993] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: : Instructing follower 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:10:03.037220 30304 raft_consensus.cc:1081] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Signalling peer 6a5191aaff76469bbf6c4735dc9e6509 to start an election
I20260709 13:10:03.045413 30627 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.045992 30627 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.048380 30627 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:03.049336 23451 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 25 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:10:03.082535 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:03.083376 23451 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
W20260709 13:10:03.094506 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 43517.5us
I20260709 13:10:03.027192 30628 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.095563 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:10:02.989187 30608 consensus_queue.cc:1175] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c45e1805f049467697a7358fb566de2f" responder_term: 19 status { last_received { term: 19 index: 16 } last_committed_idx: 16 last_received_current_leader { term: 19 index: 16 } }
W20260709 13:10:03.163784 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 58974.3us
I20260709 13:10:03.104205 30376 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:03.197292 30376 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.094857 30455 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.093s
I20260709 13:10:03.208258 30343 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:03.209887 30716 log.cc:927] Time spent T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
W20260709 13:10:03.211467 22722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:03.216308 30714 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:10:03.208796 30343 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.103482 30628 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.207070 30376 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:03.257838 30323 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) with higher term. Message: Invalid argument: T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 22. Current term is 23.
W20260709 13:10:03.259620 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 327215us
I20260709 13:10:03.275449 30343 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:03.280339 23453 raft_consensus.cc:1240] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Rejecting Update request from peer 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:10:03.272396 30627 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.282022 30627 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.284119 30627 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:03.293721 30604 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:03.294435 30604 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.300762 22828 raft_consensus.cc:2468] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:10:03.302449 30604 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:03.316646 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 201241us
I20260709 13:10:03.319995 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:03.325078 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:03.325745 23453 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:10:03.340111 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:03.340974 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a957de2a744260812f8da6f1bd9b87" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:03.343757 22903 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:10:03.346784 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:03.354486 30717 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:03.354928 30717 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:03.355324 30717 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:03.358256 30697 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:03.358773 30604 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:03.359324 30604 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.362049 30604 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:03.363655 22756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:03.364241 22756 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:10:03.371547 30717 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.358806 30697 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:10:03.343391 30702 consensus_queue.cc:1059] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:10:03.385944 30376 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.387053 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:03.388257 30376 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:03.388710 30732 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:03.374284 30717 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:03.381705 30724 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:10:03.389415 30732 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.402645 30697 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:03.351176 30670 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:10:03.407567 30689 log.cc:927] Time spent T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.731s	user 0.022s	sys 0.028s
I20260709 13:10:03.411430 30732 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:03.412693 30630 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:03.397820 30376 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:03.421342 30376 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.422911 30628 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:03.419621 30670 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:03.424865 30581 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:03.425608 30670 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:03.428524 22896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1902) took 1610 ms. Trace:
I20260709 13:10:03.428905 22896 rpcz_store.cc:276] 0709 13:10:01.818339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.818482 (+   143us) service_pool.cc:224] Handling call
0709 13:10:01.818862 (+   380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:01.821228 (+  2366us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:10:01.828709 (+  7481us) log.cc:844] Serialized 14 byte log entry
0709 13:10:02.288567 (+459858us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:02.300287 (+ 11720us) log.cc:844] Serialized 52 byte log entry
0709 13:10:02.300523 (+   236us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:10:02.300595 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.300705 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:03.407327 (+1106622us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b240023db78. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71b949c4 00007fcb71b92acb 00007fcb71d4ddc9 00007fcb71d4da55 00007fcb71d4d09d 00007fcb7497f10e 00007fcb7499688c 00007fcb74995bb6 00007fcb749957d8 00007fcb7499f221
0709 13:10:03.428204 (+ 20877us) raft_consensus.cc:1685] finished
0709 13:10:03.428302 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:03.428448 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":133436,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":2401,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":3406,"spinlock_wait_cycles":24318208,"thread_start_us":387469,"threads_started":2,"wal-append.queue_time_us":604008}
I20260709 13:10:03.430919 30622 raft_consensus.cc:3055] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:03.431468 30622 raft_consensus.cc:740] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:03.419858 22794 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:03.449460 30627 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:03.432510 30622 consensus_queue.cc:260] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.460825 30622 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:03.461723 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:03.470089 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:03.470744 22900 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 21: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:10:03.472689 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:03.478068 22900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:03.478617 22900 raft_consensus.cc:3060] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:03.480473 23416 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:03.482995 30517 raft_consensus.cc:2804] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:10:03.483441 30517 raft_consensus.cc:697] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 25 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:03.484299 30376 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:03.484213 30517 consensus_queue.cc:237] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.485656 23416 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:03.487530 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:03.491724 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:03.492298 22747 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:10:03.496677 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:03.497603 22900 raft_consensus.cc:2468] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 25.
I20260709 13:10:03.499650 23445 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:10:03.494760 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:03.501274 22747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d"
dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
 from {username='slave'} at 127.0.0.1:38488
W20260709 13:10:03.501611 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 72517.8us
I20260709 13:10:03.501829 22747 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.502266 22747 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:03.504104 23449 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:10:03.503353 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:03.505501 23444 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:03.511080 22755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:03.513005 22754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:03.514204 22752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:10:03.517751 22751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:03.521745 30669 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.521999 22747 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:03.522321 30669 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:03.524077 22747 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:03.524164 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:10:03.524454 30669 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:03.527109 30323 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.527585 30323 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.502519 30510 raft_consensus.cc:2804] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:03.538295 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:03.539048 22897 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:03.553709 22747 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:10:03.558866 23444 raft_consensus.cc:2468] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:10:03.556623 22794 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:10:03.594260 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 44934.3us
W20260709 13:10:03.610387 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 44033.3us
I20260709 13:10:03.553422 30726 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:03.545157 30510 raft_consensus.cc:697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:03.618925 22794 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:03.624572 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:03.594002 30611 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:10:03.652056 30611 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:03.652366 30625 raft_consensus.cc:2804] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:03.652915 30625 raft_consensus.cc:697] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:03.652977 30611 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.628758 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:03.655167 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:03.632579 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:10:03.656374 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 110208us
I20260709 13:10:03.632862 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:03.566936 30323 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:03.556370 30304 consensus_queue.cc:1059] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:10:03.616252 30732 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.660343 30732 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:03.541883 30699 consensus_queue.cc:1048] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 13:10:03.661262 23451 raft_consensus.cc:2393] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 23: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:10:03.618383 30726 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:03.649492 21629 tablet_server.cc:179] TabletServer@127.21.31.65:0 shutting down...
I20260709 13:10:03.650811 30510 consensus_queue.cc:237] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.667128 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:03.667707 22901 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:10:03.669545 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:03.618149 22747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509" is_pre_election: true
I20260709 13:10:03.668751 23445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:03.675089 23445 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:10:03.685379 22897 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:10:03.694417 22748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6a5191aaff76469bbf6c4735dc9e6509"
I20260709 13:10:03.695143 22748 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:03.662858 30726 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:03.719239 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49761.5us
W20260709 13:10:03.721597 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 57478.1us
I20260709 13:10:03.673571 30376 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:03.726356 22797 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:10:03.734447 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 58632.9us
I20260709 13:10:03.737912 30304 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:03.743096 30625 consensus_queue.cc:237] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.745035 22748 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:10:03.746047 23419 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:03.746763 30630 raft_consensus.cc:2804] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:03.747274 30630 raft_consensus.cc:697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:03.748126 30630 consensus_queue.cc:237] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.680956 30699 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 13:10:03.728288 30394 tablet_bootstrap.cc:492] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:03.759507 30731 raft_consensus.cc:1064] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
W20260709 13:10:03.759912 30731 raft_consensus.cc:1068] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Unable to promote non-voter c45e1805f049467697a7358fb566de2f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:10:03.760157 30394 tablet_bootstrap.cc:492] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:10:03.760988 30394 ts_tablet_manager.cc:1403] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 3.393s	user 0.086s	sys 0.037s
W20260709 13:10:03.761688 22794 consensus_peers.cc:597] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:03.762183 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 48863.1us
I20260709 13:10:03.723614 30343 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:10:03.680974 30679 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:10:03.778383 22827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:03.784260 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:03.791950 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:03.780102 30621 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:03.810103 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:03.810691 22822 raft_consensus.cc:3055] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:03.810958 22822 raft_consensus.cc:740] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:03.811700 22822 consensus_queue.cc:260] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:03.812773 22822 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:03.831840 30644 raft_consensus.cc:493] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.832587 30644 raft_consensus.cc:515] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:03.833752 30743 log.cc:927] Time spent T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:10:03.836807 30644 leader_election.cc:290] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:03.802776 30669 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:03.842756 30669 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.843113 30669 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:03.843814 30728 raft_consensus.cc:3055] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:10:03.844012 22653 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33010 (request call id 228) took 31910 ms (client timeout 15000 ms). Trace:
I20260709 13:10:03.844285 30728 raft_consensus.cc:740] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:03.845567 30728 consensus_queue.cc:260] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:03.854333 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 104372us
I20260709 13:10:03.855324 22719 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509; no voters: 
I20260709 13:10:03.873260 22822 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
W20260709 13:10:03.847196 22653 rpcz_store.cc:269] 0709 13:09:31.933153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.937501 (+  4348us) service_pool.cc:224] Handling call
0709 13:09:32.285611 (+348110us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b54000ab4d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d65090 00007fcb74e03f3a 00007fcb7abba1cc 00007fcb7acb837e 00007fcb791792a4 00007fcb79179232 00007fcb79179161 00007fcb791790dc 00007fcb79178392 00007fcb7313648c 00007fcb731358c6
0709 13:10:03.843488 (+31557877us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:01.164112 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:01.164624 (+   512us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:01.164670 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:10:01.164701 (+    31us) tablet.cc:662] Decoding operations
0709 13:10:01.460282 (+295581us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:01.495159 (+ 34877us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:01.499245 (+  4086us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:10:01.653764 (+154519us) tablet.cc:701] Row locks acquired
0709 13:10:01.653802 (+    38us) write_op.cc:260] PREPARE: finished
0709 13:10:01.653977 (+   175us) write_op.cc:270] Start()
0709 13:10:01.654116 (+   139us) write_op.cc:276] Timestamp: P: 1783602601653916 usec, L: 0
0709 13:10:01.654150 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:10:01.657631 (+  3481us) log.cc:844] Serialized 3574 byte log entry
0709 13:10:02.384594 (+726963us) op_driver.cc:464] REPLICATION: finished
0709 13:10:02.785514 (+400920us) write_op.cc:301] APPLY: starting
0709 13:10:02.789072 (+  3558us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:02.869200 (+ 80128us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:02.869549 (+   349us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:03.285954 (+416405us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:03.286254 (+   300us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 13:10:03.286317 (+    63us) write_op.cc:312] APPLY: finished
0709 13:10:03.286672 (+   355us) log.cc:844] Serialized 89 byte log entry
0709 13:10:03.382791 (+ 96119us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:03.399063 (+ 16272us) write_op.cc:454] Released schema lock
0709 13:10:03.408615 (+  9552us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:03.409101 (+   486us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:03.409134 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":198,"mutex_wait_us":640014,"reported_tablets":42,"spinlock_wait_cycles":45730688,"child_traces":[["op",{"apply.queue_time_us":362102,"apply.run_cpu_time_us":26031,"apply.run_wall_time_us":641172,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41747,"num_ops":9,"prepare.queue_time_us":410130,"prepare.run_cpu_time_us":34056,"prepare.run_wall_time_us":607373,"raft.queue_time_us":408657,"raft.run_cpu_time_us":30374,"raft.run_wall_time_us":363710,"replication_time_us":664780,"thread_start_us":856370,"threads_started":5,"wal-append.queue_time_us":309565,"wal-append.run_cpu_time_us":38807,"wal-append.run_wall_time_us":1320060}]]}
I20260709 13:10:03.877610 30728 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:03.879526 30737 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:03.847143 30621 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:03.936214 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 188063us
I20260709 13:10:03.877985 30510 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:03.911252 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:03.913793 22831 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:03.914366 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:03.917840 30394 raft_consensus.cc:359] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:03.905231 30699 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.878006 30704 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:03.941268 22824 raft_consensus.cc:2393] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 23: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:10:03.943537 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:03.913012 30726 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.960143 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:03.965943 22652 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 21 to 25, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 25 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.967869 22652 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 15 to 19, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 19 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:03.969902 30742 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:03.970794 30621 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:03.940722 30394 raft_consensus.cc:740] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
I20260709 13:10:03.974526 22652 catalog_manager.cc:5697] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 10 to 11, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 11 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:03.979710 22831 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:10:03.989868 23449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18051671d25f4aa7b30d4a299909f91a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:03.942553 30737 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:03.993095 22825 raft_consensus.cc:2468] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:10:03.978878 22652 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:03.955897 30623 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:03.997033 22652 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 14 to 21. New cstate: current_term: 21 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:03.941344 30510 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.010108 22652 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 12, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:10:03.944626 30699 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:03.945032 30704 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:04.041343 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.045008 22871 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.041105 22652 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 21 to 23, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 23 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.046962 22652 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 19 to 20, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 20 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.048821 22652 catalog_manager.cc:5697] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 9 to 16, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 16 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:10:04.062304 22652 catalog_manager.cc:5697] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
W20260709 13:10:04.072817 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.073582 23453 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: [18.15-19.16]
W20260709 13:10:04.084401 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.081634 22652 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 15, term changed from 23 to 24, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to <none>, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 24 committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:10:04.089910 22652 catalog_manager.cc:5697] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 18 to 21, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 21 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.091845 22652 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 18 to 21, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 21 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.093272 22652 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 21 to 22, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 22 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.105851 22652 catalog_manager.cc:5697] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 25 to 29, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 29 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.107281 22652 catalog_manager.cc:5697] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 16 to 18, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 18 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:04.040067 30394 consensus_queue.cc:260] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:04.113226 30669 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.129914 30699 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.130733 30669 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Requested vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.111881 22652 catalog_manager.cc:5697] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 14 to 17, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
W20260709 13:10:04.132551 30750 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
W20260709 13:10:04.141281 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 241298us
I20260709 13:10:04.101770 30737 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:04.148706 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 216545us
W20260709 13:10:04.147447 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 187356us
I20260709 13:10:04.137765 22652 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 12 to 15, term changed from 18 to 23, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.150405 22823 raft_consensus.cc:1240] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Rejecting Update request from peer 6a5191aaff76469bbf6c4735dc9e6509 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:04.151124 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:04.160825 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:10:04.161222 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.161397 23448 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:10:04.161633 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 76610.4us
I20260709 13:10:04.158790 22652 catalog_manager.cc:5697] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:04.162530 23447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:04.163643 22652 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 24 to 26, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 26 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:10:04.162220 22871 leader_election.cc:336] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:04.170543 22898 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 13:10:04.172152 30746 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:10:04.165978 22652 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:04.173668 30517 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:04.174248 30517 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.174024 22652 catalog_manager.cc:5697] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 19 to 22, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 22 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.175704 22652 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.178220 22652 catalog_manager.cc:5697] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 16 to 20, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 20 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:04.181681 30752 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:10:04.182082 22652 catalog_manager.cc:5697] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 20 to 25. New cstate: current_term: 25 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:04.185072 22652 catalog_manager.cc:5697] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.186713 22652 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 18 to 21, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 21 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:10:04.189611 22652 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 17 to 21, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 21 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:10:04.190841 22652 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 25 to 26, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 26 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.192075 22652 catalog_manager.cc:5697] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 22 to 25, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 25 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:04.197669 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:04.198247 22901 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:04.197815 22652 catalog_manager.cc:5697] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.151767 30726 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:04.204290 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 48148.7us
I20260709 13:10:04.202741 30704 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:04.215181 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:04.217025 22868 leader_election.cc:304] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:04.205771 30511 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:10:04.214036 30608 consensus_queue.cc:1059] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:10:04.218935 30731 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:04.219739 30731 raft_consensus.cc:3060] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:04.225502 30702 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:10:04.212092 30717 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.242489 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:04.244024 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:04.242133 30623 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.250692 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:04.251242 23446 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:10:04.254889 30723 raft_consensus.cc:3055] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:04.255286 30723 raft_consensus.cc:740] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:04.256115 30723 consensus_queue.cc:260] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:04.264923 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.257427 30723 raft_consensus.cc:3060] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:04.276060 30517 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:04.282557 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:04.283463 30702 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:04.283849 30702 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:04.284175 30702 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:04.282558 23448 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:10:04.293128 30644 raft_consensus.cc:1064] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
W20260709 13:10:04.293560 30644 raft_consensus.cc:1068] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Unable to promote non-voter c45e1805f049467697a7358fb566de2f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:10:04.293841 30394 ts_tablet_manager.cc:1434] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.532s	user 0.008s	sys 0.001s
W20260709 13:10:04.294174 30394 ts_tablet_manager.cc:1467] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Tablet startup took 532ms
W20260709 13:10:04.294486 30394 ts_tablet_manager.cc:1469] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Trace:
0709 13:09:54.883641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:54.885501 (+  1860us) service_pool.cc:224] Handling call
0709 13:09:54.948585 (+ 63084us) ts_tablet_manager.cc:934] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
0709 13:09:57.199190 (+2250605us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:57.864561 (+665371us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:57.864676 (+   115us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:58.003743 (+139067us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:58.003996 (+   253us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:58.004335 (+   339us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:58.004391 (+    56us) inbound_call.cc:177] Queueing success response
0709 13:09:58.667893 (+663502us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b4c002fdd80. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb730c7a20 00007fcb730d8a29 00007fcb730d913f 00007fcb7309d2ab 00007fcb730caf3c 00007fcb730cb461 00007fcb730cb768 00007fcb78e5166b 00007fcb79d8e15b 00007fcb79d8df98 00007fcb79d7f322
0709 13:10:00.367531 (+1699638us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:10:03.761383 (+3393852us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:10:03.846111 (+ 84728us) tablet_replica.cc:261] Starting instrumentation
0709 13:10:03.846691 (+   580us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":60164,"mutex_wait_us":817,"raft.queue_time_us":20233,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":2653,"spinlock_wait_cycles":9608576,"tablet-copy.queue_time_us":61929,"tablet-download-pool.queue_time_us":182681,"tablet-download-pool.run_cpu_time_us":4239,"tablet-download-pool.run_wall_time_us":1029418,"thread_start_us":156683,"threads_started":2}
I20260709 13:10:04.295382 30731 raft_consensus.cc:2749] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:04.267575 22797 leader_election.cc:336] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:04.300575 30510 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:04.244978 30717 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:04.306378 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.299880 22797 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:04.307545 30717 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.309630 30376 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:04.310783 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 44760.4us
I20260709 13:10:04.313778 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:04.320901 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 71659.5us
W20260709 13:10:04.311441 22797 consensus_peers.cc:597] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.323550 22901 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
W20260709 13:10:04.323194 22797 consensus_peers.cc:597] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.330644 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:04.330839 22797 consensus_peers.cc:597] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.247637 30623 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.365293 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:04.366408 22823 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:10:04.366876 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:04.367427 22828 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 27: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:10:04.369427 22823 raft_consensus.cc:3060] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:10:04.333284 22797 consensus_peers.cc:597] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.374205 22828 raft_consensus.cc:1275] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 13:10:04.372344 22797 consensus_peers.cc:597] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.384127 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:04.364837 30726 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.371068 30669 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.381583 22840 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-4e4b3212e80646379ccf77b5952a4363 received from {username='slave'} at 127.0.0.1:47110
I20260709 13:10:04.399886 22794 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:04.416638 22899 raft_consensus.cc:1275] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:10:04.395509 30621 raft_consensus.cc:2749] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:04.412055 22840 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-4e4b3212e80646379ccf77b5952a4363 on tablet 4e4b3212e80646379ccf77b5952a4363 with peer c45e1805f049467697a7358fb566de2f
I20260709 13:10:04.440042 22823 raft_consensus.cc:1275] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:10:04.417126 30732 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:04.397722 30726 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.402393 30669 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.462819 22825 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 24: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:10:04.467330 30625 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.467857 30726 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.467902 30625 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:10:04.468694 22721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.469846 30713 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:04.470398 30625 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:04.479311 30623 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.486958 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:04.487123 23448 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 13:10:04.470377 30713 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:04.495498 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:04.493821 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:04.499591 23448 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:10:04.459038 30732 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:04.495640 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.514048 23448 raft_consensus.cc:1275] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:10:04.512976 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:04.498751 30623 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:04.520617 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.527058 23446 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 19.
I20260709 13:10:04.501282 30764 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.477188 30702 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.493991 30510 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:10:04.516113 30616 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:04.499713 30725 consensus_queue.cc:1048] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:10:04.523934 30503 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:10:04.502046 30668 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 13:10:04.515250 30343 consensus_queue.cc:1048] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:10:04.542125 22898 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: []
W20260709 13:10:04.555932 22797 consensus_peers.cc:597] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.556495 22797 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.558300 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:04.558902 22825 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 20.
I20260709 13:10:04.529633 30620 consensus_queue.cc:1048] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:10:04.537844 30503 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:04.556973 22797 leader_election.cc:336] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:04.564162 22797 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:04.569679 30702 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.534066 30764 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.572098 30764 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.530797 30623 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:04.575484 30756 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:10:04.562228 30713 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.541774 30630 consensus_queue.cc:1048] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 13:10:04.566454 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:10:04.574506 22797 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.588603 22797 leader_election.cc:336] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:04.589037 22797 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:04.564693 30503 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:10:04.592295 22797 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.539180 30616 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:04.597647 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:04.604535 23446 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 23.
I20260709 13:10:04.594058 30764 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:04.595214 30756 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:04.618820 30737 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:04.622335 30764 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:04.644050 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:04.648097 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:04.639035 30620 raft_consensus.cc:493] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.643853 30764 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:10:04.656162 22797 consensus_peers.cc:597] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.645160 30737 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.666509 30616 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:04.667755 30623 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:04.668007 30732 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:04.679171 30503 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.694105 30620 raft_consensus.cc:515] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.707221 22826 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 25: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:10:04.708045 30620 leader_election.cc:290] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.696632 30669 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.703269 30737 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.705047 23453 raft_consensus.cc:1217] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 LEARNER]: Deduplicated request from leader. Original: 19.11->[19.12-19.12]   Dedup: 19.12->[]
I20260709 13:10:04.716707 30503 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:04.718497 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.721010 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 193287us
I20260709 13:10:04.722730 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:04.723420 22824 raft_consensus.cc:2393] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 20: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
W20260709 13:10:04.722168 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.727847 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:10:04.751333 22797 leader_election.cc:336] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:04.751909 22797 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:10:04.732681 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.755479 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:04.756160 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.761788 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:04.763579 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:04.764552 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11e8d51b5b18431cb7d6df2884a4e879" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:04.753155 22797 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.762403 23446 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 26.
W20260709 13:10:04.768347 23419 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:04.768930 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:04.770623 23453 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:10:04.768975 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:04.773289 22797 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:10:04.779388 22797 leader_election.cc:336] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:04.767231 30669 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:04.779878 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:04.769487 30764 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.785135 30618 consensus_queue.cc:1059] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.003s
I20260709 13:10:04.788271 30764 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.776677 30343 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:10:04.800506 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 178897us
W20260709 13:10:04.812433 22797 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.815171 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:04.816155 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:04.824231 23419 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:04.824694 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:04.828022 22794 leader_election.cc:304] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:04.830340 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:04.832123 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:04.833163 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:04.833698 22830 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:04.834048 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:04.834401 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:04.836019 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:10:04.838263 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 204025us
I20260709 13:10:04.817534 30305 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:04.823814 30623 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:10:04.820595 30770 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.854058 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
W20260709 13:10:04.854259 22638 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 135e2f7520a34b53aaf4ddb360a8a029 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica c45e1805f049467697a7358fb566de2f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:04.858839 22827 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 13:10:04.860652 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.860759 23444 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:10:04.861130 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
W20260709 13:10:04.894197 22638 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 80718.1us
I20260709 13:10:04.894919 30711 raft_consensus.cc:1064] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
W20260709 13:10:04.895349 30711 raft_consensus.cc:1068] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: Unable to promote non-voter c45e1805f049467697a7358fb566de2f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:10:04.897329 30764 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:04.897754 30742 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:04.897881 30764 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.898283 30742 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.900370 30764 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.902148 30764 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:04.902598 30764 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.908706 30764 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.909785 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:04.911257 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:04.919334 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:04.919905 22797 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:10:04.920990 22797 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:04.916046 22868 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:04.922910 30773 raft_consensus.cc:2749] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:10:04.923516 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:04.856719 30623 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:04.942883 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 81812.8us
I20260709 13:10:04.951236 30779 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:04.951742 30779 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:04.953701 30779 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.957096 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:04.957813 23451 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 25: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:10:04.940979 30623 raft_consensus.cc:3060] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:04.936844 23444 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:10:04.900386 30742 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:04.990954 22830 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 21.
I20260709 13:10:04.998374 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:04.998983 22824 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 25.
W20260709 13:10:05.011399 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48010.2us
I20260709 13:10:04.956538 30726 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 13:10:05.024717 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:04.865001 30770 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.026232 30726 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.042325 30778 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:05.052597 30778 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:05.034071 30737 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:05.042647 30723 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:05.055572 30726 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:10:05.064805 30616 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:05.072628 30698 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:05.069708 30742 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:05.072625 30771 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:10:05.116431 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 249502us
I20260709 13:10:05.131081 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:05.086701 30679 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:05.088702 30764 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:05.073660 30737 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.078018 30723 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.163884 30764 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.164703 30723 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.123898 30758 raft_consensus.cc:3055] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:05.144323 23453 raft_consensus.cc:2468] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 25.
I20260709 13:10:05.168035 22868 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.119118 30676 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:05.130578 30439 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
W20260709 13:10:05.185369 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 159646us
I20260709 13:10:05.165964 30764 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.126238 30698 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:05.127869 30771 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.165285 30723 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:05.207396 30676 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.132838 30742 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:05.186944 30780 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:10:05.187098 30732 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:05.102713 30616 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.233919 30713 raft_consensus.cc:2749] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:05.171224 30758 raft_consensus.cc:740] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:05.173657 30679 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:05.284862 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 122820us
I20260709 13:10:05.207494 30439 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.297509 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.300217 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:05.298592 23453 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:10:05.311239 30676 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.284492 30780 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:05.304445 30793 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:05.296499 30778 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.320326 22719 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:10:05.323237 30439 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:05.323770 30791 log.cc:927] Time spent T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:10:05.322491 30679 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.329268 30698 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.304628 30723 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:05.342073 22794 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:05.345173 22901 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:10:05.346074 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.347920 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:05.349120 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 69896us
I20260709 13:10:05.350180 22902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.350363 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.351341 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.317304 30780 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:05.351752 22897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.352178 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:05.334825 30343 consensus_meta.cc:287] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.002s
I20260709 13:10:05.354648 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.330195 30737 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.354985 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.356169 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:05.362174 30783 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.225s	user 0.001s	sys 0.001s
I20260709 13:10:05.367638 30343 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:10:05.326565 30676 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:10:05.369347 30784 log.cc:927] Time spent T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 13:10:05.370208 30618 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:10:05.370464 30726 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.372844 30668 raft_consensus.cc:1064] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:10:05.353120 30608 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:10:05.376013 30787 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:10:05.386729 22896 raft_consensus.cc:1240] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Rejecting Update request from peer 7aad11e70ca94fce8f672db05f1c28c3 for earlier term 21. Current term is 22. Ops: [21.19-21.19]
I20260709 13:10:05.387351 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.395399 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.396746 30770 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.422907 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.428475 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.429934 30764 raft_consensus.cc:493] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:05.430374 30698 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:05.430452 30764 raft_consensus.cc:515] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:05.431200 30608 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Becoming Leader. State: Replica: 6a5191aaff76469bbf6c4735dc9e6509, State: Running, Role: LEADER
I20260709 13:10:05.431659 30778 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:05.432523 30764 leader_election.cc:290] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.407400 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.432137 30778 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.433537 30737 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:05.433648 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.433956 30723 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:05.434083 30737 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:05.434684 30778 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.434460 30723 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.435199 30778 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:05.436151 30737 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.397504 30616 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.414817 30771 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.388867 30702 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.436786 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.437160 30702 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:05.414499 30788 log.cc:927] Time spent T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.226s	user 0.002s	sys 0.000s
I20260709 13:10:05.430987 30698 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:05.439162 30702 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:05.342743 30758 consensus_queue.cc:260] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.19, Last appended by leader: 19, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.439374 30803 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:05.435101 30439 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:05.439819 30698 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.439893 30803 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:05.440541 30439 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:05.386430 30611 raft_consensus.cc:2804] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:10:05.441684 30611 raft_consensus.cc:697] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:05.442188 30803 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.436519 30723 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.442577 30611 consensus_queue.cc:237] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.435890 30778 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:05.422267 30742 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.409987 30623 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.444722 30778 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.446483 30623 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.447957 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.451828 30439 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.432102 30608 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.455057 30707 raft_consensus.cc:1064] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
W20260709 13:10:05.457504 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 64814.9us
I20260709 13:10:05.459056 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.460167 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:10:05.461974 23419 leader_election.cc:336] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:05.462534 23419 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:05.439965 30758 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:05.492529 30726 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 28 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.493508 30805 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:05.498742 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.499344 23450 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 27.
I20260709 13:10:05.501637 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.502910 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.503562 22823 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 28: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:10:05.498550 30780 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.505538 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.506381 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.513386 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.514119 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.507061 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.515127 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.515503 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.516317 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.511473 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd3d6688dfb4621b8ecfd11278d5b26" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.523073 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67449ba22ccb4938be8a5e28664cc566" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.483383 30623 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:05.504134 30742 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:05.529752 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.529770 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.512259 30723 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:10:05.534866 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 133825us
I20260709 13:10:05.538614 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.541955 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.550786 23450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:05.541447 30802 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:05.551642 23450 raft_consensus.cc:2393] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 25: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:10:05.535163 30723 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:10:05.552896 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.553902 30778 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:05.554446 30723 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:05.563163 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.546186 30707 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 15, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.567324 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:05.572324 30532 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:05.566910 22871 leader_election.cc:336] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:05.575249 30780 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.578189 22903 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:10:05.579998 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:05.580538 23451 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:10:05.582718 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.584132 22898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.584316 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.584666 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.585449 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.593840 30668 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.597486 30732 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.580654 30439 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:10:05.598141 30732 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.590184 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.619614 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:05.620527 23444 raft_consensus.cc:2393] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 in current term 25: Already voted for candidate c45e1805f049467697a7358fb566de2f in this term.
I20260709 13:10:05.623818 23450 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 25 LEARNER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:10:05.623605 23451 raft_consensus.cc:2468] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 28.
W20260709 13:10:05.625695 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 58451.6us
W20260709 13:10:05.628481 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 170015us
I20260709 13:10:05.629884 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.636274 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.601764 30770 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.639562 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.631832 30608 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:05.649484 30778 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:05.641005 30623 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:05.639050 30532 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:10:05.657843 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.658900 30532 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:05.634115 30780 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 13:10:05.659420 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:05.660693 30608 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:05.641817 30770 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.666517 30770 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.668201 30798 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:10:05.672351 22794 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:05.676120 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:05.691933 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:05.692312 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:05.695498 30608 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:10:05.695873 30608 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.696177 30608 raft_consensus.cc:3060] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:10:05.696918 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.697996 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:05.699352 30532 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:05.705736 30742 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:10:05.709532 30732 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:05.691449 30611 raft_consensus.cc:2804] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:10:05.711843 30611 raft_consensus.cc:697] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 28 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:05.712677 30611 consensus_queue.cc:237] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.717191 22901 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:10:05.687635 30770 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:05.720172 23448 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:10:05.739424 30323 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:10:05.766152 22871 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:05.766579 22871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 109578us
I20260709 13:10:05.766794 30809 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:05.770133 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.770742 22904 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 25.
I20260709 13:10:05.771394 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:05.771950 22831 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 25.
I20260709 13:10:05.772235 22903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:05.807399 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:05.812194 30790 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:10:05.812628 30790 raft_consensus.cc:493] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.812988 30790 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:10:05.824383 30742 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.824994 30742 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:05.827558 30742 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:05.856808 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:05.989269 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 124211us
W20260709 13:10:05.989365 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 105092us
W20260709 13:10:05.970127 30783 log.cc:927] Time spent T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:10:06.005512 30816 log.cc:927] Time spent T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.127s	user 0.003s	sys 0.013s
W20260709 13:10:06.014379 23419 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 87 similar messages]
I20260709 13:10:05.884286 30608 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:06.072295 30809 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:06.073026 30809 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:06.075987 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 85957.2us
I20260709 13:10:06.078764 30809 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:06.081638 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:06.083313 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f765d75643f47dd939002936724a315" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:06.085578 22794 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:06.087764 30627 raft_consensus.cc:2749] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:06.055717 30790 raft_consensus.cc:515] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:06.103155 30608 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 28 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:06.150337 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 143011us
I20260709 13:10:06.187565 23448 raft_consensus.cc:1275] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:10:06.189734 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:06.190445 23451 raft_consensus.cc:2393] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a5191aaff76469bbf6c4735dc9e6509 in current term 28: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:10:06.199540 30611 consensus_queue.cc:1048] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:06.172629 30758 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:06.207731 30758 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:06.210762 30758 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:06.211889 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:06.213070 30790 leader_election.cc:290] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:06.225188 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:06.226001 22831 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:10:06.228636 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:06.229214 22899 raft_consensus.cc:3060] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:10:06.237576 23446 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:10:06.241412 22899 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:10:06.252450 22831 raft_consensus.cc:2468] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:10:06.243682 30616 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.014s
W20260709 13:10:06.274785 30823 log.cc:927] Time spent T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: Append to log took a long time: real 0.255s	user 0.006s	sys 0.004s
I20260709 13:10:06.286218 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:06.288883 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3583f8fda02d46af9ef0ff2ae6f1b443" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:06.290009 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b7d2a5f73cc4afd98b29c55cc0eb72b" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:06.291307 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:06.291682 22831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:06.292098 22826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:06.292960 22826 raft_consensus.cc:2468] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a5191aaff76469bbf6c4735dc9e6509 in term 26.
I20260709 13:10:06.294108 22823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:06.301713 22719 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:06.304371 22638 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 135e2f7520a34b53aaf4ddb360a8a029 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica c45e1805f049467697a7358fb566de2f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.304776 30623 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:06.304955 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:06.306275 30623 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:06.306378 22719 leader_election.cc:304] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:06.307397 30623 raft_consensus.cc:2749] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:06.309211 30797 raft_consensus.cc:2955] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 12 to 13, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.310621 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:06.311347 30771 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:10:06.312536 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:06.313237 30771 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:06.320238 22822 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 6a5191aaff76469bbf6c4735dc9e6509: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:10:06.321698 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:06.339073 22822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:06.340664 23451 raft_consensus.cc:1275] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 13:10:06.338158 23416 leader_election.cc:304] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:06.353305 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ff32d0405c426786004110e1c1f955" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:06.354161 22830 raft_consensus.cc:2368] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 25. Current term is 26.
I20260709 13:10:06.358311 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:06.339650 22902 raft_consensus.cc:2955] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.366993 22825 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:10:06.350039 30725 raft_consensus.cc:2804] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:10:06.388370 22719 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:06.392491 22719 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:06.365774 30608 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:06.396731 30725 raft_consensus.cc:697] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 26 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:06.406370 30725 consensus_queue.cc:237] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:06.416440 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:06.417634 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:06.418460 30832 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:06.418731 30771 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:06.401398 30831 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:06.393426 30790 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:06.432395 30790 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:06.434516 30790 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:06.393771 30796 consensus_queue.cc:1048] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 13:10:06.439805 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:06.441938 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:06.453794 22901 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 22: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:10:06.454864 22868 leader_election.cc:400] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 25. Current term is 26.
I20260709 13:10:06.455303 22868 leader_election.cc:403] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:10:06.436882 30608 raft_consensus.cc:2804] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:10:06.456930 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:10:06.462031 22797 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:06.464540 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:06.458173 30608 raft_consensus.cc:493] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:06.466854 30608 raft_consensus.cc:3060] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:10:06.475857 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:06.476604 22825 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:10:06.482365 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2
I20260709 13:10:06.458015 30832 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:10:06.458143 30780 raft_consensus.cc:2749] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003) with higher term. Message: Invalid argument: T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3b626c41600d4737a178ebab8a7ebfc2 for earlier term 25. Current term is 26.
I20260709 13:10:06.463464 30830 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:06.494692 30343 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:06.495105 30343 raft_consensus.cc:493] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:06.495487 30343 raft_consensus.cc:3060] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:06.478513 30623 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 13:10:06.476605 30742 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:10:06.516827 30756 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:06.517400 30756 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:06.519733 30756 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:06.522750 30608 raft_consensus.cc:515] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:06.530355 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:06.538991 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 54704.3us
W20260709 13:10:06.554328 22797 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 90022.9us
I20260709 13:10:06.574044 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:06.575295 30837 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:06.578970 30837 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:10:06.507356 30343 raft_consensus.cc:515] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:06.587433 23446 raft_consensus.cc:2955] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.656908 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:06.671031 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:06.672093 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41fb40751eb74da580d83c9bb69ae58a" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:06.672883 22904 raft_consensus.cc:2393] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 22: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:10:06.679203 30608 leader_election.cc:290] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:06.683279 22830 raft_consensus.cc:2468] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:10:06.687963 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efec6b734c374a169f3dc3b729ef2f07" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:06.685035 23416 leader_election.cc:304] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:06.691706 22719 leader_election.cc:304] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:06.694079 30725 raft_consensus.cc:2804] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:06.696758 30832 raft_consensus.cc:2749] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:10:06.673024 30343 leader_election.cc:290] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:06.712780 22797 consensus_peers.cc:597] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:06.710573 30725 raft_consensus.cc:697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:06.718307 30725 consensus_queue.cc:237] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.19, Last appended by leader: 19, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:06.724753 22797 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:06.733918 22797 leader_election.cc:336] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:06.734340 22797 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:06.744313 30713 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:06.747208 22797 consensus_peers.cc:597] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:06.748474 22797 leader_election.cc:336] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:06.749063 22797 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:06.751159 30717 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:06.763370 30707 raft_consensus.cc:2955] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Committing config change with OpId 25.17: config changed from index 15 to 17, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.766361 23444 raft_consensus.cc:2955] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Committing config change with OpId 25.17: config changed from index 15 to 17, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.768220 22824 raft_consensus.cc:2955] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Committing config change with OpId 25.17: config changed from index 15 to 17, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.776528 22651 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 18 to 20, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 20 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.793151 22651 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 24 to 23, leader changed from <none> to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 23 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
I20260709 13:10:06.796335 22825 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:10:06.797859 30667 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:10:06.806524 22651 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 21 to 22, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 22 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:06.829464 22651 catalog_manager.cc:5697] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 21 to 26. New cstate: current_term: 26 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.831773 22651 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 13 to 23, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 23 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.833886 22651 catalog_manager.cc:5697] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 15 to 19, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:06.835602 22651 catalog_manager.cc:5697] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 9 to 13, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 13 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:06.845244 22652 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 345) took 5550 ms. Trace:
I20260709 13:10:06.845783 22652 rpcz_store.cc:276] 0709 13:10:01.294670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.294947 (+   277us) service_pool.cc:224] Handling call
0709 13:10:06.845140 (+5550193us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:04.533730 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:04.572855 (+ 39125us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:04.572892 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:10:04.572910 (+    18us) tablet.cc:662] Decoding operations
0709 13:10:04.577434 (+  4524us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:04.577485 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:04.577516 (+    31us) tablet.cc:685] Acquiring locks for 31 operations
0709 13:10:04.579865 (+  2349us) tablet.cc:701] Row locks acquired
0709 13:10:04.579884 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:10:04.580011 (+   127us) write_op.cc:270] Start()
0709 13:10:04.580122 (+   111us) write_op.cc:276] Timestamp: P: 1783602604579967 usec, L: 0
0709 13:10:04.580145 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:10:04.580780 (+   635us) log.cc:844] Serialized 12179 byte log entry
0709 13:10:05.080743 (+499963us) op_driver.cc:464] REPLICATION: finished
0709 13:10:05.367792 (+287049us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b4400022b4c. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb71b96db1 00007fcb71c34c85 00007fcb71c356a2 00007fcb749a71db 00007fcb749a2aaf 00007fcb71cf10ac 00007fcb7ab4bab9 00007fcb71d0d69b 00007fcb71d0b93f 00007fcb71d0dc02 00007fcb78831ef0
0709 13:10:05.542159 (+174367us) write_op.cc:301] APPLY: starting
0709 13:10:05.542247 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:05.648871 (+106624us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:05.648903 (+    32us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:05.703007 (+ 54104us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b54000ab4d4. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb74d64571 00007fcb74df832a 00007fcb74d7df14 00007fcb74d7dec3 00007fcb74d7de38 00007fcb74d7ddf3 00007fcb74d7d0dc 00007fcb74da6e16 00007fcb74d8ecbb 00007fcb74d7939d 00007fcb74d7cd75
0709 13:10:06.301662 (+598655us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b18000aa740. stack: 00007fcb71cdc343 00007fcb71cdc240 00007fcb7166b1d4 00007fcb7b3a9304 00007fcb7b3a9299 00007fcb7abd6de1 00007fcb7abe35b1 00007fcb74b3a0ec 00007fcb74b86c0a 00007fcb74b864f7 00007fcb74b86490 00007fcb74b3df48 00007fcb788521bb 00007fcb788995d4 00007fcb787458da 00007fcb787483cb
0709 13:10:06.426046 (+124384us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:06.426412 (+   366us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=62,key_file_lookups=62,delta_file_lookups=62,mrs_lookups=0
0709 13:10:06.426465 (+    53us) write_op.cc:312] APPLY: finished
0709 13:10:06.427024 (+   559us) log.cc:844] Serialized 268 byte log entry
0709 13:10:06.548004 (+120980us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:06.585488 (+ 37484us) write_op.cc:454] Released schema lock
0709 13:10:06.591464 (+  5976us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:06.629528 (+ 38064us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:06.630875 (+  1347us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17778,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":455921,"apply.run_cpu_time_us":80538,"apply.run_wall_time_us":1058387,"cfile_cache_hit":126,"cfile_cache_hit_bytes":133707,"num_ops":31,"prepare.queue_time_us":141327,"prepare.run_cpu_time_us":30224,"prepare.run_wall_time_us":103638,"raft.queue_time_us":359386,"raft.run_cpu_time_us":32530,"raft.run_wall_time_us":658622,"replication_time_us":481758,"spinlock_wait_cycles":27486080,"thread_start_us":770190,"threads_started":5,"wal-append.queue_time_us":227293,"wal-append.run_cpu_time_us":40750,"wal-append.run_wall_time_us":1207839}]]}
I20260709 13:10:06.875752 22899 raft_consensus.cc:1275] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:10:06.877285 30725 consensus_queue.cc:1048] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:10:06.837211 22651 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 15 to 23, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 23 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.886766 22651 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 15 to 20, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 20 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.888381 22651 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:06.905752 22651 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 12 to 19, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 19 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:06.910076 22651 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 12 to 21, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 21 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:10:06.946919 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:06.972486 30802 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:06.973021 30802 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:06.975090 30802 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:06.977581 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:06.978274 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:06.979784 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:06.981623 22719 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:06.982606 30831 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:07.040555 30831 raft_consensus.cc:493] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:07.041100 30831 raft_consensus.cc:515] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.043666 30831 leader_election.cc:290] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.045655 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.049078 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70588fa182854c9dbc786bdd4f807fc9" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.055703 22719 leader_election.cc:304] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.056941 30802 raft_consensus.cc:2749] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:07.126122 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:07.127044 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.128228 22904 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 20. (index mismatch)
I20260709 13:10:07.130710 30725 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:10:07.143774 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.159864 22824 raft_consensus.cc:1275] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 21 index: 19. Preceding OpId from leader: term: 22 index: 20. (index mismatch)
I20260709 13:10:07.161345 30667 consensus_queue.cc:1048] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:10:07.180052 22653 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 25 to 26, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 26 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.181113 22653 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 18 to 21, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 21 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.182488 22653 catalog_manager.cc:5697] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 17 to 23, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 23 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.183602 22653 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 21 to 23, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 23 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.187192 22653 catalog_manager.cc:5697] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 22 to 25, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 25 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.190042 22653 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 20 to 21, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 21 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.192885 22653 catalog_manager.cc:5697] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 21 to 23, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 23 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.194666 22653 catalog_manager.cc:5697] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 20 to 28, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 28 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.196944 22653 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:10:07.199489 22651 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 342) took 11398 ms (client timeout 15000 ms). Trace:
I20260709 13:10:07.200759 22653 catalog_manager.cc:5697] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.201833 30803 raft_consensus.cc:493] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:07.202411 30803 raft_consensus.cc:515] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.202399 22653 catalog_manager.cc:5697] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.206872 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:07.206980 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.206506 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ca1af745cb4fd48a40af06ae1c43ca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.204310 30803 leader_election.cc:290] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:07.199779 22651 rpcz_store.cc:269] 0709 13:09:55.799710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.804467 (+  4757us) service_pool.cc:224] Handling call
0709 13:10:07.197883 (+11393416us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:06.994388 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:06.997522 (+  3134us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:06.997559 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:10:06.997578 (+    19us) tablet.cc:662] Decoding operations
0709 13:10:07.006000 (+  8422us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:07.006074 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:07.006099 (+    25us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:10:07.007034 (+   935us) tablet.cc:701] Row locks acquired
0709 13:10:07.007053 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:10:07.007206 (+   153us) write_op.cc:270] Start()
0709 13:10:07.007315 (+   109us) write_op.cc:276] Timestamp: P: 1783602607007158 usec, L: 0
0709 13:10:07.007337 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:10:07.007997 (+   660us) log.cc:844] Serialized 4776 byte log entry
0709 13:10:07.046067 (+ 38070us) op_driver.cc:464] REPLICATION: finished
0709 13:10:07.049015 (+  2948us) write_op.cc:301] APPLY: starting
0709 13:10:07.049097 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:07.055063 (+  5966us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:07.055160 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:07.071530 (+ 16370us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:07.071760 (+   230us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:10:07.071919 (+   159us) write_op.cc:312] APPLY: finished
0709 13:10:07.072478 (+   559us) log.cc:844] Serialized 113 byte log entry
0709 13:10:07.072639 (+   161us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:07.072764 (+   125us) write_op.cc:454] Released schema lock
0709 13:10:07.072809 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:07.073181 (+   372us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:07.073226 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41796,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":2699,"apply.run_cpu_time_us":14559,"apply.run_wall_time_us":24116,"cfile_cache_hit":48,"cfile_cache_hit_bytes":50160,"num_ops":12,"prepare.queue_time_us":24110,"prepare.run_cpu_time_us":4277,"prepare.run_wall_time_us":14132,"replication_time_us":38576,"thread_start_us":23494,"threads_started":1}]]}
I20260709 13:10:07.210423 22719 leader_election.cc:304] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.209216 22653 catalog_manager.cc:5697] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 12 to 15, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 15 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:10:07.225870 22797 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.227895 30770 raft_consensus.cc:2749] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:07.227854 22653 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 17 to 22, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 22 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.234400 22653 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 23 to 25, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67), now has a pending config: VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65), VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66), VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67), VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 25 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } } pending_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.240942 30770 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.241511 30770 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:07.243583 30770 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:07.248117 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:07.249534 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.253888 22719 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:07.254683 30831 raft_consensus.cc:2749] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:10:07.267084 22797 consensus_peers.cc:597] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:07.280989 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.327342 22904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84ea43361e534fbebad704327c9f6367"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46616
I20260709 13:10:07.327931 22904 raft_consensus.cc:606] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Received request to transfer leadership
I20260709 13:10:07.333853 22904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84ea43361e534fbebad704327c9f6367"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46616
I20260709 13:10:07.334375 22904 raft_consensus.cc:606] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Received request to transfer leadership
I20260709 13:10:07.341269 22904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84ea43361e534fbebad704327c9f6367"
dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46616
I20260709 13:10:07.341784 22904 raft_consensus.cc:606] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Received request to transfer leadership
W20260709 13:10:07.359624 22702 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:10:07.360081 22702 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 84ea43361e534fbebad704327c9f6367 already in progress
W20260709 13:10:07.364559 22702 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 84ea43361e534fbebad704327c9f6367 already in progress
I20260709 13:10:07.363951 30835 raft_consensus.cc:993] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: : Instructing follower c45e1805f049467697a7358fb566de2f to start an election
I20260709 13:10:07.367580 30835 raft_consensus.cc:1081] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Signalling peer c45e1805f049467697a7358fb566de2f to start an election
W20260709 13:10:07.360446 22797 consensus_peers.cc:597] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.369529 23444 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84ea43361e534fbebad704327c9f6367"
dest_uuid: "c45e1805f049467697a7358fb566de2f"
 from {username='slave'} at 127.0.0.1:38064
I20260709 13:10:07.370209 23444 raft_consensus.cc:493] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:07.370543 30831 raft_consensus.cc:493] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:07.371270 30831 raft_consensus.cc:515] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.374800 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.375777 30802 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:07.376291 30802 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.377126 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6436de6ceb37488b9ee8b6d261356a40" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.378382 30802 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.379886 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.382332 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac8ca58483824d0bbd1e56d06088b4da" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.384708 22719 leader_election.cc:304] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.385649 30770 raft_consensus.cc:2749] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:07.370610 23444 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:10:07.400947 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:07.401736 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.373186 30831 leader_election.cc:290] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.385936 22719 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.405172 30770 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:07.418413 30831 raft_consensus.cc:493] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:07.418902 30831 raft_consensus.cc:515] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.420758 30831 leader_election.cc:290] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.437778 30770 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:07.438338 30770 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:07.438985 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.440505 30770 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:07.442198 21629 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 22781)
I20260709 13:10:07.447009 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42e3464af3e4e5eb580790b400a6eff" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.450742 30770 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:07.451234 30770 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.453974 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:07.454123 30770 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.454595 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.456595 23444 raft_consensus.cc:515] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.465050 22719 leader_election.cc:304] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
W20260709 13:10:07.465798 23419 leader_election.cc:336] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:07.466976 30803 raft_consensus.cc:2749] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:10:07.467351 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): ChangeConfig:REMOVE_PEER RPC failed for tablet 135e2f7520a34b53aaf4ddb360a8a029: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:10:07.471135 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: true dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
W20260709 13:10:07.471441 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.471763 22830 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:10:07.472662 23444 leader_election.cc:290] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:07.473635 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:07.475021 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.476820 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:07.476922 22797 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.481735 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ea43361e534fbebad704327c9f6367" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: true dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:07.482470 22904 raft_consensus.cc:3055] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:10:07.482820 22904 raft_consensus.cc:740] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:07.485038 22719 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:07.486127 30831 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:07.486936 22830 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:10:07.485143 22904 consensus_queue.cc:260] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.490801 22904 raft_consensus.cc:3060] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:10:07.519573 30831 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:07.520121 30831 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.524158 30831 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.524997 22719 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.526031 30770 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:07.528354 22904 raft_consensus.cc:2468] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:10:07.530380 23416 leader_election.cc:304] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:07.531329 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:07.534009 30790 raft_consensus.cc:2804] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:10:07.534549 30790 raft_consensus.cc:697] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 26 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:07.535305 30790 consensus_queue.cc:237] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:07.541895 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.554663 22653 catalog_manager.cc:5697] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 22 to 26, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 26 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.560485 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bbeaee00aa942ccb232c812c9e14827" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.561728 22653 catalog_manager.cc:5697] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 21 to 28, leader changed from c45e1805f049467697a7358fb566de2f (127.21.31.68) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 28 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.563704 22653 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 12 to 13, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.565158 22653 catalog_manager.cc:5697] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 21 to 22, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 22 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.567056 22653 catalog_manager.cc:5697] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 23 to 25, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 25 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.568380 22653 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 15 to 17, term changed from 25 to 26, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to <none>, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65), VOTER 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66), VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67), VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 26 committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:07.580374 30803 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:07.580935 30803 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.583037 30803 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.585639 22719 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:10:07.596519 22797 consensus_peers.cc:597] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.599112 30803 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:07.599561 30831 raft_consensus.cc:493] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:07.600692 30831 raft_consensus.cc:515] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:07.601351 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.603684 30831 leader_election.cc:290] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.617498 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.618026 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.618480 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a98a0b933184c5590c80b44a6e3a544" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.618667 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a28fec9b1516463db9c7feac432f9ad5" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.619558 30802 raft_consensus.cc:493] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:07.620095 30802 raft_consensus.cc:515] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.622413 30802 leader_election.cc:290] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.632840 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.640568 22719 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.641989 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea49b0ee95d4eafbb647436a98fcaa1" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.645228 30802 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:07.646476 22719 leader_election.cc:304] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.659307 30831 raft_consensus.cc:2749] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:10:07.660529 30770 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:07.662719 22719 leader_election.cc:304] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.661041 30770 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.715293 30802 raft_consensus.cc:2749] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:07.751973 22719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 89308.1us
I20260709 13:10:07.761837 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.761060 30802 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:07.763293 30770 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:07.784785 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.785950 30802 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:07.788160 30802 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:07.795898 22650 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 341) took 29494 ms (client timeout 15000 ms). Trace:
W20260709 13:10:07.796270 22650 rpcz_store.cc:269] 0709 13:09:38.301523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.313547 (+ 12024us) service_pool.cc:224] Handling call
0709 13:10:07.795711 (+29482164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":200390,"reported_tablets":46}
W20260709 13:10:07.809935 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 86333.9us
I20260709 13:10:07.814646 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c5a324a80f42d4965b69d86915a65d" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.831470 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:07.832013 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cc99e6a1f14071a587af061e82fee0" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.836011 30802 raft_consensus.cc:493] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:07.836513 30802 raft_consensus.cc:515] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:07.838354 22797 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.838450 30802 leader_election.cc:290] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.832623 22648 catalog_manager.cc:5697] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 17 to 20, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 20 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.845028 22648 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 21 to 22, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65). New cstate: current_term: 22 leader_uuid: "6a5191aaff76469bbf6c4735dc9e6509" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.851362 22648 catalog_manager.cc:5697] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 reported cstate change: term changed from 17 to 21, leader changed from 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 21 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } }
I20260709 13:10:07.858505 30802 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:07.860349 30802 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:07.863229 22719 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.865162 30831 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:07.867455 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.872817 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7701b9b4e69441f7ae9de9554134de69" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.863587 30802 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:07.878674 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:07.879464 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3833f41661774b918ddf80499fd85f2f" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:07.880719 22653 catalog_manager.cc:5697] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f reported cstate change: leader changed from <none> to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 26 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.890152 22719 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:07.891798 30861 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:07.896739 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.899995 22719 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.908674 30861 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:10:07.912366 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.906785 30802 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:07.913270 30802 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:07.913964 22797 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:07.915267 22719 leader_election.cc:304] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:07.917418 30770 raft_consensus.cc:2749] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:07.921173 30802 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:07.935822 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:07.938724 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c8f692d4994a148c306609ab807ab7" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:07.946468 22719 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:07.947870 30861 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:07.972009 22830 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:10:07.973382 30667 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:10:08.007704 22901 raft_consensus.cc:1275] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:10:08.009337 30667 consensus_queue.cc:1048] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:10:08.011526 22797 consensus_peers.cc:597] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:08.040627 22648 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 339) took 1320 ms. Trace:
I20260709 13:10:08.041949 22648 rpcz_store.cc:276] 0709 13:10:06.719862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.720105 (+   243us) service_pool.cc:224] Handling call
0709 13:10:08.040541 (+1320436us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:07.861170 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:07.862173 (+  1003us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:07.862209 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:10:07.862228 (+    19us) tablet.cc:662] Decoding operations
0709 13:10:07.862750 (+   522us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:07.862802 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:07.862827 (+    25us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:10:07.863115 (+   288us) tablet.cc:701] Row locks acquired
0709 13:10:07.863131 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:10:07.863250 (+   119us) write_op.cc:270] Start()
0709 13:10:07.863359 (+   109us) write_op.cc:276] Timestamp: P: 1783602607863207 usec, L: 0
0709 13:10:07.863381 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:10:07.863912 (+   531us) log.cc:844] Serialized 1308 byte log entry
0709 13:10:07.866347 (+  2435us) op_driver.cc:464] REPLICATION: finished
0709 13:10:07.867660 (+  1313us) write_op.cc:301] APPLY: starting
0709 13:10:07.867729 (+    69us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:07.869440 (+  1711us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:07.869522 (+    82us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:07.871278 (+  1756us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:07.871442 (+   164us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:10:07.871490 (+    48us) write_op.cc:312] APPLY: finished
0709 13:10:07.871660 (+   170us) log.cc:844] Serialized 41 byte log entry
0709 13:10:07.875620 (+  3960us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:07.875702 (+    82us) write_op.cc:454] Released schema lock
0709 13:10:07.875749 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:07.876042 (+   293us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:07.876072 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":65555,"reported_tablets":46,"spinlock_wait_cycles":324480,"child_traces":[["op",{"apply.queue_time_us":1115,"apply.run_cpu_time_us":4564,"apply.run_wall_time_us":8360,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12540,"num_ops":3,"prepare.queue_time_us":99,"prepare.run_cpu_time_us":2550,"prepare.run_wall_time_us":2991,"replication_time_us":2862}]]}
I20260709 13:10:08.054778 22652 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41804 (request call id 346) took 1081 ms. Trace:
I20260709 13:10:08.055346 22652 rpcz_store.cc:276] 0709 13:10:06.973672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.973824 (+   152us) service_pool.cc:224] Handling call
0709 13:10:08.054716 (+1080892us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33225,"reported_tablets":22}
I20260709 13:10:08.063508 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:10:08.067456 22797 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:08.069759 21629 tablet_replica.cc:333] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.074282 21629 raft_consensus.cc:2243] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.075423 21629 raft_consensus.cc:2272] T a42e3464af3e4e5eb580790b400a6eff P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.078413 21629 tablet_replica.cc:333] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.079228 21629 raft_consensus.cc:2243] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.079995 21629 raft_consensus.cc:2272] T efec6b734c374a169f3dc3b729ef2f07 P 6a5191aaff76469bbf6c4735dc9e6509 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:08.093942 30868 log.cc:927] Time spent T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:10:08.097244 21629 tablet_replica.cc:333] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.098315 21629 raft_consensus.cc:2243] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.099117 21629 raft_consensus.cc:2272] T 1879799f5d0c462c9d2c95bd8a88862e P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.111233 21629 tablet_replica.cc:333] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.112118 21629 raft_consensus.cc:2243] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.113080 21629 raft_consensus.cc:2272] T 6ce4678cb66d4d1e8325b447eaf2a630 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.125054 21629 tablet_replica.cc:333] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.125895 21629 raft_consensus.cc:2243] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.126485 21629 raft_consensus.cc:2272] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.147018 21629 tablet_replica.cc:333] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.147789 21629 raft_consensus.cc:2243] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.148391 21629 raft_consensus.cc:2272] T a28fec9b1516463db9c7feac432f9ad5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.149971 30802 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.150429 30802 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:08.151065 21629 tablet_replica.cc:333] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.151927 21629 raft_consensus.cc:2243] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.152343 30802 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:08.152506 21629 raft_consensus.cc:2272] T f8c8f692d4994a148c306609ab807ab7 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.155331 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:08.152870 30802 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.155982 30802 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:08.157807 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d74135bf874a0bb4b857b686effcfd" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:10:08.158953 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:08.160447 30802 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:08.161372 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:08.168200 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53f146bb25444ce8d2072559efe5e59" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:08.168373 22719 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
W20260709 13:10:08.169848 22797 consensus_peers.cc:597] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:08.171530 22719 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:08.170573 30861 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:08.173736 30803 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:10:08.179320 21629 tablet_replica.cc:333] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.180248 21629 raft_consensus.cc:2243] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.181268 21629 raft_consensus.cc:2272] T 8da6707b1a3d47a48c3671a6b7401196 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.187151 21629 tablet_replica.cc:333] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.188099 21629 raft_consensus.cc:2243] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.198060 30802 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.198563 30802 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:08.199793 21629 raft_consensus.cc:2272] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.201274 30802 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:08.202670 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:08.203047 21629 tablet_replica.cc:333] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.203859 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f881fed251c4393ac053086bd572549" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:08.219781 22719 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:08.221051 30802 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:08.222062 21629 raft_consensus.cc:2243] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.222612 21629 raft_consensus.cc:2272] T d0d74135bf874a0bb4b857b686effcfd P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.223423 30802 raft_consensus.cc:493] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.225600 21629 tablet_replica.cc:333] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.223981 30802 raft_consensus.cc:515] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:08.226415 21629 raft_consensus.cc:2243] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.227378 21629 raft_consensus.cc:2272] T f7d0e2e573344ec49ba2412e5de1d519 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.230502 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:08.237775 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00324d171dc0468fa259f13c1d4d74ea" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:08.239364 22719 leader_election.cc:304] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:08.240162 30861 raft_consensus.cc:2749] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:08.236539 30802 leader_election.cc:290] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:08.244477 21629 tablet_replica.cc:333] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.245256 21629 raft_consensus.cc:2243] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.251322 21629 raft_consensus.cc:2272] T 00324d171dc0468fa259f13c1d4d74ea P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.264132 21629 tablet_replica.cc:333] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.264920 21629 raft_consensus.cc:2243] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.265630 21629 raft_consensus.cc:2272] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.269501 21629 tablet_replica.cc:333] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
W20260709 13:10:08.273387 22797 consensus_peers.cc:597] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:08.279860 21629 raft_consensus.cc:2243] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.281755 21629 raft_consensus.cc:2272] T 70588fa182854c9dbc786bdd4f807fc9 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.289757 21629 tablet_replica.cc:333] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
W20260709 13:10:08.301800 22649 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33010 (request call id 229) took 17271 ms (client timeout 15000 ms). Trace:
W20260709 13:10:08.302311 22649 rpcz_store.cc:269] 0709 13:09:51.030637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.030928 (+   291us) service_pool.cc:224] Handling call
0709 13:10:08.301685 (+17270757us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141520,"reported_tablets":42}
I20260709 13:10:08.305158 21629 raft_consensus.cc:2243] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.306243 21629 raft_consensus.cc:2272] T 2147efadd4ea46a09bda395402a10299 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.308907 21629 tablet_replica.cc:333] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.309664 21629 raft_consensus.cc:2243] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.323793 21629 raft_consensus.cc:2272] T 3a98a0b933184c5590c80b44a6e3a544 P 6a5191aaff76469bbf6c4735dc9e6509 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.351876 21629 tablet_replica.cc:333] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
W20260709 13:10:08.351872 22654 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41780 (request call id 338) took 18157 ms (client timeout 15000 ms). Trace:
W20260709 13:10:08.352548 22654 rpcz_store.cc:269] 0709 13:09:50.194245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.194516 (+   271us) service_pool.cc:224] Handling call
0709 13:10:08.351675 (+18157159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":532548,"reported_tablets":46}
I20260709 13:10:08.353154 21629 raft_consensus.cc:2243] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.353991 21629 raft_consensus.cc:2272] T dbd3d6688dfb4621b8ecfd11278d5b26 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.356841 21629 tablet_replica.cc:333] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.366822 21629 raft_consensus.cc:2243] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.374527 22651 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41790 (request call id 343) took 1031 ms. Trace:
I20260709 13:10:08.375005 22651 rpcz_store.cc:276] 0709 13:10:07.342942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:07.343205 (+   263us) service_pool.cc:224] Handling call
0709 13:10:08.374457 (+1031252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54674,"reported_tablets":42}
I20260709 13:10:08.379611 21629 raft_consensus.cc:2272] T 267f1f9ab3f742418b8505ba4859ab18 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.415009 21629 tablet_replica.cc:333] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.427021 21629 raft_consensus.cc:2243] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.443995 21629 raft_consensus.cc:2272] T ac8ca58483824d0bbd1e56d06088b4da P 6a5191aaff76469bbf6c4735dc9e6509 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.460765 21629 tablet_replica.cc:333] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.460685 22824 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:08.461653 21629 raft_consensus.cc:2243] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.467239 21629 raft_consensus.cc:2272] T 4bbeaee00aa942ccb232c812c9e14827 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.474233 21629 tablet_replica.cc:333] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.479313 21629 raft_consensus.cc:2243] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.483728 21629 raft_consensus.cc:2272] T 67449ba22ccb4938be8a5e28664cc566 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.487730 22901 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:10:08.487805 23448 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:10:08.491138 30867 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:08.492341 21629 tablet_replica.cc:333] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.493396 21629 raft_consensus.cc:2243] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.494093 21629 raft_consensus.cc:2272] T 6436de6ceb37488b9ee8b6d261356a40 P 6a5191aaff76469bbf6c4735dc9e6509 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.493201 30726 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:08.496788 21629 tablet_replica.cc:333] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.497586 21629 raft_consensus.cc:2243] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.498167 21629 raft_consensus.cc:2272] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.501296 21629 tablet_replica.cc:333] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.502532 21629 raft_consensus.cc:2243] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.503298 21629 raft_consensus.cc:2272] T 42c5a324a80f42d4965b69d86915a65d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.505896 21629 tablet_replica.cc:333] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.507061 21629 raft_consensus.cc:2243] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.508275 21629 raft_consensus.cc:2272] T ab1914222f984a8f9db2e3956a97bad0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.518352 30862 raft_consensus.cc:493] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader c45e1805f049467697a7358fb566de2f)
I20260709 13:10:08.518896 30862 raft_consensus.cc:515] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:08.520989 30862 leader_election.cc:290] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:08.522449 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:08.522912 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c3bd6e29d8478b8aa6c100c8fd84c2" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:08.523604 30726 raft_consensus.cc:2955] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:08.526589 22901 raft_consensus.cc:2955] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:08.531518 22719 leader_election.cc:304] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:08.532984 30862 raft_consensus.cc:2749] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:10:08.539209 21629 tablet_replica.cc:333] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.539997 21629 raft_consensus.cc:2243] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.540208 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4e4b3212e80646379ccf77b5952a4363 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:08.540940 21629 raft_consensus.cc:2272] T 8f0563f7efc8452c86a8c67c0189fe81 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.544543 21629 tablet_replica.cc:333] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.545300 21629 raft_consensus.cc:2243] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.546113 21629 pending_rounds.cc:70] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Trying to abort 2 pending ops.
I20260709 13:10:08.546332 21629 pending_rounds.cc:77] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting op as it isn't in flight: id { term: 19 index: 16 } timestamp: 7305636257546649600 op_type: NO_OP noop_request { }
I20260709 13:10:08.546674 21629 raft_consensus.cc:2889] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:08.546949 21629 pending_rounds.cc:77] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509: Aborting op as it isn't in flight: id { term: 18 index: 15 } timestamp: 7305636236294750208 op_type: NO_OP noop_request { }
I20260709 13:10:08.547261 21629 raft_consensus.cc:2889] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:08.547570 21629 raft_consensus.cc:2272] T 1f881fed251c4393ac053086bd572549 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.551293 23446 raft_consensus.cc:2955] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:08.557484 30862 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:08.557998 30862 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:08.559872 30862 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:08.562085 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:08.566567 22654 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 13 to 14, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 19 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:08.572348 21629 tablet_replica.cc:333] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.573187 21629 raft_consensus.cc:2243] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.573688 21629 raft_consensus.cc:2272] T f53f146bb25444ce8d2072559efe5e59 P 6a5191aaff76469bbf6c4735dc9e6509 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.576155 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce771dbb93248d397c8d5b222b27fca" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:08.577489 22719 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:08.578272 30861 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:08.580541 21629 tablet_replica.cc:333] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.581343 21629 raft_consensus.cc:2243] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.582435 21629 raft_consensus.cc:2272] T 135e2f7520a34b53aaf4ddb360a8a029 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.585074 21629 tablet_replica.cc:333] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.585911 21629 raft_consensus.cc:2243] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.595748 21629 raft_consensus.cc:2272] T 18a127a20cfa4496bad51cb634bd0d0f P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.598238 21629 tablet_replica.cc:333] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.599152 21629 raft_consensus.cc:2243] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.599728 21629 raft_consensus.cc:2272] T 53ca1af745cb4fd48a40af06ae1c43ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.602139 21629 tablet_replica.cc:333] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.603015 21629 raft_consensus.cc:2243] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.603720 21629 raft_consensus.cc:2272] T 7701b9b4e69441f7ae9de9554134de69 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.606928 21629 tablet_replica.cc:333] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.607839 21629 raft_consensus.cc:2243] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.608754 21629 raft_consensus.cc:2272] T 4e4b3212e80646379ccf77b5952a4363 P 6a5191aaff76469bbf6c4735dc9e6509 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.612483 21629 tablet_replica.cc:333] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.613334 21629 raft_consensus.cc:2243] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.614341 21629 raft_consensus.cc:2272] T 4b9578eb52c342078c7555d5100bdcad P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.616993 21629 tablet_replica.cc:333] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.617079 30861 raft_consensus.cc:493] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:08.617687 21629 raft_consensus.cc:2243] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.617646 30861 raft_consensus.cc:515] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:08.618594 21629 raft_consensus.cc:2272] T 4d8a8f3cf129492eb679da24703f5b84 P 6a5191aaff76469bbf6c4735dc9e6509 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.620033 30861 leader_election.cc:290] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:08.621236 21629 tablet_replica.cc:333] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.621371 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:08.621733 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccab55924ea9473faf1d4d97313e9284" candidate_uuid: "6a5191aaff76469bbf6c4735dc9e6509" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:08.621987 21629 raft_consensus.cc:2243] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.623031 21629 raft_consensus.cc:2272] T 82d0d73de20d49d3a64fef8b052288e5 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.624944 22719 leader_election.cc:304] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a5191aaff76469bbf6c4735dc9e6509; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:08.625484 21629 tablet_replica.cc:333] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.626408 21629 raft_consensus.cc:2243] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.626817 30861 raft_consensus.cc:2749] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:08.627339 21629 raft_consensus.cc:2272] T 847c8a2cc0bb4233a1d55ca04643418d P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.636104 21629 tablet_replica.cc:333] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.637159 21629 raft_consensus.cc:2243] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.638155 21629 raft_consensus.cc:2272] T 264d1889b1f144c5897b33c834b1beef P 6a5191aaff76469bbf6c4735dc9e6509 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.640813 21629 tablet_replica.cc:333] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.641532 21629 raft_consensus.cc:2243] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.642107 21629 raft_consensus.cc:2272] T 3833f41661774b918ddf80499fd85f2f P 6a5191aaff76469bbf6c4735dc9e6509 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.645486 21629 tablet_replica.cc:333] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.646569 21629 raft_consensus.cc:2243] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.647343 21629 raft_consensus.cc:2272] T e4cc99e6a1f14071a587af061e82fee0 P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.650040 21629 tablet_replica.cc:333] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.650792 21629 raft_consensus.cc:2243] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.651530 21629 raft_consensus.cc:2272] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 6a5191aaff76469bbf6c4735dc9e6509 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.654842 21629 tablet_replica.cc:333] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.655866 21629 raft_consensus.cc:2243] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.656960 21629 raft_consensus.cc:2272] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 6a5191aaff76469bbf6c4735dc9e6509 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.659801 21629 tablet_replica.cc:333] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.660735 21629 raft_consensus.cc:2243] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:08.662109 21629 raft_consensus.cc:2272] T c7d4d50470a044f698ff4852a4b634ca P 6a5191aaff76469bbf6c4735dc9e6509 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.666352 21629 tablet_replica.cc:333] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.667014 21629 raft_consensus.cc:2243] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.667544 21629 raft_consensus.cc:2272] T 6ce771dbb93248d397c8d5b222b27fca P 6a5191aaff76469bbf6c4735dc9e6509 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.670732 21629 tablet_replica.cc:333] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.671344 21629 raft_consensus.cc:2243] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.671859 21629 raft_consensus.cc:2272] T ccab55924ea9473faf1d4d97313e9284 P 6a5191aaff76469bbf6c4735dc9e6509 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:08.674783 21629 tablet_replica.cc:333] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509: stopping tablet replica
I20260709 13:10:08.675402 21629 raft_consensus.cc:2243] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:08.676025 21629 raft_consensus.cc:2272] T 84ea43361e534fbebad704327c9f6367 P 6a5191aaff76469bbf6c4735dc9e6509 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:08.694180 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4e4b3212e80646379ccf77b5952a4363: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
W20260709 13:10:09.047240 22797 consensus_peers.cc:597] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:09.284873 30865 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.285676 30864 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.286578 30865 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.294150 30865 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:09.286626 30864 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.304834 30864 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:09.326092 22871 leader_election.cc:336] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.331454 30889 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.338101 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:09.339124 22830 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:10:09.332568 30889 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.347612 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:10:09.349877 22797 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:09.350781 30875 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:09.351181 30875 raft_consensus.cc:493] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.351522 30875 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:10:09.358101 22797 leader_election.cc:336] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:09.364501 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 67927.6us
I20260709 13:10:09.368717 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:09.374574 22871 leader_election.cc:336] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.383198 30889 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:09.394743 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:09.396965 22868 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:09.397858 30889 raft_consensus.cc:2749] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:09.435314 30889 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.436118 30889 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.441232 30889 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:09.444602 22871 leader_election.cc:336] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.446174 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 27 candidate_status { last_received { term: 26 index: 23 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:09.447649 22868 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:09.449079 30889 raft_consensus.cc:2749] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:10:09.456692 30889 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.457522 30889 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:09.465449 22871 leader_election.cc:336] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.466697 30889 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:09.467892 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:09.469480 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:09.470628 30865 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:09.476986 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.473865 30875 raft_consensus.cc:515] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.479530 22899 raft_consensus.cc:2393] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7aad11e70ca94fce8f672db05f1c28c3 in current term 23: Already voted for candidate 3b626c41600d4737a178ebab8a7ebfc2 in this term.
I20260709 13:10:09.486198 30890 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.486917 30890 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.500176 30875 leader_election.cc:290] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:09.500886 23419 leader_election.cc:336] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.503573 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ce4678cb66d4d1e8325b447eaf2a630" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:09.504317 22824 raft_consensus.cc:3060] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:10:09.504720 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:09.505297 22871 leader_election.cc:336] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.507892 30890 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.514986 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 26 index: 23 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:09.515702 22899 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:10:09.516760 30867 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.517454 30867 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.519958 30867 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.522601 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.524559 30790 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:10:09.524955 30790 raft_consensus.cc:493] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.525307 30790 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:10:09.527436 30865 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.528080 30865 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:09.528784 22797 leader_election.cc:336] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.530352 30865 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:09.534837 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:09.535737 22830 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:10:09.536568 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:10:09.537550 30875 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:09.538020 30875 raft_consensus.cc:493] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.538496 30875 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:09.536579 22794 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.549674 30867 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.550401 30867 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.557922 30867 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:09.561806 22797 consensus_peers.cc:597] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.562707 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:09.562597 30875 raft_consensus.cc:515] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.564908 22899 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:10:09.566629 30875 leader_election.cc:290] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:09.568037 22871 leader_election.cc:336] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.567873 30790 raft_consensus.cc:515] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:09.569270 22797 leader_election.cc:336] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.573660 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da6707b1a3d47a48c3671a6b7401196" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:09.574595 22830 raft_consensus.cc:3060] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:09.577466 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:09.579162 23419 leader_election.cc:336] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.584029 30790 leader_election.cc:290] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.585152 22830 raft_consensus.cc:2468] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:10:09.571810 22824 raft_consensus.cc:2468] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:10:09.569064 30867 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.586545 30867 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.588842 30867 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.589833 30894 raft_consensus.cc:2749] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:09.571678 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.592000 22868 leader_election.cc:304] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.593516 30865 raft_consensus.cc:2804] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:09.594059 30865 raft_consensus.cc:697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:09.594888 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.595198 30865 consensus_queue.cc:237] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.596489 30894 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:09.596880 30894 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.597254 30894 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:09.599524 30790 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.600279 30790 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:09.600920 22871 leader_election.cc:336] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.605713 22868 leader_election.cc:304] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.606832 30875 raft_consensus.cc:2804] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:09.607617 30875 raft_consensus.cc:697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:09.608736 30875 consensus_queue.cc:237] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:09.611493 22797 leader_election.cc:336] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:09.613680 23419 leader_election.cc:336] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.614053 30790 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.616251 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267f1f9ab3f742418b8505ba4859ab18" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 26 index: 23 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:09.616870 22901 raft_consensus.cc:3060] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:10:09.619619 22794 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:09.620584 30864 raft_consensus.cc:2749] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:09.624876 30891 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.624811 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:09.625747 30891 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.628721 30891 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:09.616612 30894 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.635468 30894 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.637331 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:09.637517 22794 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.639320 30867 raft_consensus.cc:2749] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:09.639550 22901 raft_consensus.cc:2468] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 27.
I20260709 13:10:09.639335 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12cb0ef4bd490aa6a0afd9cd76ff18" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:09.640882 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:09.641194 22899 raft_consensus.cc:3060] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:09.645824 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.653627 22794 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f
I20260709 13:10:09.654348 30891 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:09.658561 22904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:09.659166 22904 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:10:09.660741 23416 leader_election.cc:304] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.661942 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.663376 30893 raft_consensus.cc:2804] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:10:09.663861 30893 raft_consensus.cc:697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 27 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:09.665575 30873 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:09.665969 30873 raft_consensus.cc:493] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.666316 30873 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:09.664902 30893 consensus_queue.cc:237] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 26.23, Last appended by leader: 23, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:09.677497 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.678193 30875 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.678790 30875 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:09.684696 22797 leader_election.cc:336] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.686923 30875 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:09.687445 22797 leader_election.cc:336] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.690174 30873 raft_consensus.cc:515] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.710225 30873 leader_election.cc:290] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.692698 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:09.697247 22649 catalog_manager.cc:5697] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 22 to 23, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 23 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:09.690809 22899 raft_consensus.cc:2468] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 22.
I20260709 13:10:09.714861 30890 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:10:09.698825 22871 leader_election.cc:336] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.720836 30890 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.723531 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:09.724459 30875 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:09.725965 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:09.726649 22830 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:10:09.728135 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:09.723585 30890 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:09.729223 30890 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:09.729668 30890 raft_consensus.cc:493] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.731081 30890 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:09.731863 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d0e2e573344ec49ba2412e5de1d519" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:09.732410 22901 raft_consensus.cc:3060] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:09.738660 22794 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.739706 30891 raft_consensus.cc:2804] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:09.740202 30891 raft_consensus.cc:697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:10:09.741829 22797 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.743896 30893 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.744524 22901 raft_consensus.cc:2468] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:10:09.745888 30865 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.746624 30865 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.748158 23416 leader_election.cc:304] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:09.748920 30873 raft_consensus.cc:2804] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:10:09.749089 23419 leader_election.cc:336] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.750547 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:09.751142 22824 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:10:09.753013 30865 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:09.753991 30867 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.754627 30867 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.756870 30867 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:09.759589 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:09.744508 30893 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.741147 30891 consensus_queue.cc:237] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.763144 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:09.762001 22871 leader_election.cc:336] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.766741 22868 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:09.749452 30873 raft_consensus.cc:697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:09.770045 30875 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.770757 30875 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:09.771535 22797 leader_election.cc:336] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.772029 22797 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
W20260709 13:10:09.773005 22797 consensus_peers.cc:597] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.773945 30873 consensus_queue.cc:237] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.774825 30889 raft_consensus.cc:2749] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:09.775929 30894 raft_consensus.cc:2749] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:09.779276 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:09.779315 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:09.800279 30867 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.793772 22651 catalog_manager.cc:5697] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 26 to 27, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 27 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:09.801209 30867 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.805645 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:09.777500 30875 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:09.811534 30893 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.808408 30889 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:09.817332 30900 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.807449 30890 raft_consensus.cc:515] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.817934 30900 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.819193 30890 leader_election.cc:290] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:09.819994 22797 leader_election.cc:336] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.820685 30900 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:09.820829 22871 leader_election.cc:336] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:09.823436 23419 leader_election.cc:336] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.825680 30867 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:09.826998 30867 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.827822 30867 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.830416 30867 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:09.830857 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:09.831188 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:09.842923 22794 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:09.844467 30894 raft_consensus.cc:2749] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:09.851751 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:09.852402 22899 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
W20260709 13:10:09.831799 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.832381 22654 catalog_manager.cc:5697] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 21 to 22, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 22 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:09.855886 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:09.856346 23419 leader_election.cc:336] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.856693 23419 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.834224 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:09.857417 30900 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:09.835212 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147efadd4ea46a09bda395402a10299" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:09.858407 22830 raft_consensus.cc:3060] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:10:09.836431 22797 leader_election.cc:336] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.837862 30889 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.861681 30889 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.863909 22794 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509, c45e1805f049467697a7358fb566de2f
I20260709 13:10:09.864689 30894 raft_consensus.cc:2749] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:09.840327 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:09.865778 22824 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:10:09.866133 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
W20260709 13:10:09.866369 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.866758 22825 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:10:09.868515 23416 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:09.864008 30889 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:09.869302 30790 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:10:09.869742 30790 raft_consensus.cc:493] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.870112 30790 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:10:09.871418 22871 leader_election.cc:336] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.874928 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.866024 22830 raft_consensus.cc:2468] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 23.
I20260709 13:10:09.876374 30864 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.877751 30864 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:09.879920 30864 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.880481 30889 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:10:09.881102 23419 leader_election.cc:336] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:09.881660 23419 leader_election.cc:336] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.880848 30889 raft_consensus.cc:493] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.883013 30889 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:09.888154 30790 raft_consensus.cc:515] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.893584 30867 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.895813 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:09.896589 22899 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 16.
I20260709 13:10:09.897989 23416 leader_election.cc:304] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:09.894176 30867 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.902312 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:10:09.914314 30900 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.915041 30900 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:09.917443 22797 leader_election.cc:336] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.911778 30890 raft_consensus.cc:2804] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:09.923026 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:09.923523 22830 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:09.926214 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:09.926935 22825 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
W20260709 13:10:09.929237 21629 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22777)
I20260709 13:10:09.934610 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:09.925552 30894 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:09.928359 30890 raft_consensus.cc:697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:10:09.941072 23419 leader_election.cc:336] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:09.946079 23419 leader_election.cc:336] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.938396 30873 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:09.950727 30873 raft_consensus.cc:493] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.951314 30873 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:09.965636 30907 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.942040 30894 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:09.945182 30790 leader_election.cc:290] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:09.977710 30890 consensus_queue.cc:237] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:09.984418 30867 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:09.986424 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4e4b3212e80646379ccf77b5952a4363: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:09.989784 22830 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:10:09.977134 30894 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:09.975950 30875 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
W20260709 13:10:09.994911 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60412us
I20260709 13:10:09.996366 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a127a20cfa4496bad51cb634bd0d0f" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:09.996954 22899 raft_consensus.cc:3060] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:09.998885 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:10.005172 22899 raft_consensus.cc:2468] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 24.
I20260709 13:10:10.020665 30875 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.023027 30875 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:10.025094 30790 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.028237 30790 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:10.027669 22871 leader_election.cc:336] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.029773 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:10.030423 23444 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
I20260709 13:10:10.026041 30889 raft_consensus.cc:515] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:10.032734 30889 leader_election.cc:290] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:10.033689 30900 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:09.981213 30907 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.036875 30907 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:10.037899 23446 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 23.
I20260709 13:10:10.043988 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:10:10.046545 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:09.982167 30873 raft_consensus.cc:515] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.050139 30894 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:10.053858 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:10.054324 23419 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:10.054554 22797 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.057287 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:10:10.058952 22797 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:10.058204 30905 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:10:10.062009 30905 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.062392 30905 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:10.063446 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1914222f984a8f9db2e3956a97bad0" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:10.064072 22899 raft_consensus.cc:3060] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:10.064704 30790 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:10.072783 30873 leader_election.cc:290] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:10.073026 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847c8a2cc0bb4233a1d55ca04643418d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:10.073592 30873 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.073698 22825 raft_consensus.cc:3060] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:10:10.068519 22871 leader_election.cc:336] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.075582 30873 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.077540 30873 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:10.059293 30889 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:10:10.078243 30889 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.078751 30889 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:10.080526 30909 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.081144 30909 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.081914 30867 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.082515 30867 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.083856 30909 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:10.083992 22825 raft_consensus.cc:2468] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:10:10.060472 30894 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:10.085500 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:10.086053 22901 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:10:10.088474 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:10.088827 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:10.089103 23446 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 19.
I20260709 13:10:10.089424 23444 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:10:10.095304 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:10.096089 30909 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:10.096597 30909 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.084564 30867 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:10.090673 22649 catalog_manager.cc:5697] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 22 to 23, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 23 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:10.097060 30909 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:10.097635 22651 catalog_manager.cc:5697] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 21 to 22, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 22 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:10.103222 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:10.104219 30889 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:10.105235 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 48801.1us
I20260709 13:10:10.106546 22899 raft_consensus.cc:2468] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 17.
I20260709 13:10:10.107867 22868 leader_election.cc:304] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:10.109388 30875 raft_consensus.cc:2804] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:10.109867 30913 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:10.110102 30875 raft_consensus.cc:697] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:10.110323 30913 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.110842 30913 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:10.111092 30875 consensus_queue.cc:237] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:10.112241 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.113571 22794 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3; no voters: 
I20260709 13:10:10.098059 22649 catalog_manager.cc:5697] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 22 to 23, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 23 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:10.114796 22797 leader_election.cc:336] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.115274 30867 raft_consensus.cc:2804] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:10:10.120527 23419 leader_election.cc:336] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:10.121132 23419 leader_election.cc:336] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.123521 30867 raft_consensus.cc:697] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:10.124475 30867 consensus_queue.cc:237] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:10.125576 23416 leader_election.cc:304] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:10.130122 30873 raft_consensus.cc:2804] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:10:10.130673 30873 raft_consensus.cc:697] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 24 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:10.131711 30873 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.132272 30909 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.106559 30905 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.135507 30912 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.136639 30912 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.138762 22825 raft_consensus.cc:1275] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:10:10.138784 30912 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:10.142545 23446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:10.143107 23446 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:10.145609 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:10.146135 23444 raft_consensus.cc:3060] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:10.146728 22899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:10.147277 22899 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:10:10.150595 22871 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.135267 30909 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:10.144094 22797 leader_election.cc:336] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:10.152731 22797 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.154727 30889 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:10.157272 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:10.157853 22899 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
W20260709 13:10:10.143623 23419 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.158107 23444 raft_consensus.cc:2468] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 24.
I20260709 13:10:10.163040 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "264d1889b1f144c5897b33c834b1beef" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:10.163599 22824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:10.164049 22825 raft_consensus.cc:2393] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 24: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:10:10.164163 22824 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:10:10.164366 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0563f7efc8452c86a8c67c0189fe81" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:10.136157 30905 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:10.164974 22828 raft_consensus.cc:3060] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:10.167361 30865 consensus_queue.cc:1048] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:10.173282 23446 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 20.
I20260709 13:10:10.176923 22899 raft_consensus.cc:1275] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 26 index: 23. Preceding OpId from leader: term: 27 index: 24. (index mismatch)
I20260709 13:10:10.179292 22899 raft_consensus.cc:1275] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
W20260709 13:10:10.182401 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:10.183394 23419 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.184839 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9578eb52c342078c7555d5100bdcad" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:10.185354 23444 raft_consensus.cc:3060] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:10.190547 22828 raft_consensus.cc:2468] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
I20260709 13:10:10.197218 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:10.207219 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:10.120642 30913 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.208588 23444 raft_consensus.cc:2468] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:10:10.216173 30790 consensus_queue.cc:1048] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 13:10:10.217943 22794 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:10.218454 22828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:10.219072 30911 raft_consensus.cc:2804] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:10:10.219385 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3" is_pre_election: true
I20260709 13:10:10.219559 30911 raft_consensus.cc:697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:10.219986 22830 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
W20260709 13:10:10.209331 22797 leader_election.cc:336] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:10.223239 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60848.8us
W20260709 13:10:10.223556 22797 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.218413 30910 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:10.236038 30911 consensus_queue.cc:237] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:10.239073 23419 leader_election.cc:336] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:10.239640 23419 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.240957 22828 raft_consensus.cc:2393] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c45e1805f049467697a7358fb566de2f in current term 21: Already voted for candidate 7aad11e70ca94fce8f672db05f1c28c3 in this term.
I20260709 13:10:10.236145 22651 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 23 to 24, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 24 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:10.221179 30912 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:10.246429 22794 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:10.247330 30915 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:10.247824 30915 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
I20260709 13:10:10.247968 22654 catalog_manager.cc:5697] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 20 to 21, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 21 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:10.240360 30910 raft_consensus.cc:493] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.250445 30910 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:10.251780 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:10.249249 30915 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.256796 22649 catalog_manager.cc:5697] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 16 to 17, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:10.258724 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:10.260673 22871 leader_election.cc:336] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:10.276293 22797 consensus_peers.cc:597] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.290016 30913 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:10.292332 23444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad75d8751c54bf78bcf4fa8cc6578c1" candidate_uuid: "7aad11e70ca94fce8f672db05f1c28c3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:10.292899 23444 raft_consensus.cc:3060] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:10:10.304324 22797 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.306820 23444 raft_consensus.cc:2468] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad11e70ca94fce8f672db05f1c28c3 in term 21.
I20260709 13:10:10.308403 22794 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:10.309871 30913 raft_consensus.cc:2804] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:10.312992 30913 raft_consensus.cc:697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Becoming Leader. State: Replica: 7aad11e70ca94fce8f672db05f1c28c3, State: Running, Role: LEADER
W20260709 13:10:10.318564 22797 consensus_peers.cc:597] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:10.315882 30913 consensus_queue.cc:237] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:10.325330 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 113932us
I20260709 13:10:10.326602 30912 consensus_queue.cc:1048] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:10:10.330924 23416 leader_election.cc:304] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:10.332383 30912 raft_consensus.cc:2804] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:10.332731 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:10.332868 30912 raft_consensus.cc:697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:10.333885 30912 consensus_queue.cc:237] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.335107 22868 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:10.336292 30889 raft_consensus.cc:2804] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:10:10.337496 30889 raft_consensus.cc:697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:10.337623 22899 raft_consensus.cc:1275] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:10:10.338469 30889 consensus_queue.cc:237] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.339546 30867 consensus_queue.cc:1048] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:10.341854 22830 raft_consensus.cc:1275] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:10:10.350066 30910 raft_consensus.cc:515] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.366271 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:10.369203 30893 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:10.369647 30893 raft_consensus.cc:493] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6a5191aaff76469bbf6c4735dc9e6509)
I20260709 13:10:10.370018 30893 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:10.408365 30922 consensus_queue.cc:1048] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:10:10.377355 30907 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:10.411063 30928 log.cc:927] Time spent T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:10:10.413456 22797 consensus_peers.cc:597] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:10.426956 30910 leader_election.cc:290] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:10.447201 23419 leader_election.cc:336] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.463878 22825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82d0d73de20d49d3a64fef8b052288e5" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:10.464450 22825 raft_consensus.cc:3060] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:10.458173 22651 catalog_manager.cc:5697] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 20 to 21, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 21 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:10.430924 22649 catalog_manager.cc:5697] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 23 to 24, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 24 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:10.471242 22649 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 19 to 20, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 20 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:10.483609 22825 raft_consensus.cc:2468] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:10:10.484072 30893 raft_consensus.cc:515] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.488967 23416 leader_election.cc:304] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
W20260709 13:10:10.510241 22797 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:10.534727 30908 raft_consensus.cc:2804] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:10:10.544618 22638 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 18a127a20cfa4496bad51cb634bd0d0f with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER failed with leader c45e1805f049467697a7358fb566de2f (127.21.31.68:42017) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:10:10.544080 22654 catalog_manager.cc:5697] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 23 to 24, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 24 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:10.560060 23419 leader_election.cc:336] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:10.559188 30908 raft_consensus.cc:697] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:10.569231 30893 leader_election.cc:290] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Requested vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:10.581784 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:10.584733 22797 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:10.588071 22797 consensus_peers.cc:597] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.590741 22896 raft_consensus.cc:1275] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:10:10.576121 30908 consensus_queue.cc:237] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.610100 30913 consensus_queue.cc:1048] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:10:10.628293 22797 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:10.633200 22830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d4d50470a044f698ff4852a4b634ca" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7aad11e70ca94fce8f672db05f1c28c3"
I20260709 13:10:10.633980 22830 raft_consensus.cc:3060] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:10.640242 23419 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:10.640743 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.645717 22830 raft_consensus.cc:2468] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 22.
I20260709 13:10:10.647336 22896 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:10:10.648986 22828 raft_consensus.cc:1275] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:10:10.661911 30907 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:10:10.668912 22828 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:10:10.672333 23416 leader_election.cc:304] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:10.673133 30935 raft_consensus.cc:2804] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:10.673780 30935 raft_consensus.cc:697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:10.674896 30935 consensus_queue.cc:237] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:10.677625 30900 consensus_queue.cc:1048] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:10:10.699699 22797 consensus_peers.cc:597] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.679447 22654 catalog_manager.cc:5697] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: term changed from 20 to 21, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66). New cstate: current_term: 21 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:10.713608 23453 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:10:10.713737 30941 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:10:10.722596 30905 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:10:10.751289 22830 raft_consensus.cc:1275] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 13:10:10.752803 22871 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.757498 23444 raft_consensus.cc:1275] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:10:10.757853 22830 raft_consensus.cc:1275] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:10:10.765841 30900 consensus_queue.cc:1048] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:10:10.776562 30923 consensus_queue.cc:1048] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:10:10.761397 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:10.786852 23446 raft_consensus.cc:1275] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:10:10.788374 22902 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:10:10.789253 30904 consensus_queue.cc:1048] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:10:10.798370 23446 raft_consensus.cc:1275] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:10:10.798223 22654 catalog_manager.cc:5697] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 21 to 22, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 22 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:10.800513 22654 catalog_manager.cc:5697] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 21 to 22, leader changed from 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 22 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:10.802706 30867 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:10:10.809051 22797 consensus_peers.cc:597] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:10.813448 22797 consensus_peers.cc:597] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:10.816038 30867 consensus_queue.cc:1048] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:10:10.833755 30925 consensus_queue.cc:1048] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.010s
W20260709 13:10:10.868860 22797 consensus_peers.cc:597] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:10.920459 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 77238.8us
W20260709 13:10:10.921810 23419 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:10.930171 21629 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:10.946421 30951 log.cc:927] Time spent T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:10:10.946976 22702 auto_rebalancer.cc:629] Failed to schedule move for tablet 847c8a2cc0bb4233a1d55ca04643418d: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:10:10.960839 22871 consensus_peers.cc:597] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:11.007851 30949 log.cc:927] Time spent T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 13:10:11.055989 23419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111) [suppressed 370 similar messages]
I20260709 13:10:11.095022 22830 raft_consensus.cc:1275] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:10:11.107002 22797 consensus_peers.cc:597] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:11.108075 30890 consensus_queue.cc:1048] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:10:11.136945 22828 raft_consensus.cc:1275] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:10:11.138631 30900 consensus_queue.cc:1048] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:10:11.178220 23419 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:11.198166 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4e4b3212e80646379ccf77b5952a4363: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:11.222160 23419 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:11.747004 22797 consensus_peers.cc:597] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:11.858373 22797 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:11.878737 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:11.938787 21629 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:12.035497 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.130918 22797 consensus_peers.cc:597] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.131635 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:10:12.171485 22830 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
W20260709 13:10:12.195962 22797 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.219442 22898 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:10:12.238690 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.245385 22797 consensus_peers.cc:597] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.246763 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.259469 22794 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer c45e1805f049467697a7358fb566de2f (127.21.31.68:42017): Couldn't send request to peer c45e1805f049467697a7358fb566de2f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1879799f5d0c462c9d2c95bd8a88862e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.272508 30967 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:12.331504 30956 raft_consensus.cc:2955] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:10:12.346632 22899 raft_consensus.cc:2955] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
I20260709 13:10:12.365603 22649 catalog_manager.cc:5697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index -1 to 11, NON_VOTER c45e1805f049467697a7358fb566de2f (127.21.31.68) added. New cstate: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } } }
W20260709 13:10:12.366425 22797 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.383287 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.473124 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.489032 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.510118 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.582027 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.604290 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.611816 22797 consensus_peers.cc:597] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:10:12.626773 23453 consensus_queue.cc:237] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:12.633749 22797 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:10:12.689572 22828 raft_consensus.cc:1275] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:10:12.697769 30966 consensus_queue.cc:1048] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:10:12.737591 23419 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:12.749043 30978 ts_tablet_manager.cc:933] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: Initiating tablet copy from peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:12.788753 23419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 51196.3us
W20260709 13:10:12.786079 22797 consensus_peers.cc:597] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:12.794345 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.828660 30978 tablet_copy_client.cc:323] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.31.66:39003
W20260709 13:10:12.855677 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.856596 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:42017, user_credentials={real_user=slave}} blocked reactor thread for 56133.6us
W20260709 13:10:12.900300 22797 consensus_peers.cc:597] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.903440 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:12.904028 23419 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:12.907639 30790 raft_consensus.cc:2955] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 24 LEADER]: Committing config change with OpId 24.18: config changed from index 15 to 18, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:12.934999 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 18a127a20cfa4496bad51cb634bd0d0f with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 13:10:12.935402 22840 tablet_copy_service.cc:140] P 7aad11e70ca94fce8f672db05f1c28c3: Received BeginTabletCopySession request for tablet 1879799f5d0c462c9d2c95bd8a88862e from peer c45e1805f049467697a7358fb566de2f ({username='slave'} at 127.0.0.1:47110)
I20260709 13:10:12.936079 22840 tablet_copy_service.cc:161] P 7aad11e70ca94fce8f672db05f1c28c3: Beginning new tablet copy session on tablet 1879799f5d0c462c9d2c95bd8a88862e from peer c45e1805f049467697a7358fb566de2f at {username='slave'} at 127.0.0.1:47110: session id = c45e1805f049467697a7358fb566de2f-1879799f5d0c462c9d2c95bd8a88862e
W20260709 13:10:12.939262 21629 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 22777)
I20260709 13:10:12.945379 22830 raft_consensus.cc:2955] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Committing config change with OpId 24.18: config changed from index 15 to 18, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:10:12.955962 23419 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:12.971138 22654 catalog_manager.cc:5697] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 reported cstate change: config changed from index 15 to 18, VOTER 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67) evicted. New cstate: current_term: 24 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:12.974351 22840 tablet_copy_source_session.cc:215] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:12.984948 22892 tablet_service.cc:1558] Processing DeleteTablet for tablet 18a127a20cfa4496bad51cb634bd0d0f with delete_type TABLET_DATA_TOMBSTONED (TS 3b626c41600d4737a178ebab8a7ebfc2 not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:46616
I20260709 13:10:12.998809 30986 tablet_replica.cc:333] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:13.000041 30986 raft_consensus.cc:2243] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:13.000985 30986 raft_consensus.cc:2272] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:13.027890 30986 ts_tablet_manager.cc:1916] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:13.034112 30978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1879799f5d0c462c9d2c95bd8a88862e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:13.082548 30986 ts_tablet_manager.cc:1929] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 24.17
I20260709 13:10:13.083169 30986 log.cc:1199] T 18a127a20cfa4496bad51cb634bd0d0f P 3b626c41600d4737a178ebab8a7ebfc2: Deleting WAL directory at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/wals/18a127a20cfa4496bad51cb634bd0d0f
I20260709 13:10:13.085261 22638 catalog_manager.cc:5028] TS 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): tablet 18a127a20cfa4496bad51cb634bd0d0f (table test-workload [id=f093a267312f4a1898f3cb161cef46b7]) successfully deleted
I20260709 13:10:13.087626 30978 tablet_copy_client.cc:806] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:13.088284 30978 tablet_copy_client.cc:670] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: tablet copy: Starting download of 1 WAL segments...
W20260709 13:10:13.182626 22797 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:10:13.190008 30978 tablet_copy_client.cc:538] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:10:13.191658 22797 consensus_peers.cc:597] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:13.208876 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:13.226184 30978 tablet_bootstrap.cc:492] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: Bootstrap starting.
W20260709 13:10:13.247977 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:13.252946 22797 consensus_peers.cc:597] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:13.253453 22797 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:13.274722 22797 consensus_peers.cc:597] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:13.320752 22797 consensus_peers.cc:597] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:13.351611 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4e4b3212e80646379ccf77b5952a4363: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:13.397353 23419 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:13.436867 22797 consensus_peers.cc:597] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:10:13.456908 30978 tablet_bootstrap.cc:492] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=2 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:13.458086 30978 tablet_bootstrap.cc:492] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: Bootstrap complete.
I20260709 13:10:13.458935 30978 ts_tablet_manager.cc:1403] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: Time spent bootstrapping tablet: real 0.233s	user 0.028s	sys 0.024s
I20260709 13:10:13.471143 30978 raft_consensus.cc:359] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:13.475581 30978 raft_consensus.cc:740] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: c45e1805f049467697a7358fb566de2f, State: Initialized, Role: LEARNER
W20260709 13:10:13.480588 22797 consensus_peers.cc:597] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:13.480759 30978 consensus_queue.cc:260] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: NON_VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: true } }
I20260709 13:10:13.524451 30978 ts_tablet_manager.cc:1434] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.000s
I20260709 13:10:13.574052 22840 tablet_copy_service.cc:342] P 7aad11e70ca94fce8f672db05f1c28c3: Request end of tablet copy session c45e1805f049467697a7358fb566de2f-1879799f5d0c462c9d2c95bd8a88862e received from {username='slave'} at 127.0.0.1:47110
I20260709 13:10:13.574522 22840 tablet_copy_service.cc:434] P 7aad11e70ca94fce8f672db05f1c28c3: ending tablet copy session c45e1805f049467697a7358fb566de2f-1879799f5d0c462c9d2c95bd8a88862e on tablet 1879799f5d0c462c9d2c95bd8a88862e with peer c45e1805f049467697a7358fb566de2f
W20260709 13:10:13.577342 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:13.578971 22871 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:13.579515 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:10:13.674278 22825 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:13.728017 23446 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:10:13.728076 22899 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:10:13.729602 30980 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:10:13.730399 30991 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:10:13.748204 23419 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:13.767269 30991 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Committing config change with OpId 20.17: config changed from index 11 to 17, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:13.791944 22638 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 135e2f7520a34b53aaf4ddb360a8a029 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:10:13.793700 23446 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Committing config change with OpId 20.17: config changed from index 11 to 17, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:13.812491 22654 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f reported cstate change: config changed from index 11 to 17, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New cstate: current_term: 20 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:13.822149 22899 raft_consensus.cc:2955] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Committing config change with OpId 20.17: config changed from index 11 to 17, VOTER 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:13.824043 23453 raft_consensus.cc:1217] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 LEARNER]: Deduplicated request from leader. Original: 17.10->[17.11-17.11]   Dedup: 17.11->[]
W20260709 13:10:13.860674 23419 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:13.939741 21629 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:14.051400 22871 consensus_peers.cc:597] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:14.365973 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 135e2f7520a34b53aaf4ddb360a8a029: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:14.406673 30991 raft_consensus.cc:1064] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: attempting to promote NON_VOTER c45e1805f049467697a7358fb566de2f to VOTER
I20260709 13:10:14.434440 30991 consensus_queue.cc:237] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:14.467832 23453 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 LEARNER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:14.469588 22898 raft_consensus.cc:1275] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Refusing update from remote peer 7aad11e70ca94fce8f672db05f1c28c3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:14.469597 30981 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:10:14.477682 22797 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:14.478199 30991 consensus_queue.cc:1048] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:14.508867 30981 raft_consensus.cc:2955] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 11 to 12, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
I20260709 13:10:14.513630 22898 raft_consensus.cc:2955] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:10:14.520844 22797 consensus_peers.cc:597] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:14.539938 22871 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:14.531953 22649 catalog_manager.cc:5697] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: config changed from index 11 to 12, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "7aad11e70ca94fce8f672db05f1c28c3" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:10:14.543474 22797 consensus_peers.cc:597] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:10:14.552413 23453 raft_consensus.cc:2955] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, c45e1805f049467697a7358fb566de2f (127.21.31.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } } }
W20260709 13:10:14.579483 22797 consensus_peers.cc:597] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:14.620734 22797 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:14.627099 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:14.638875 22797 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:14.770227 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:14.787892 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:14.936869 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:14.940348 21629 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:14.964032 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.000536 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.021648 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.029626 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.048171 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.053333 22797 consensus_peers.cc:597] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.127210 23419 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:15.157884 23419 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:15.181535 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:15.200599 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.242830 22797 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.283082 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.371462 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.532819 23419 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:15.571334 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:15.573225 22797 consensus_peers.cc:597] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:15.580096 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.626183 22797 consensus_peers.cc:597] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.660775 22797 consensus_peers.cc:597] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:15.698769 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:15.715826 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.749095 22797 consensus_peers.cc:597] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:15.940788 21629 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:15.974471 22797 consensus_peers.cc:597] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.975297 22797 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:15.994436 22797 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:16.005339 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 137724us
W20260709 13:10:16.007650 22797 consensus_peers.cc:597] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:16.037493 22871 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:16.038071 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:16.065544 22871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111) [suppressed 438 similar messages]
W20260709 13:10:16.101696 23419 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:16.191953 22797 consensus_peers.cc:597] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:16.239283 23419 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:16.353868 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 135e2f7520a34b53aaf4ddb360a8a029: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:16.446197 23419 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:16.661440 22794 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 45467.2us
W20260709 13:10:16.790500 22871 consensus_peers.cc:597] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:16.941244 21629 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:16.949842 22797 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:16.952057 22797 consensus_peers.cc:597] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:16.972959 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:16.983147 22871 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:17.222122 22797 consensus_peers.cc:597] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.240192 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.260215 22797 consensus_peers.cc:597] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:17.343003 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.363409 22797 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.386744 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.407714 22797 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:17.457748 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.465013 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.497646 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4e4b3212e80646379ccf77b5952a4363: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:17.501236 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.561998 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.604663 22797 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.693105 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.713255 22797 consensus_peers.cc:597] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.758287 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.758762 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:17.774755 23419 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:17.865459 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:17.896929 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:17.910480 23419 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:17.942278 21629 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:18.027410 22797 consensus_peers.cc:597] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:18.068859 23419 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:18.077821 22797 consensus_peers.cc:597] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:18.087471 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:18.124217 22797 consensus_peers.cc:597] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:18.168776 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:18.175076 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:18.185711 22797 consensus_peers.cc:597] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:18.189359 22797 consensus_peers.cc:597] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:18.196209 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:18.246948 22797 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:18.299292 22797 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:18.469301 22797 consensus_peers.cc:597] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:18.514227 22871 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:18.519490 23419 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:18.608798 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:18.714293 22797 consensus_peers.cc:597] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:18.734666 23419 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:18.942757 21629 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:19.018266 23419 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:19.186383 22797 consensus_peers.cc:597] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:19.257560 22871 consensus_peers.cc:597] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:19.378046 22797 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:19.384855 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 53490.6us
W20260709 13:10:19.506664 22871 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:19.577172 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:19.666277 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68078.7us
W20260709 13:10:19.677259 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:19.734519 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:19.883234 22797 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:19.887185 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:19.889986 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:19.894534 22797 consensus_peers.cc:597] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:19.929715 22797 consensus_peers.cc:597] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:19.943361 21629 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:19.946632 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.033931 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.039498 22797 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.175639 22797 consensus_peers.cc:597] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.177309 22797 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.176159 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.249055 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.331030 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.342833 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.347986 23419 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:20.441018 22797 consensus_peers.cc:597] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.472913 23419 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:20.534672 22797 consensus_peers.cc:597] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:20.540563 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:20.612502 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.640745 23419 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:20.668000 22797 consensus_peers.cc:597] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:20.679849 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.706593 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:20.698136 22797 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.739364 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.742125 22797 consensus_peers.cc:597] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.759667 22871 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:20.829684 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:20.833873 22797 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:20.879940 23419 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:20.943846 21629 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:21.033648 22797 consensus_peers.cc:597] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:21.067936 22871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111) [suppressed 447 similar messages]
W20260709 13:10:21.218681 22797 consensus_peers.cc:597] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:21.246308 23419 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:21.277452 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:21.354621 22797 consensus_peers.cc:597] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:21.428293 23419 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:21.867376 22871 consensus_peers.cc:597] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:21.915019 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:21.944334 21629 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:22.013545 22797 consensus_peers.cc:597] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.026453 22797 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:10:22.036751 22871 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:22.142603 22797 consensus_peers.cc:597] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.151857 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.184505 22797 consensus_peers.cc:597] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:22.230026 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.233592 22797 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:22.312237 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.392835 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.440280 22797 consensus_peers.cc:597] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.528407 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.605185 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.630565 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.647562 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 135e2f7520a34b53aaf4ddb360a8a029: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:22.700196 23419 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:22.717154 22797 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.734124 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.780050 22797 consensus_peers.cc:597] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.885792 23419 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:22.922704 22797 consensus_peers.cc:597] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:22.933936 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.941488 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:22.944860 21629 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:22.979346 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:23.003979 22797 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:23.095197 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:23.126003 22797 consensus_peers.cc:597] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:23.150238 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:23.177824 22797 consensus_peers.cc:597] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:23.190855 23419 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:23.197117 23419 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:23.218994 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:23.219986 22797 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:23.252039 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:23.281222 22871 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:23.286089 22797 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:23.360692 22797 consensus_peers.cc:597] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:23.388844 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:23.626994 23419 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:23.687688 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:23.726838 22797 consensus_peers.cc:597] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:23.768637 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:35797, user_credentials={real_user=slave}} blocked reactor thread for 98681.9us
W20260709 13:10:23.814782 23419 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:23.846644 22797 consensus_peers.cc:597] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:23.945969 21629 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 22777)
W20260709 13:10:24.230136 22797 consensus_peers.cc:597] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:24.328549 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:24.390019 22871 consensus_peers.cc:597] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:24.527436 22871 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:24.529906 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:24.537369 22797 consensus_peers.cc:597] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:10:24.551481 22797 consensus_peers.cc:597] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:24.685256 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:24.752588 22797 consensus_peers.cc:597] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:24.816183 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:24.930480 22797 consensus_peers.cc:597] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:24.938822 22797 consensus_peers.cc:597] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:24.975080 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:24.984470 22797 consensus_peers.cc:597] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.042194 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.203974 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.229452 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.258112 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.281670 23419 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:25.345165 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.392858 22797 consensus_peers.cc:597] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:25.416009 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:10:25.429910 22797 consensus_peers.cc:597] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.572548 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.733850 23419 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:25.734522 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:25.751749 23419 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:25.753371 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.757050 23419 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:25.806627 22797 consensus_peers.cc:597] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:10:25.808343 21629 tablet_server.cc:196] TabletServer@127.21.31.65:0 shutdown complete.
W20260709 13:10:25.813637 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:25.817178 22797 consensus_peers.cc:597] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.817904 22797 consensus_peers.cc:597] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.833194 22797 consensus_peers.cc:597] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.834725 22797 consensus_peers.cc:597] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.852766 22797 consensus_peers.cc:597] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:25.873522 22641 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.31.65:37213, user_credentials={real_user=slave}} blocked reactor thread for 65600.4us
W20260709 13:10:25.874774 23416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51676.1us
W20260709 13:10:25.884413 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4e4b3212e80646379ccf77b5952a4363: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:25.910338 23419 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:25.951359 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:25.992776 22871 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:25.993518 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:26.079556 23419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111) [suppressed 439 similar messages]
I20260709 13:10:26.228833 21629 tablet_server.cc:179] TabletServer@127.21.31.66:0 shutting down...
W20260709 13:10:26.279342 22868 consensus_peers.cc:597] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.288034 22868 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.288781 22868 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.317714 23416 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.328580 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:26.354212 22868 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.381582 23416 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.394176 23416 consensus_peers.cc:597] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.405206 23416 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.414793 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:26.416096 21629 tablet_replica.cc:333] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.419414 21629 raft_consensus.cc:2243] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.420769 21629 raft_consensus.cc:2272] T efec6b734c374a169f3dc3b729ef2f07 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.430263 22868 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.432384 21629 tablet_replica.cc:333] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
W20260709 13:10:26.434118 23416 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.434383 21629 raft_consensus.cc:2243] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.436859 21629 raft_consensus.cc:2272] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.439193 23416 consensus_peers.cc:597] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.440447 21629 tablet_replica.cc:333] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.443529 21629 raft_consensus.cc:2243] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.445852 21629 raft_consensus.cc:2272] T d0d74135bf874a0bb4b857b686effcfd P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.450791 21629 tablet_replica.cc:333] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.451685 21629 raft_consensus.cc:2243] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.452618 21629 raft_consensus.cc:2272] T f8c8f692d4994a148c306609ab807ab7 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.455390 21629 tablet_replica.cc:333] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.456367 21629 raft_consensus.cc:2243] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.457857 21629 raft_consensus.cc:2272] T 1879799f5d0c462c9d2c95bd8a88862e P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.460649 21629 tablet_replica.cc:333] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.461622 21629 raft_consensus.cc:2243] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.462882 21629 raft_consensus.cc:2272] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.465674 21629 tablet_replica.cc:333] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.466542 21629 raft_consensus.cc:2243] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.467453 21629 raft_consensus.cc:2272] T 8da6707b1a3d47a48c3671a6b7401196 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.470400 21629 tablet_replica.cc:333] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.471273 21629 raft_consensus.cc:2243] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.473511 21629 raft_consensus.cc:2272] T 00324d171dc0468fa259f13c1d4d74ea P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.478067 21629 tablet_replica.cc:333] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.478878 21629 raft_consensus.cc:2243] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.480197 21629 raft_consensus.cc:2272] T 4bbeaee00aa942ccb232c812c9e14827 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.484130 21629 tablet_replica.cc:333] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.484973 21629 raft_consensus.cc:2243] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.485777 21629 raft_consensus.cc:2272] T 67449ba22ccb4938be8a5e28664cc566 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.492040 21629 tablet_replica.cc:333] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.492865 21629 raft_consensus.cc:2243] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.493526 21629 raft_consensus.cc:2272] T 60a957de2a744260812f8da6f1bd9b87 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.497030 21629 tablet_replica.cc:333] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.498150 21629 raft_consensus.cc:2243] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.499514 21629 raft_consensus.cc:2272] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 7aad11e70ca94fce8f672db05f1c28c3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.503227 21629 tablet_replica.cc:333] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.504192 21629 raft_consensus.cc:2243] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.505338 21629 raft_consensus.cc:2272] T dbd3d6688dfb4621b8ecfd11278d5b26 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.519310 22868 consensus_peers.cc:597] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.536289 23416 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.538648 21629 tablet_replica.cc:333] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.539683 21629 raft_consensus.cc:2243] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:10:26.546967 22868 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.542253 21629 raft_consensus.cc:2272] T 90274c6677ab4d37b51190e940c9608e P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.551813 21629 tablet_replica.cc:333] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.558043 21629 raft_consensus.cc:2243] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.558830 21629 raft_consensus.cc:2272] T 7f765d75643f47dd939002936724a315 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.561682 21629 tablet_replica.cc:333] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.564320 21629 raft_consensus.cc:2243] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.565174 21629 raft_consensus.cc:2272] T 2147efadd4ea46a09bda395402a10299 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.568236 21629 tablet_replica.cc:333] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.569191 21629 raft_consensus.cc:2243] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.569933 21629 raft_consensus.cc:2272] T 6ce4678cb66d4d1e8325b447eaf2a630 P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.572880 21629 tablet_replica.cc:333] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.573763 21629 raft_consensus.cc:2243] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.574980 21629 raft_consensus.cc:2272] T 469a8bc68a8b4e62b19f382c167cf50c P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.579561 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080017b4e0 after lost signal to thread 21632
W20260709 13:10:26.580993 22685 debug-util.cc:398] Leaking SignalData structure 0x7b08001e34e0 after lost signal to thread 22688
W20260709 13:10:26.582535 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080003b7a0 after lost signal to thread 22855
I20260709 13:10:26.582671 21629 tablet_replica.cc:333] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
W20260709 13:10:26.584376 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080024fde0 after lost signal to thread 22929
I20260709 13:10:26.583690 21629 raft_consensus.cc:2243] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.587633 21629 raft_consensus.cc:2272] T 135e2f7520a34b53aaf4ddb360a8a029 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.593524 21629 tablet_replica.cc:333] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.594419 21629 raft_consensus.cc:2243] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:10:26.594609 23416 consensus_peers.cc:597] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.595570 21629 raft_consensus.cc:2272] T db154be77d6e4b5388f9a3622f91a4ba P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.602933 23416 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.603638 21629 tablet_replica.cc:333] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.604571 21629 raft_consensus.cc:2243] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:10:26.604730 22685 debug-util.cc:398] Leaking SignalData structure 0x7b080040e200 after lost signal to thread 23478
I20260709 13:10:26.606164 21629 raft_consensus.cc:2272] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.606225 23416 consensus_peers.cc:597] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.609827 21629 tablet_replica.cc:333] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.610729 21629 raft_consensus.cc:2243] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.611937 21629 raft_consensus.cc:2272] T ab1914222f984a8f9db2e3956a97bad0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.615209 21629 tablet_replica.cc:333] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.616218 21629 raft_consensus.cc:2243] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.616988 21629 raft_consensus.cc:2272] T 18a127a20cfa4496bad51cb634bd0d0f P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.619704 21629 tablet_replica.cc:333] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.621227 21629 raft_consensus.cc:2243] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.622177 21629 raft_consensus.cc:2272] T 84ea43361e534fbebad704327c9f6367 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.627897 21629 tablet_replica.cc:333] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.628758 21629 raft_consensus.cc:2243] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.630347 21629 raft_consensus.cc:2272] T 6ce771dbb93248d397c8d5b222b27fca P 7aad11e70ca94fce8f672db05f1c28c3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.632982 21629 tablet_replica.cc:333] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.633838 21629 raft_consensus.cc:2243] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.634790 21629 raft_consensus.cc:2272] T ccab55924ea9473faf1d4d97313e9284 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.645241 21629 tablet_replica.cc:333] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.646064 21629 raft_consensus.cc:2243] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.647277 21629 raft_consensus.cc:2272] T 18051671d25f4aa7b30d4a299909f91a P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.680433 22868 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:26.681767 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.684859 21629 tablet_replica.cc:333] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
W20260709 13:10:26.686165 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.687304 21629 raft_consensus.cc:2243] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.687980 21629 raft_consensus.cc:2272] T e8ff32d0405c426786004110e1c1f955 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.690742 21629 tablet_replica.cc:333] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.692169 21629 raft_consensus.cc:2243] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:10:26.692859 22871 consensus_peers.cc:597] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:10:26.693100 21629 raft_consensus.cc:2272] T 27b79b7fd4bf4e90a1a7f357a253688d P 7aad11e70ca94fce8f672db05f1c28c3 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.695138 23416 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.695766 21629 tablet_replica.cc:333] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.696821 21629 raft_consensus.cc:2243] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.697996 21629 raft_consensus.cc:2272] T 4d8a8f3cf129492eb679da24703f5b84 P 7aad11e70ca94fce8f672db05f1c28c3 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.700945 21629 tablet_replica.cc:333] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.701793 21629 raft_consensus.cc:2243] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.702873 21629 raft_consensus.cc:2272] T 09a9ba6091094973a526986ef573e2a9 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.705432 21629 tablet_replica.cc:333] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.706218 21629 raft_consensus.cc:2243] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.707264 21629 raft_consensus.cc:2272] T 4e4b3212e80646379ccf77b5952a4363 P 7aad11e70ca94fce8f672db05f1c28c3 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.710973 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.713600 21629 tablet_replica.cc:333] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.714416 21629 raft_consensus.cc:2243] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.715267 21629 raft_consensus.cc:2272] T 82d0d73de20d49d3a64fef8b052288e5 P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.717926 21629 tablet_replica.cc:333] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.720017 21629 raft_consensus.cc:2243] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.720624 21629 raft_consensus.cc:2272] T 847c8a2cc0bb4233a1d55ca04643418d P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.723541 21629 tablet_replica.cc:333] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.724261 21629 raft_consensus.cc:2243] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.725342 21629 raft_consensus.cc:2272] T 317f6a16e0af41e2984a9b1626ab0edc P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.725567 23416 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:26.727950 21629 tablet_replica.cc:333] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.728730 21629 raft_consensus.cc:2243] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.729950 21629 raft_consensus.cc:2272] T f53f146bb25444ce8d2072559efe5e59 P 7aad11e70ca94fce8f672db05f1c28c3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.732455 21629 tablet_replica.cc:333] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.733275 21629 raft_consensus.cc:2243] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.734124 21629 raft_consensus.cc:2272] T 41fb40751eb74da580d83c9bb69ae58a P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.753825 22794 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.760301 21629 tablet_replica.cc:333] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.761514 21629 raft_consensus.cc:2243] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.762567 21629 raft_consensus.cc:2272] T 264d1889b1f144c5897b33c834b1beef P 7aad11e70ca94fce8f672db05f1c28c3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.766136 21629 tablet_replica.cc:333] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.770607 21629 raft_consensus.cc:2243] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.772486 21629 raft_consensus.cc:2272] T e4cc99e6a1f14071a587af061e82fee0 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.785074 21629 tablet_replica.cc:333] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.786090 21629 raft_consensus.cc:2243] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.787168 21629 raft_consensus.cc:2272] T 3833f41661774b918ddf80499fd85f2f P 7aad11e70ca94fce8f672db05f1c28c3 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.791486 21629 tablet_replica.cc:333] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.792941 21629 raft_consensus.cc:2243] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.794342 21629 raft_consensus.cc:2272] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.812558 21629 tablet_replica.cc:333] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
W20260709 13:10:26.814304 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.815605 21629 raft_consensus.cc:2243] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.818147 21629 raft_consensus.cc:2272] T c7d4d50470a044f698ff4852a4b634ca P 7aad11e70ca94fce8f672db05f1c28c3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.827387 21629 tablet_replica.cc:333] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.828303 21629 raft_consensus.cc:2243] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:10:26.829360 21629 raft_consensus.cc:2272] T 1f881fed251c4393ac053086bd572549 P 7aad11e70ca94fce8f672db05f1c28c3 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.831981 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.839015 21629 tablet_replica.cc:333] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
W20260709 13:10:26.854441 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.874864 21629 raft_consensus.cc:2243] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.880846 21629 raft_consensus.cc:2272] T 8f0563f7efc8452c86a8c67c0189fe81 P 7aad11e70ca94fce8f672db05f1c28c3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.884011 21629 tablet_replica.cc:333] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.884812 21629 raft_consensus.cc:2243] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.885510 21629 raft_consensus.cc:2272] T 11e8d51b5b18431cb7d6df2884a4e879 P 7aad11e70ca94fce8f672db05f1c28c3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:26.891824 21629 tablet_replica.cc:333] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3: stopping tablet replica
I20260709 13:10:26.892423 21629 raft_consensus.cc:2243] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:26.892936 21629 raft_consensus.cc:2272] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P 7aad11e70ca94fce8f672db05f1c28c3 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:26.898984 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:26.987754 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:27.046267 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:27.086297 22795 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:27.107604 22797 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:27.131624 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:27.142023 22796 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:27.201115 23419 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:27.273757 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:27.285293 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:27.325970 22871 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:10:27.575052 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:10:27.586685 31067 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.587443 31067 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.591136 31067 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.593660 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:27.601729 23416 leader_election.cc:336] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.602723 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.604833 31067 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:27.629520 31076 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.630112 31076 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.632733 31076 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:27.634995 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:27.639276 22868 leader_election.cc:336] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.639925 22868 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:27.641443 31076 raft_consensus.cc:2749] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:10:27.649915 31076 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.650573 31076 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:27.661279 22868 leader_election.cc:336] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:27.662019 22871 leader_election.cc:336] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.662477 22871 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.663206 31075 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:27.663422 31076 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.686254 31067 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.686916 31072 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.686797 31067 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.687486 31072 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.689534 31067 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.689608 31072 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.690178 31076 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.690696 31076 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:27.692569 31076 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.694773 31077 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.695263 31077 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.701167 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:27.702668 23416 leader_election.cc:336] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:27.706146 23419 leader_election.cc:336] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.706764 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:27.707026 22871 leader_election.cc:336] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:27.703150 23416 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:27.708840 22868 leader_election.cc:336] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.710440 31072 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.710793 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.710942 31072 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.711479 31076 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:27.712913 31072 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.697168 31077 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.717568 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:27.721052 23416 leader_election.cc:336] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.721726 31067 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:27.722086 23419 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.723716 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.723075 23416 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:27.725090 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:10:27.725540 31067 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:27.725564 31077 raft_consensus.cc:2749] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:27.727360 31076 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.727849 31076 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:27.729991 23416 leader_election.cc:336] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.730444 31076 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:27.730484 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.731479 31077 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:27.735844 22868 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:27.736316 22871 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.736701 22871 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.737397 31075 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:27.738875 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:27.741408 31077 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.741917 31077 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.745543 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:27.746117 22901 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 26.
I20260709 13:10:27.747063 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:27.747783 31067 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:10:27.748144 31067 raft_consensus.cc:493] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.748476 31067 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:10:27.753896 31077 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:27.758033 23416 leader_election.cc:336] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.759017 31067 raft_consensus.cc:515] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.762603 31067 leader_election.cc:290] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.766547 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cf0bfa7a80d4f0286947f1e6ff89772" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:27.769735 22901 raft_consensus.cc:3060] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:10:27.771377 31067 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.772324 31067 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.775454 31067 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.779161 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:27.779858 22896 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
W20260709 13:10:27.780797 23416 leader_election.cc:336] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.783102 31067 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.783710 31067 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.790475 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:27.791076 23416 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.786515 31067 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.794199 22901 raft_consensus.cc:2468] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 27.
W20260709 13:10:27.794648 23416 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.795403 23416 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.796883 31077 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:27.796949 23419 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.800339 23419 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:27.801350 23419 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.801812 31077 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:27.803678 23416 leader_election.cc:304] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.805187 31077 raft_consensus.cc:2804] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:10:27.805790 31077 raft_consensus.cc:697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 27 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:27.806759 31077 consensus_queue.cc:237] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.817242 31075 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.817785 31075 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.819835 31075 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:27.825341 22868 leader_election.cc:336] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.826900 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:27.827515 23453 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:10:27.828543 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.834199 31067 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.834702 31067 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.836722 31067 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.838299 31076 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.838811 31076 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:27.840832 31076 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.842622 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:27.843267 22901 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
W20260709 13:10:27.847815 22871 leader_election.cc:336] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.831063 31075 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:27.849727 31075 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.850760 31075 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:27.851090 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:27.852443 31076 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.858357 31076 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:27.860321 31076 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.860767 31076 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:10:27.850322 22868 leader_election.cc:336] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.862998 22868 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.857867 31083 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:10:27.864205 22871 leader_election.cc:336] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.864425 31083 raft_consensus.cc:493] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:10:27.857234 23416 leader_election.cc:336] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.864822 31083 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:27.869722 31072 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.870252 31072 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.872237 31072 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.878690 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:27.861189 31076 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:27.883596 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:10:27.884176 31076 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.885715 31083 raft_consensus.cc:515] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:27.887923 23416 leader_election.cc:336] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.888386 31072 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.889205 22654 catalog_manager.cc:5697] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 26 to 27, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 27 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:27.889691 31072 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.890604 31077 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.890537 31076 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:27.891335 31077 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.891912 31072 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:27.892299 22871 leader_election.cc:336] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:27.897985 22868 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47691.9us
I20260709 13:10:27.892675 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.899960 31076 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:10:27.900935 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:10:27.900411 31072 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:27.900556 31076 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:27.899997 31083 leader_election.cc:290] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.897100 31075 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.909045 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90274c6677ab4d37b51190e940c9608e" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:27.909615 22896 raft_consensus.cc:3060] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:27.910704 31075 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:27.903756 31076 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.907998 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:27.913040 31077 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:27.922608 22896 raft_consensus.cc:2468] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 21.
W20260709 13:10:27.926611 22868 leader_election.cc:336] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.929805 22868 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.930617 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
W20260709 13:10:27.932525 23416 leader_election.cc:336] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:27.933754 23416 leader_election.cc:336] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.934435 31076 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:27.935670 22868 leader_election.cc:336] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.936197 22868 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.938481 31076 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:27.931198 22871 leader_election.cc:336] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.940714 31076 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.941285 31076 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.944262 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db154be77d6e4b5388f9a3622f91a4ba" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:27.944360 31075 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.944908 23453 raft_consensus.cc:3060] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:27.945009 31075 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:27.948858 31076 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:27.948823 31075 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.949514 31072 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:10:27.950273 22868 leader_election.cc:336] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.956697 22868 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.956665 31072 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.959002 31072 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:27.961478 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:27.962070 23451 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:10:27.962057 31075 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:27.964395 23419 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:27.965377 22868 leader_election.cc:336] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.936930 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.955672 23453 raft_consensus.cc:2468] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
I20260709 13:10:27.973505 31072 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:27.974671 23416 leader_election.cc:304] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.978327 31072 raft_consensus.cc:2804] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:10:27.976930 22868 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.979027 31072 raft_consensus.cc:697] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
W20260709 13:10:27.979231 22868 leader_election.cc:336] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:27.979950 31072 consensus_queue.cc:237] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:27.980544 22868 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.983460 31076 raft_consensus.cc:2804] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:27.984051 31076 raft_consensus.cc:697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:27.984987 31076 consensus_queue.cc:237] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:27.989229 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:27.995321 22871 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:27.995838 22871 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:27.994136 31075 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:27.999102 31075 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.001052 31075 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:27.991448 23416 leader_election.cc:336] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.002727 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.003150 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:28.003728 31072 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:10:28.004396 23453 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 26.
W20260709 13:10:28.008206 22868 leader_election.cc:336] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.009006 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.010397 31072 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.010413 31085 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:10:28.011137 31085 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.011530 31085 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:10:28.015120 31075 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:28.019784 31085 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.011132 31072 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.030776 31072 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:28.033831 31085 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:28.039117 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "469a8bc68a8b4e62b19f382c167cf50c" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:28.039708 23453 raft_consensus.cc:3060] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:10:28.043089 22868 leader_election.cc:336] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:28.043820 23419 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:28.048170 23453 raft_consensus.cc:2468] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 27.
I20260709 13:10:28.051798 22868 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.052811 31085 raft_consensus.cc:2804] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:10:28.053264 31085 raft_consensus.cc:697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:28.054133 31085 consensus_queue.cc:237] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.054584 22649 catalog_manager.cc:5697] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 20 to 21, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 21 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:28.023922 23416 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.059918 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.060693 31072 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:28.021723 22871 leader_election.cc:336] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:28.061400 22871 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.064762 31076 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:28.066665 22654 catalog_manager.cc:5697] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 20 to 21, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 21 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:28.069367 23416 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.069976 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.084678 31072 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:10:28.087009 31076 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.087507 31076 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:10:28.089426 31076 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
I20260709 13:10:28.090341 31076 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
W20260709 13:10:28.095093 22868 leader_election.cc:336] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:28.096309 22871 leader_election.cc:336] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:28.096740 22871 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.097858 31075 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:28.092264 31076 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.105005 31076 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:28.105450 31075 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.109826 31075 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.111740 31075 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:28.106151 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:28.108947 22868 leader_election.cc:336] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.112481 23453 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 20.
I20260709 13:10:28.112190 22654 catalog_manager.cc:5697] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 26 to 27, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 27 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.114462 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.115801 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:28.115355 31076 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:28.117163 31076 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.117614 31076 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:10:28.121876 22868 leader_election.cc:336] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.122290 22868 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 7aad11e70ca94fce8f672db05f1c28c3, c45e1805f049467697a7358fb566de2f
I20260709 13:10:28.123334 31085 raft_consensus.cc:2749] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:28.132524 31085 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.133003 31085 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.135056 31085 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:28.137943 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:28.138576 23453 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
W20260709 13:10:28.141410 22868 leader_election.cc:336] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.142027 31072 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.142140 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.142555 31072 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.144539 31075 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:28.144673 31072 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:28.144944 31075 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.145414 31075 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:28.145557 31085 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.146082 31085 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.148015 31085 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:28.152560 31075 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.154428 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:28.155020 23453 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
W20260709 13:10:28.156140 22868 leader_election.cc:336] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.156957 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:28.160080 23419 leader_election.cc:336] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:28.163164 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:10:28.163911 31084 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:28.164309 31084 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.164645 31084 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:28.132033 31076 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.175485 31083 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.175987 31083 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.167289 31075 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:28.177822 31083 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:28.176365 23416 leader_election.cc:336] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.178828 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.179774 31083 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:28.185792 31084 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.189226 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:28.189814 22901 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 19.
I20260709 13:10:28.190402 31084 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:28.190588 21629 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22852)
I20260709 13:10:28.191222 23453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09a9ba6091094973a526986ef573e2a9" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:28.191715 23453 raft_consensus.cc:3060] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:28.188730 22868 leader_election.cc:336] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:28.193176 22868 leader_election.cc:336] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.198410 31076 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:28.199709 23453 raft_consensus.cc:2468] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:10:28.202755 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317f6a16e0af41e2984a9b1626ab0edc" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:28.210382 23451 raft_consensus.cc:3060] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:28.203173 23448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "135e2f7520a34b53aaf4ddb360a8a029" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:28.217646 23448 raft_consensus.cc:3060] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:28.230723 23448 raft_consensus.cc:2468] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 21.
W20260709 13:10:28.207969 22868 leader_election.cc:336] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.206833 23416 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 
I20260709 13:10:28.235196 31083 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:28.235564 31083 raft_consensus.cc:493] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.235728 22868 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.235917 31083 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:28.236810 22868 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.239578 31076 raft_consensus.cc:2804] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:28.240094 31076 raft_consensus.cc:697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:28.241010 31076 consensus_queue.cc:237] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.244455 31084 raft_consensus.cc:2804] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:28.244946 31084 raft_consensus.cc:697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:28.245797 31084 consensus_queue.cc:237] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.233860 23451 raft_consensus.cc:2468] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 22.
I20260709 13:10:28.250120 31083 raft_consensus.cc:515] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:28.250826 23416 leader_election.cc:336] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.258069 31083 leader_election.cc:290] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:28.266703 22868 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.267386 22654 catalog_manager.cc:5697] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 21 to 22, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 22 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.268407 31067 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.274592 31072 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.280272 31072 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.280814 31093 raft_consensus.cc:2804] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:28.277486 22901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e4b3212e80646379ccf77b5952a4363" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2"
I20260709 13:10:28.279531 31067 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.281750 22901 raft_consensus.cc:3060] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:10:28.282656 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:10:28.286899 31093 raft_consensus.cc:697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:28.290726 31093 consensus_queue.cc:237] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.292289 31072 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:28.293944 31067 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:28.297096 23416 leader_election.cc:336] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.298084 31076 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.300534 31076 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.302361 31076 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:28.298463 22901 raft_consensus.cc:2468] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 20.
W20260709 13:10:28.309260 22868 leader_election.cc:336] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.311094 23416 leader_election.cc:304] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.311812 31072 raft_consensus.cc:2804] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:28.312258 31072 raft_consensus.cc:697] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Becoming Leader. State: Replica: c45e1805f049467697a7358fb566de2f, State: Running, Role: LEADER
I20260709 13:10:28.313185 31072 consensus_queue.cc:237] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.315140 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
W20260709 13:10:28.315323 23419 leader_election.cc:336] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:28.315744 23451 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 23.
W20260709 13:10:28.315834 23419 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:28.316918 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.317711 31096 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:10:28.318399 31096 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:28.319013 31096 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:28.334095 31096 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.336077 31096 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Requested vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:28.350638 22868 leader_election.cc:336] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:28.351315 23416 leader_election.cc:336] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.352020 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:28.353785 23416 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:28.354162 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.355547 31072 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:28.356796 31067 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:10:28.359025 22654 catalog_manager.cc:5697] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f reported cstate change: term changed from 19 to 20, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to c45e1805f049467697a7358fb566de2f (127.21.31.68). New cstate: current_term: 20 leader_uuid: "c45e1805f049467697a7358fb566de2f" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:28.361027 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27b79b7fd4bf4e90a1a7f357a253688d" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f"
I20260709 13:10:28.361562 23451 raft_consensus.cc:3060] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:28.368752 23451 raft_consensus.cc:2468] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 24.
I20260709 13:10:28.376343 22868 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:28.377410 31093 raft_consensus.cc:2804] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:10:28.378080 31093 raft_consensus.cc:697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Becoming Leader. State: Replica: 3b626c41600d4737a178ebab8a7ebfc2, State: Running, Role: LEADER
I20260709 13:10:28.378903 31093 consensus_queue.cc:237] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:28.388151 22649 catalog_manager.cc:5697] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 20 to 21, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 21 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.389889 22649 catalog_manager.cc:5697] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 21 to 22, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 22 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:28.395527 23416 consensus_peers.cc:597] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:28.395874 22901 raft_consensus.cc:1275] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:10:28.399741 31067 consensus_queue.cc:1048] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:10:28.431034 22901 raft_consensus.cc:1275] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:10:28.432782 31072 consensus_queue.cc:1048] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:10:28.442484 23451 raft_consensus.cc:1275] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
W20260709 13:10:28.444801 23416 consensus_peers.cc:597] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:28.444784 31075 consensus_queue.cc:1048] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:10:28.454300 23448 raft_consensus.cc:1275] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 13:10:28.457623 31076 consensus_queue.cc:1048] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
W20260709 13:10:28.475018 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:10:28.494053 22649 catalog_manager.cc:5697] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 reported cstate change: term changed from 23 to 24, leader changed from 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66) to 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67). New cstate: current_term: 24 leader_uuid: "3b626c41600d4737a178ebab8a7ebfc2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:28.509734 22868 consensus_peers.cc:597] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:28.544955 22868 consensus_peers.cc:597] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:28.573593 23419 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:28.589347 23419 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:28.601845 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:28.626029 22868 consensus_peers.cc:597] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:28.636863 22868 consensus_peers.cc:597] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:28.656209 22868 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:28.672869 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:28.682029 23416 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:28.686836 22871 consensus_peers.cc:597] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:28.687626 22868 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:28.703050 22868 consensus_peers.cc:597] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:28.745065 23416 consensus_peers.cc:597] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:28.751993 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:10:28.768395 23448 raft_consensus.cc:1275] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:10:28.771356 31075 consensus_queue.cc:1048] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:10:28.779520 22868 consensus_peers.cc:597] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:28.783329 23451 raft_consensus.cc:1275] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:10:28.783903 22868 consensus_peers.cc:597] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:28.785346 31075 consensus_queue.cc:1048] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:10:28.790475 23416 consensus_peers.cc:597] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:28.792580 23453 raft_consensus.cc:1275] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:10:28.794291 31075 consensus_queue.cc:1048] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:10:28.811259 22871 consensus_peers.cc:597] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:28.824864 23416 consensus_peers.cc:597] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:28.845949 23416 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:28.891161 22868 consensus_peers.cc:597] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:28.912634 23416 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:28.933941 22896 raft_consensus.cc:1275] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Refusing update from remote peer c45e1805f049467697a7358fb566de2f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:10:28.936271 31083 consensus_queue.cc:1048] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:10:28.940019 23416 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:28.982328 23419 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:10:28.996943 23453 raft_consensus.cc:1275] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Refusing update from remote peer 3b626c41600d4737a178ebab8a7ebfc2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:10:28.998379 23416 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:29.020113 23416 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:29.021620 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:29.032737 22868 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:29.037362 22868 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:29.064582 31075 consensus_queue.cc:1048] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:10:29.057644 23416 consensus_peers.cc:597] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:29.080042 23416 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:29.083906 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:29.128819 23419 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:29.144019 23416 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:29.168988 23419 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:29.186807 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:10:29.193475 21629 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 22852)
I20260709 13:10:29.303864 31076 raft_consensus.cc:493] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.304602 31076 raft_consensus.cc:515] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:29.309191 31076 leader_election.cc:290] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:29.307211 22871 consensus_peers.cc:597] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:29.321055 22871 leader_election.cc:336] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:29.326817 22868 leader_election.cc:336] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:29.329497 22868 leader_election.cc:304] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.331115 31085 raft_consensus.cc:2749] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:29.339483 22868 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:29.385016 31083 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.385537 31083 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:29.387557 31083 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:29.391834 23416 leader_election.cc:336] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:29.392333 23419 leader_election.cc:336] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.392724 23419 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.393908 31083 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:29.447548 23416 consensus_peers.cc:597] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:29.475327 22868 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:29.481818 31077 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.482539 31077 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:29.492223 22896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "c45e1805f049467697a7358fb566de2f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3b626c41600d4737a178ebab8a7ebfc2" is_pre_election: true
I20260709 13:10:29.492863 22896 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c45e1805f049467697a7358fb566de2f in term 14.
W20260709 13:10:29.524659 23416 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:29.490440 31077 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:29.528244 31085 raft_consensus.cc:493] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.528755 31085 raft_consensus.cc:515] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:29.531170 31085 leader_election.cc:290] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:29.540410 23419 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.541484 23419 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.544672 31077 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:29.554252 22871 leader_election.cc:336] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:29.556674 22868 leader_election.cc:336] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:29.558126 22868 leader_election.cc:304] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.623749 31116 raft_consensus.cc:493] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.625224 31076 raft_consensus.cc:493] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.626225 31076 raft_consensus.cc:515] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:29.627148 31075 raft_consensus.cc:2749] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:29.629845 31076 leader_election.cc:290] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:29.630455 31076 raft_consensus.cc:493] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.631193 31076 raft_consensus.cc:515] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:29.631451 31075 raft_consensus.cc:493] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.637897 31075 raft_consensus.cc:515] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:29.639474 31116 raft_consensus.cc:515] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
I20260709 13:10:29.640324 31075 leader_election.cc:290] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:29.642040 31116 leader_election.cc:290] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:29.643410 22868 leader_election.cc:336] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:29.643580 31076 leader_election.cc:290] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:29.653992 22868 leader_election.cc:336] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:29.654397 22868 leader_election.cc:336] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:29.654937 22868 leader_election.cc:336] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:29.663642 22871 leader_election.cc:336] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.664311 22871 leader_election.cc:304] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.665216 31115 raft_consensus.cc:2749] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:29.672065 22871 leader_election.cc:336] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.672513 22871 leader_election.cc:304] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.673282 31115 raft_consensus.cc:2749] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:29.674355 22871 leader_election.cc:336] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.674788 22871 leader_election.cc:304] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.675820 31115 raft_consensus.cc:2749] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:29.676882 22871 leader_election.cc:336] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.677397 22871 leader_election.cc:304] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.678290 31115 raft_consensus.cc:2749] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:29.701894 23419 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:10:29.713272 31083 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.714185 31083 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:29.719517 31083 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:29.724514 23416 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:29.728550 23419 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.729022 23419 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.729915 31083 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:10:29.750617 31117 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.751371 31117 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:29.753801 31117 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:29.758993 23416 leader_election.cc:336] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:29.780412 23419 leader_election.cc:336] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.781032 23419 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.781839 31117 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:29.787411 31117 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.788144 31117 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
W20260709 13:10:29.794253 23416 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:29.796972 31117 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:29.797680 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:29.814014 23419 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.814821 23419 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.828064 31117 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:29.843755 31117 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.844611 31117 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:29.847920 31117 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:29.855109 23416 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:29.866276 23419 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:29.866772 23419 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:29.867794 31117 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:10:29.943109 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:29.945704 22871 consensus_peers.cc:597] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:10:29.973516 31115 raft_consensus.cc:493] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:29.974485 31115 raft_consensus.cc:515] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } }
W20260709 13:10:29.994438 22871 consensus_peers.cc:597] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:29.997496 22871 leader_election.cc:336] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:30.009186 22868 leader_election.cc:336] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:30.004340 31115 leader_election.cc:290] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
I20260709 13:10:30.009804 22868 leader_election.cc:304] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:30.010769 31076 raft_consensus.cc:2749] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:30.120604 31076 raft_consensus.cc:493] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:30.121202 31076 raft_consensus.cc:515] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } }
I20260709 13:10:30.123633 31076 leader_election.cc:290] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213)
W20260709 13:10:30.132016 22871 consensus_peers.cc:597] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:30.132436 22871 leader_election.cc:336] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:30.194056 21629 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 22852)
W20260709 13:10:30.230448 22868 leader_election.cc:336] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:30.241287 22868 leader_election.cc:304] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:30.242237 31115 raft_consensus.cc:2749] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:30.246565 31115 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:30.252943 31115 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:30.263633 31115 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
W20260709 13:10:30.273154 22871 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:30.279820 22868 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:30.282591 22868 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:30.283856 31115 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:30.293210 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1879799f5d0c462c9d2c95bd8a88862e" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:30.294145 23451 raft_consensus.cc:2468] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 17.
I20260709 13:10:30.353807 31119 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:30.354516 31119 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:30.356812 31119 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:30.361660 23419 leader_election.cc:336] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:30.363688 23416 leader_election.cc:336] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:30.364971 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:30.365697 31119 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:30.372084 31119 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:30.372692 31119 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:30.377012 31119 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:30.378144 23419 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:30.380131 23416 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:30.380473 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:30.381386 31119 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:10:30.423818 22871 consensus_peers.cc:597] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:30.459501 22871 consensus_peers.cc:597] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:30.574097 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:30.639802 22871 consensus_peers.cc:597] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:30.726604 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:10:30.742599 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:30.782330 23416 consensus_peers.cc:597] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:30.878685 23419 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:30.892918 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 135e2f7520a34b53aaf4ddb360a8a029: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:30.901566 21629 tablet_server.cc:196] TabletServer@127.21.31.66:0 shutdown complete.
W20260709 13:10:30.922955 23416 consensus_peers.cc:597] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:31.006109 22871 consensus_peers.cc:597] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:10:31.038611 31115 raft_consensus.cc:493] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:31.039279 31115 raft_consensus.cc:515] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } attrs { replace: true } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:31.041481 21629 tablet_server.cc:179] TabletServer@127.21.31.67:0 shutting down...
I20260709 13:10:31.044729 31115 leader_election.cc:290] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), c45e1805f049467697a7358fb566de2f (127.21.31.68:42017)
I20260709 13:10:31.047310 23451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d8a8f3cf129492eb679da24703f5b84" candidate_uuid: "3b626c41600d4737a178ebab8a7ebfc2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "c45e1805f049467697a7358fb566de2f" is_pre_election: true
I20260709 13:10:31.048066 23451 raft_consensus.cc:2468] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b626c41600d4737a178ebab8a7ebfc2 in term 14.
W20260709 13:10:31.049729 22868 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:31.055871 23416 consensus_peers.cc:597] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:31.056546 22871 leader_election.cc:336] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:31.057003 22871 leader_election.cc:304] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3b626c41600d4737a178ebab8a7ebfc2, c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:31.059070 31115 raft_consensus.cc:2749] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:31.069469 22868 consensus_peers.cc:597] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:10:31.072541 23416 consensus_peers.cc:597] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:31.095988 23416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111) [suppressed 601 similar messages]
W20260709 13:10:31.106240 23419 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:10:31.119482 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:10:31.120090 23419 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:10:31.120689 21629 tablet_replica.cc:333] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.121673 21629 raft_consensus.cc:2243] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.122314 21629 raft_consensus.cc:2272] T 9cf0bfa7a80d4f0286947f1e6ff89772 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.124377 21629 tablet_replica.cc:333] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.124923 21629 raft_consensus.cc:2243] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.125909 21629 raft_consensus.cc:2272] T 1879799f5d0c462c9d2c95bd8a88862e P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.127853 21629 tablet_replica.cc:333] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.128402 21629 raft_consensus.cc:2243] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.129272 21629 raft_consensus.cc:2272] T efec6b734c374a169f3dc3b729ef2f07 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.131127 21629 tablet_replica.cc:333] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.131683 21629 raft_consensus.cc:2243] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.132442 21629 raft_consensus.cc:2272] T f8c8f692d4994a148c306609ab807ab7 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.134279 21629 tablet_replica.cc:333] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.134816 21629 raft_consensus.cc:2243] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.135412 21629 raft_consensus.cc:2272] T a28fec9b1516463db9c7feac432f9ad5 P 3b626c41600d4737a178ebab8a7ebfc2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.137079 21629 tablet_replica.cc:333] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.137666 21629 raft_consensus.cc:2243] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.138245 21629 raft_consensus.cc:2272] T 9e12cb0ef4bd490aa6a0afd9cd76ff18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.140054 21629 tablet_replica.cc:333] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.140601 21629 raft_consensus.cc:2243] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.141265 21629 raft_consensus.cc:2272] T 8ea49b0ee95d4eafbb647436a98fcaa1 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.143765 21629 tablet_replica.cc:333] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.144438 21629 raft_consensus.cc:2243] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.145251 21629 raft_consensus.cc:2272] T a42e3464af3e4e5eb580790b400a6eff P 3b626c41600d4737a178ebab8a7ebfc2 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.147590 21629 tablet_replica.cc:333] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.148331 21629 raft_consensus.cc:2243] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.149240 21629 raft_consensus.cc:2272] T 3a98a0b933184c5590c80b44a6e3a544 P 3b626c41600d4737a178ebab8a7ebfc2 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.150993 21629 tablet_replica.cc:333] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.151479 21629 raft_consensus.cc:2243] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.152002 21629 raft_consensus.cc:2272] T dbd3d6688dfb4621b8ecfd11278d5b26 P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.153838 21629 tablet_replica.cc:333] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.154316 21629 raft_consensus.cc:2243] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.155176 21629 raft_consensus.cc:2272] T ac8ca58483824d0bbd1e56d06088b4da P 3b626c41600d4737a178ebab8a7ebfc2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.156917 21629 tablet_replica.cc:333] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.157485 21629 raft_consensus.cc:2243] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.158474 21629 raft_consensus.cc:2272] T 67449ba22ccb4938be8a5e28664cc566 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.161427 21629 tablet_replica.cc:333] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.162400 21629 raft_consensus.cc:2243] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.163717 21629 raft_consensus.cc:2272] T 8da6707b1a3d47a48c3671a6b7401196 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.166510 21629 tablet_replica.cc:333] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.167227 21629 raft_consensus.cc:2243] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.167786 21629 raft_consensus.cc:2272] T 6436de6ceb37488b9ee8b6d261356a40 P 3b626c41600d4737a178ebab8a7ebfc2 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.171444 21629 tablet_replica.cc:333] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.172223 21629 raft_consensus.cc:2243] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.173666 21629 raft_consensus.cc:2272] T 60a957de2a744260812f8da6f1bd9b87 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:31.174189 23416 consensus_peers.cc:597] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:31.176136 21629 tablet_replica.cc:333] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.176908 21629 raft_consensus.cc:2243] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.177752 21629 raft_consensus.cc:2272] T 90274c6677ab4d37b51190e940c9608e P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.180331 21629 tablet_replica.cc:333] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.181043 21629 raft_consensus.cc:2243] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.181675 21629 raft_consensus.cc:2272] T 267f1f9ab3f742418b8505ba4859ab18 P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.184230 21629 tablet_replica.cc:333] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.184945 21629 raft_consensus.cc:2243] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:31.185743 23416 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:31.186234 21629 raft_consensus.cc:2272] T 3583f8fda02d46af9ef0ff2ae6f1b443 P 3b626c41600d4737a178ebab8a7ebfc2 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:31.187752 22868 consensus_peers.cc:597] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:31.188637 21629 tablet_replica.cc:333] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.189934 21629 raft_consensus.cc:2243] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:31.190804 22868 consensus_peers.cc:597] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:31.191603 21629 raft_consensus.cc:2272] T 70588fa182854c9dbc786bdd4f807fc9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.194113 21629 tablet_replica.cc:333] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.194758 21629 raft_consensus.cc:2243] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.195351 21629 raft_consensus.cc:2272] T f7d0e2e573344ec49ba2412e5de1d519 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:31.195660 23416 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:31.197854 21629 tablet_replica.cc:333] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.198668 21629 raft_consensus.cc:2243] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.199776 21629 raft_consensus.cc:2272] T 6ce4678cb66d4d1e8325b447eaf2a630 P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.202903 21629 tablet_replica.cc:333] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.203822 21629 raft_consensus.cc:2243] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:10:31.204075 23416 consensus_peers.cc:597] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:31.205034 21629 raft_consensus.cc:2272] T 42c5a324a80f42d4965b69d86915a65d P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.207156 21629 tablet_replica.cc:333] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
W20260709 13:10:31.207645 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08001dc500 after lost signal to thread 21632
I20260709 13:10:31.207945 21629 raft_consensus.cc:2243] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.208608 21629 raft_consensus.cc:2272] T 4b7d2a5f73cc4afd98b29c55cc0eb72b P 3b626c41600d4737a178ebab8a7ebfc2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:31.208709 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08001c2540 after lost signal to thread 22688
W20260709 13:10:31.210062 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08001d65a0 after lost signal to thread 22929
I20260709 13:10:31.211212 21629 tablet_replica.cc:333] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
W20260709 13:10:31.211184 22926 debug-util.cc:398] Leaking SignalData structure 0x7b08001b2140 after lost signal to thread 23478
I20260709 13:10:31.212116 21629 raft_consensus.cc:2243] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.212914 21629 raft_consensus.cc:2272] T ab1914222f984a8f9db2e3956a97bad0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.215405 21629 tablet_replica.cc:333] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.216151 21629 raft_consensus.cc:2243] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.217341 21629 raft_consensus.cc:2272] T db154be77d6e4b5388f9a3622f91a4ba P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.219957 21629 tablet_replica.cc:333] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.220634 21629 raft_consensus.cc:2243] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.221611 21629 raft_consensus.cc:2272] T 7f765d75643f47dd939002936724a315 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.224154 21629 tablet_replica.cc:333] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.224839 21629 raft_consensus.cc:2243] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.226289 21629 raft_consensus.cc:2272] T 317f6a16e0af41e2984a9b1626ab0edc P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:31.228116 23416 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:31.228694 21629 tablet_replica.cc:333] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.229975 21629 raft_consensus.cc:2243] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.231503 21629 raft_consensus.cc:2272] T 469a8bc68a8b4e62b19f382c167cf50c P 3b626c41600d4737a178ebab8a7ebfc2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.234707 21629 tablet_replica.cc:333] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.235528 21629 raft_consensus.cc:2243] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.237159 21629 raft_consensus.cc:2272] T 135e2f7520a34b53aaf4ddb360a8a029 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:31.239151 23416 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:31.239758 21629 tablet_replica.cc:333] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.240557 21629 raft_consensus.cc:2243] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.241848 21629 raft_consensus.cc:2272] T 847c8a2cc0bb4233a1d55ca04643418d P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.244519 21629 tablet_replica.cc:333] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.245285 21629 raft_consensus.cc:2243] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.245992 21629 raft_consensus.cc:2272] T 7701b9b4e69441f7ae9de9554134de69 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.248524 21629 tablet_replica.cc:333] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.249696 21629 raft_consensus.cc:2243] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.251698 21629 raft_consensus.cc:2272] T 09a9ba6091094973a526986ef573e2a9 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.254619 21629 tablet_replica.cc:333] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.255262 21629 raft_consensus.cc:2243] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.256903 21629 raft_consensus.cc:2272] T 4e4b3212e80646379ccf77b5952a4363 P 3b626c41600d4737a178ebab8a7ebfc2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.259456 21629 tablet_replica.cc:333] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.259991 21629 raft_consensus.cc:2243] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.260995 21629 raft_consensus.cc:2272] T 4b9578eb52c342078c7555d5100bdcad P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.263260 21629 tablet_replica.cc:333] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.263824 21629 raft_consensus.cc:2243] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.264894 21629 raft_consensus.cc:2272] T 4d8a8f3cf129492eb679da24703f5b84 P 3b626c41600d4737a178ebab8a7ebfc2 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.267460 21629 tablet_replica.cc:333] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
W20260709 13:10:31.267786 22868 consensus_peers.cc:597] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:31.268122 21629 raft_consensus.cc:2243] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.268754 21629 raft_consensus.cc:2272] T 53ca1af745cb4fd48a40af06ae1c43ca P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.270630 21629 tablet_replica.cc:333] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.271142 21629 raft_consensus.cc:2243] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.272264 21629 raft_consensus.cc:2272] T 84ea43361e534fbebad704327c9f6367 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.274854 21629 tablet_replica.cc:333] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.275966 21629 raft_consensus.cc:2243] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.276779 21629 raft_consensus.cc:2272] T 6ce771dbb93248d397c8d5b222b27fca P 3b626c41600d4737a178ebab8a7ebfc2 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:31.279397 23416 consensus_peers.cc:597] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:31.279644 21629 tablet_replica.cc:333] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.280462 21629 raft_consensus.cc:2243] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:10:31.281052 23416 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:31.281853 21629 raft_consensus.cc:2272] T ccab55924ea9473faf1d4d97313e9284 P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.284415 21629 tablet_replica.cc:333] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.285086 21629 raft_consensus.cc:2243] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.285653 21629 raft_consensus.cc:2272] T 11e8d51b5b18431cb7d6df2884a4e879 P 3b626c41600d4737a178ebab8a7ebfc2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.288054 21629 tablet_replica.cc:333] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.288823 21629 raft_consensus.cc:2243] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:31.289054 23416 consensus_peers.cc:597] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:31.289762 21629 raft_consensus.cc:2272] T 18051671d25f4aa7b30d4a299909f91a P 3b626c41600d4737a178ebab8a7ebfc2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.292883 21629 tablet_replica.cc:333] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.293741 21629 raft_consensus.cc:2243] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.294337 21629 raft_consensus.cc:2272] T e8ff32d0405c426786004110e1c1f955 P 3b626c41600d4737a178ebab8a7ebfc2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.296970 21629 tablet_replica.cc:333] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.297619 21629 raft_consensus.cc:2243] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:10:31.298758 21629 raft_consensus.cc:2272] T 27b79b7fd4bf4e90a1a7f357a253688d P 3b626c41600d4737a178ebab8a7ebfc2 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:31.299296 23416 consensus_peers.cc:597] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:10:31.301260 21629 tablet_replica.cc:333] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.301920 21629 raft_consensus.cc:2243] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.302470 21629 raft_consensus.cc:2272] T e4cc99e6a1f14071a587af061e82fee0 P 3b626c41600d4737a178ebab8a7ebfc2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:31.306067 21629 tablet_replica.cc:333] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2: stopping tablet replica
I20260709 13:10:31.306630 21629 raft_consensus.cc:2243] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:31.307221 21629 raft_consensus.cc:2272] T 41fb40751eb74da580d83c9bb69ae58a P 3b626c41600d4737a178ebab8a7ebfc2 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:31.318257 23416 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.332551 23419 consensus_peers.cc:597] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:31.343489 23416 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:31.344300 23416 consensus_peers.cc:597] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:31.344941 23416 consensus_peers.cc:597] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.353240 23419 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:31.359320 23416 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:31.361845 23416 consensus_peers.cc:597] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:31.397886 23416 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:31.401008 23416 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.447644 23416 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.453935 23416 consensus_peers.cc:597] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.457195 23416 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:31.499452 23419 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:31.525249 23416 consensus_peers.cc:597] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.553539 23416 consensus_peers.cc:597] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f -> Peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Couldn't send request to peer 3b626c41600d4737a178ebab8a7ebfc2. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:31.554091 23419 consensus_peers.cc:597] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:31.594182 23416 consensus_peers.cc:597] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.606104 23419 consensus_peers.cc:597] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:31.641824 23416 consensus_peers.cc:597] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.706857 23416 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.748844 23419 consensus_peers.cc:597] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:10:31.777662 23419 consensus_peers.cc:597] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:10:31.784569 31083 raft_consensus.cc:493] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:31.785049 31083 raft_consensus.cc:515] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:31.787086 31083 leader_election.cc:290] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:31.795359 23419 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:31.799393 23416 leader_election.cc:336] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:31.799849 23416 leader_election.cc:304] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:31.800583 31083 raft_consensus.cc:2749] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:10:31.812202 23416 consensus_peers.cc:597] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:10:31.904992 23419 consensus_peers.cc:597] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:10:32.110883 23416 consensus_peers.cc:597] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:32.156865 31127 raft_consensus.cc:493] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.157403 31127 raft_consensus.cc:515] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.159128 31127 leader_election.cc:290] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:32.162765 23419 leader_election.cc:336] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.164016 23416 leader_election.cc:336] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
I20260709 13:10:32.164448 23416 leader_election.cc:304] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:32.165004 31127 raft_consensus.cc:2749] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:32.182536 31119 raft_consensus.cc:493] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:32.183032 31119 raft_consensus.cc:515] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.185048 31119 leader_election.cc:290] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:32.188265 23419 leader_election.cc:336] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.192265 23416 leader_election.cc:336] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.192699 23416 leader_election.cc:304] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.193248 31119 raft_consensus.cc:2749] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:32.234958 23419 consensus_peers.cc:597] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:10:32.291656 31127 raft_consensus.cc:493] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.292112 31127 raft_consensus.cc:515] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.293850 31127 leader_election.cc:290] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:32.293752 31119 raft_consensus.cc:493] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.294502 31119 raft_consensus.cc:515] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.296414 31119 leader_election.cc:290] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:32.300313 23416 leader_election.cc:336] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:32.301010 23416 leader_election.cc:336] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
W20260709 13:10:32.301554 23416 leader_election.cc:336] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
I20260709 13:10:32.301863 23416 leader_election.cc:304] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.302398 31127 raft_consensus.cc:2749] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:32.302544 23416 leader_election.cc:336] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
I20260709 13:10:32.303006 23416 leader_election.cc:304] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.303602 31127 raft_consensus.cc:2749] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:32.313138 23419 consensus_peers.cc:597] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:10:32.331130 23419 consensus_peers.cc:597] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:10:32.408125 31127 raft_consensus.cc:493] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.408632 31127 raft_consensus.cc:515] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.410419 31127 leader_election.cc:290] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:32.411651 31127 raft_consensus.cc:493] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.412112 31127 raft_consensus.cc:515] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.413972 31127 leader_election.cc:290] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:32.414355 31119 raft_consensus.cc:493] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:32.414909 31119 raft_consensus.cc:515] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.417040 31119 leader_election.cc:290] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:32.421563 23419 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.423892 23416 leader_election.cc:336] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.424573 31119 raft_consensus.cc:493] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.425025 31119 raft_consensus.cc:515] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.427048 31119 leader_election.cc:290] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:32.430871 23419 leader_election.cc:336] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.432893 23416 leader_election.cc:336] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
I20260709 13:10:32.433382 23416 leader_election.cc:304] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.434118 31119 raft_consensus.cc:2749] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:32.435077 23416 leader_election.cc:336] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
W20260709 13:10:32.436715 23416 leader_election.cc:336] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.437135 23416 leader_election.cc:304] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.437966 31119 raft_consensus.cc:2749] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:10:32.438907 23416 leader_election.cc:336] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.439317 23416 leader_election.cc:304] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.439990 31119 raft_consensus.cc:2749] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:10:32.442397 23416 leader_election.cc:336] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
I20260709 13:10:32.442800 23416 leader_election.cc:304] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:32.443470 31119 raft_consensus.cc:2749] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:32.520354 31127 raft_consensus.cc:493] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:32.521003 31127 raft_consensus.cc:515] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.523310 31127 leader_election.cc:290] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:32.528172 23419 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.528721 23416 leader_election.cc:336] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.529248 23416 leader_election.cc:304] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.530005 31127 raft_consensus.cc:2749] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:32.538774 31127 raft_consensus.cc:493] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.539278 31127 raft_consensus.cc:515] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.540774 31127 leader_election.cc:290] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:32.545181 23416 leader_election.cc:336] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
W20260709 13:10:32.546130 23416 leader_election.cc:336] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.546473 23416 leader_election.cc:304] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.547008 31127 raft_consensus.cc:2749] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:10:32.551882 31127 raft_consensus.cc:493] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:32.552264 31127 raft_consensus.cc:515] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.554981 31083 raft_consensus.cc:493] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.555482 31083 raft_consensus.cc:515] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.557869 31083 leader_election.cc:290] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
I20260709 13:10:32.553956 31127 leader_election.cc:290] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:32.559973 23419 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.562422 23419 leader_election.cc:336] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.562872 23416 leader_election.cc:336] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.563314 23416 leader_election.cc:304] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.563848 31127 raft_consensus.cc:2749] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:10:32.565176 23416 leader_election.cc:336] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
I20260709 13:10:32.565585 23416 leader_election.cc:304] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:32.566133 31127 raft_consensus.cc:2749] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:10:32.587944 31127 raft_consensus.cc:493] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:32.588485 31127 raft_consensus.cc:515] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.590472 31127 leader_election.cc:290] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:32.594069 23419 leader_election.cc:336] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.595368 23416 leader_election.cc:336] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.595701 23416 leader_election.cc:304] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.596284 31127 raft_consensus.cc:2749] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:32.611996 31127 raft_consensus.cc:493] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.612435 31127 raft_consensus.cc:515] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.616564 31127 leader_election.cc:290] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:32.618696 23416 leader_election.cc:336] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
W20260709 13:10:32.620645 23419 leader_election.cc:336] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:32.621035 23419 leader_election.cc:304] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 6a5191aaff76469bbf6c4735dc9e6509
I20260709 13:10:32.622077 31083 raft_consensus.cc:2749] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:32.648669 31127 raft_consensus.cc:493] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.649559 31127 raft_consensus.cc:515] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.651453 31127 leader_election.cc:290] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:32.656471 23416 leader_election.cc:336] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Network error: Client connection negotiation failed: client connection to 127.21.31.67:35797: connect: Connection refused (error 111)
W20260709 13:10:32.657035 23416 leader_election.cc:336] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.657423 23416 leader_election.cc:304] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.658398 31127 raft_consensus.cc:2749] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:10:32.680066 31127 raft_consensus.cc:493] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.681011 31127 raft_consensus.cc:515] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.684091 31127 leader_election.cc:290] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:32.687551 23416 leader_election.cc:336] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.723285 31127 raft_consensus.cc:493] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3b626c41600d4737a178ebab8a7ebfc2)
I20260709 13:10:32.723729 31127 raft_consensus.cc:515] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.725625 31127 leader_election.cc:290] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:32.729290 23416 leader_election.cc:336] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.801831 31127 raft_consensus.cc:493] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:32.802491 31127 raft_consensus.cc:515] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "3b626c41600d4737a178ebab8a7ebfc2" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 35797 } attrs { replace: true } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.805176 31127 leader_election.cc:290] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003), 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797)
W20260709 13:10:32.808485 23419 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.810207 23416 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.810550 23416 leader_election.cc:304] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.811108 31127 raft_consensus.cc:2749] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:32.962256 31083 raft_consensus.cc:493] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7aad11e70ca94fce8f672db05f1c28c3)
I20260709 13:10:32.962811 31083 raft_consensus.cc:515] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6a5191aaff76469bbf6c4735dc9e6509" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 37213 } } peers { permanent_uuid: "7aad11e70ca94fce8f672db05f1c28c3" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39003 } } peers { permanent_uuid: "c45e1805f049467697a7358fb566de2f" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 42017 } attrs { promote: false } }
I20260709 13:10:32.964798 31083 leader_election.cc:290] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213), 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003)
W20260709 13:10:32.975929 23419 leader_election.cc:336] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:32.981042 23416 leader_election.cc:336] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111)
I20260709 13:10:32.981392 23416 leader_election.cc:304] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 6a5191aaff76469bbf6c4735dc9e6509, 7aad11e70ca94fce8f672db05f1c28c3
I20260709 13:10:32.981997 31083 raft_consensus.cc:2749] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:33.034029 21629 tablet_server.cc:196] TabletServer@127.21.31.67:0 shutdown complete.
W20260709 13:10:33.067577 23419 consensus_peers.cc:597] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:10:33.137168 21629 tablet_server.cc:179] TabletServer@127.21.31.68:0 shutting down...
W20260709 13:10:33.163978 23419 consensus_peers.cc:597] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:10:33.189370 23419 consensus_peers.cc:597] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f -> Peer 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): Couldn't send request to peer 6a5191aaff76469bbf6c4735dc9e6509. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:10:33.190006 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:33.191092 21629 tablet_replica.cc:333] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.192107 21629 raft_consensus.cc:2243] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.192958 21629 raft_consensus.cc:2272] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.196472 21629 tablet_replica.cc:333] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.197475 21629 raft_consensus.cc:2243] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.198830 21629 raft_consensus.cc:2272] T 84ea43361e534fbebad704327c9f6367 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.201893 21629 tablet_replica.cc:333] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.202639 21629 raft_consensus.cc:2243] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.204147 21629 raft_consensus.cc:2272] T 18a127a20cfa4496bad51cb634bd0d0f P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.207350 21629 tablet_replica.cc:333] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.208158 21629 raft_consensus.cc:2243] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.208997 21629 raft_consensus.cc:2272] T a28fec9b1516463db9c7feac432f9ad5 P c45e1805f049467697a7358fb566de2f [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.211894 21629 tablet_replica.cc:333] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.212981 21629 raft_consensus.cc:2243] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.213583 21629 raft_consensus.cc:2272] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.216713 21629 tablet_replica.cc:333] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.217624 21629 raft_consensus.cc:2243] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.218225 21629 raft_consensus.cc:2272] T 469a8bc68a8b4e62b19f382c167cf50c P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.221084 21629 tablet_replica.cc:333] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.221885 21629 raft_consensus.cc:2243] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.222654 21629 raft_consensus.cc:2272] T 70588fa182854c9dbc786bdd4f807fc9 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.225494 21629 tablet_replica.cc:333] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.226253 21629 raft_consensus.cc:2243] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.226784 21629 raft_consensus.cc:2272] T db154be77d6e4b5388f9a3622f91a4ba P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.229434 21629 tablet_replica.cc:333] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.230166 21629 raft_consensus.cc:2243] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.230888 21629 raft_consensus.cc:2272] T 264d1889b1f144c5897b33c834b1beef P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.233598 21629 tablet_replica.cc:333] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.234351 21629 raft_consensus.cc:2243] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.235296 21629 raft_consensus.cc:2272] T 41fb40751eb74da580d83c9bb69ae58a P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.238901 21629 tablet_replica.cc:333] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.239430 21629 raft_consensus.cc:2243] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.240269 21629 raft_consensus.cc:2272] T 82d0d73de20d49d3a64fef8b052288e5 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.242710 21629 tablet_replica.cc:333] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.243458 21629 raft_consensus.cc:2243] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.244359 21629 raft_consensus.cc:2272] T 3833f41661774b918ddf80499fd85f2f P c45e1805f049467697a7358fb566de2f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.246560 21629 tablet_replica.cc:333] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.247175 21629 raft_consensus.cc:2243] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:10:33.247710 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.247830 21629 raft_consensus.cc:2272] T 8ea49b0ee95d4eafbb647436a98fcaa1 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.250025 21629 tablet_replica.cc:333] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.250700 21629 raft_consensus.cc:2243] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.251683 21629 raft_consensus.cc:2272] T 267f1f9ab3f742418b8505ba4859ab18 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.253921 21629 tablet_replica.cc:333] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.254478 21629 raft_consensus.cc:2243] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.255039 21629 raft_consensus.cc:2272] T 60a957de2a744260812f8da6f1bd9b87 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.256975 21629 tablet_replica.cc:333] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.257596 21629 raft_consensus.cc:2243] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.258181 21629 raft_consensus.cc:2272] T a42e3464af3e4e5eb580790b400a6eff P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.260113 21629 tablet_replica.cc:333] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.260613 21629 raft_consensus.cc:2243] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.261559 21629 raft_consensus.cc:2272] T 00324d171dc0468fa259f13c1d4d74ea P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.263624 21629 tablet_replica.cc:333] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.264161 21629 raft_consensus.cc:2243] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.264822 21629 raft_consensus.cc:2272] T 53ca1af745cb4fd48a40af06ae1c43ca P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.266718 21629 tablet_replica.cc:333] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.267220 21629 raft_consensus.cc:2243] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.267773 21629 raft_consensus.cc:2272] T f53f146bb25444ce8d2072559efe5e59 P c45e1805f049467697a7358fb566de2f [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.269786 21629 tablet_replica.cc:333] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.270268 21629 raft_consensus.cc:2243] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.270948 21629 raft_consensus.cc:2272] T f7d0e2e573344ec49ba2412e5de1d519 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.272737 21629 tablet_replica.cc:333] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.273277 21629 raft_consensus.cc:2243] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.273734 21629 raft_consensus.cc:2272] T 317f6a16e0af41e2984a9b1626ab0edc P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.275581 21629 tablet_replica.cc:333] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.276059 21629 raft_consensus.cc:2243] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.276701 21629 raft_consensus.cc:2272] T 6436de6ceb37488b9ee8b6d261356a40 P c45e1805f049467697a7358fb566de2f [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.278600 21629 tablet_replica.cc:333] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.279095 21629 raft_consensus.cc:2243] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.279757 21629 raft_consensus.cc:2272] T 90274c6677ab4d37b51190e940c9608e P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.282150 21629 tablet_replica.cc:333] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.282900 21629 raft_consensus.cc:2243] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.283504 21629 raft_consensus.cc:2272] T 27b79b7fd4bf4e90a1a7f357a253688d P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.286002 21629 tablet_replica.cc:333] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.286512 21629 raft_consensus.cc:2243] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.287163 21629 raft_consensus.cc:2272] T e8ff32d0405c426786004110e1c1f955 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.289016 21629 tablet_replica.cc:333] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.289572 21629 raft_consensus.cc:2243] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.290223 21629 raft_consensus.cc:2272] T 18051671d25f4aa7b30d4a299909f91a P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.292100 21629 tablet_replica.cc:333] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.292616 21629 raft_consensus.cc:2243] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:10:33.293594 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.293650 21629 raft_consensus.cc:2272] T 4e4b3212e80646379ccf77b5952a4363 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.296165 21629 tablet_replica.cc:333] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.296677 21629 raft_consensus.cc:2243] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.297104 21629 raft_consensus.cc:2272] T 09a9ba6091094973a526986ef573e2a9 P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.298921 21629 tablet_replica.cc:333] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.299486 21629 raft_consensus.cc:2243] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.300007 21629 raft_consensus.cc:2272] T 4b9578eb52c342078c7555d5100bdcad P c45e1805f049467697a7358fb566de2f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.301811 21629 tablet_replica.cc:333] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.302285 21629 raft_consensus.cc:2243] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.302827 21629 raft_consensus.cc:2272] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.304658 21629 tablet_replica.cc:333] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.305150 21629 raft_consensus.cc:2243] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.305763 21629 raft_consensus.cc:2272] T 4d8a8f3cf129492eb679da24703f5b84 P c45e1805f049467697a7358fb566de2f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.307775 21629 tablet_replica.cc:333] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.308260 21629 raft_consensus.cc:2243] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.308701 21629 raft_consensus.cc:2272] T 42c5a324a80f42d4965b69d86915a65d P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.310998 21629 tablet_replica.cc:333] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.311684 21629 raft_consensus.cc:2243] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.312240 21629 raft_consensus.cc:2272] T ac8ca58483824d0bbd1e56d06088b4da P c45e1805f049467697a7358fb566de2f [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:33.312935 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.314849 21629 tablet_replica.cc:333] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
W20260709 13:10:33.315243 23416 consensus_peers.cc:597] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f -> Peer 7aad11e70ca94fce8f672db05f1c28c3 (127.21.31.66:39003): Couldn't send request to peer 7aad11e70ca94fce8f672db05f1c28c3. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39003: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:10:33.315579 21629 raft_consensus.cc:2243] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.316082 21629 raft_consensus.cc:2272] T 4bbeaee00aa942ccb232c812c9e14827 P c45e1805f049467697a7358fb566de2f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.317920 21629 tablet_replica.cc:333] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.318420 21629 raft_consensus.cc:2243] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.319252 21629 raft_consensus.cc:2272] T c7d4d50470a044f698ff4852a4b634ca P c45e1805f049467697a7358fb566de2f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.321583 21629 tablet_replica.cc:333] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.322214 21629 raft_consensus.cc:2243] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.322896 21629 raft_consensus.cc:2272] T 1f881fed251c4393ac053086bd572549 P c45e1805f049467697a7358fb566de2f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.325156 21629 tablet_replica.cc:333] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.325667 21629 raft_consensus.cc:2243] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.326059 21629 raft_consensus.cc:2272] T 3a98a0b933184c5590c80b44a6e3a544 P c45e1805f049467697a7358fb566de2f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.327769 21629 tablet_replica.cc:333] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.328226 21629 raft_consensus.cc:2243] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.328711 21629 raft_consensus.cc:2272] T d0d74135bf874a0bb4b857b686effcfd P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:33.330479 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.330503 21629 tablet_replica.cc:333] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.331243 21629 raft_consensus.cc:2243] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:10:33.331827 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.332376 21629 raft_consensus.cc:2272] T 8f0563f7efc8452c86a8c67c0189fe81 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.334331 21629 tablet_replica.cc:333] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.334863 21629 raft_consensus.cc:2243] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:33.335422 21629 raft_consensus.cc:2272] T 4ad75d8751c54bf78bcf4fa8cc6578c1 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.337232 21629 tablet_replica.cc:333] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.337782 21629 raft_consensus.cc:2243] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.338644 21629 raft_consensus.cc:2272] T 43c3bd6e29d8478b8aa6c100c8fd84c2 P c45e1805f049467697a7358fb566de2f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.340510 21629 tablet_replica.cc:333] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.340968 21629 raft_consensus.cc:2243] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.341661 21629 raft_consensus.cc:2272] T 9cf0bfa7a80d4f0286947f1e6ff89772 P c45e1805f049467697a7358fb566de2f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.343525 21629 tablet_replica.cc:333] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.343986 21629 raft_consensus.cc:2243] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.344606 21629 raft_consensus.cc:2272] T 11e8d51b5b18431cb7d6df2884a4e879 P c45e1805f049467697a7358fb566de2f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.346401 21629 tablet_replica.cc:333] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.346858 21629 raft_consensus.cc:2243] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.347784 21629 raft_consensus.cc:2272] T 2147efadd4ea46a09bda395402a10299 P c45e1805f049467697a7358fb566de2f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:33.349807 21629 tablet_replica.cc:333] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f: stopping tablet replica
I20260709 13:10:33.350247 21629 raft_consensus.cc:2243] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:10:33.350867 21629 raft_consensus.cc:2272] T 7701b9b4e69441f7ae9de9554134de69 P c45e1805f049467697a7358fb566de2f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:33.394323 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.401630 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.449651 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.461683 23419 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.472411 23418 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.526350 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.638260 23416 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.652329 23417 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.690544 30748 leader_election.cc:336] T 1879799f5d0c462c9d2c95bd8a88862e P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Service unavailable: reactor is shutting down (error 108)
W20260709 13:10:33.695242 30748 leader_election.cc:336] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Service unavailable: reactor is shutting down (error 108)
I20260709 13:10:33.695663 30748 leader_election.cc:304] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:33.697002 30748 raft_consensus.cc:2686] T 7f765d75643f47dd939002936724a315 P c45e1805f049467697a7358fb566de2f: unable to run election callback: Service unavailable: The pool has been shut down.
W20260709 13:10:33.698905 30748 leader_election.cc:336] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3b626c41600d4737a178ebab8a7ebfc2 (127.21.31.67:35797): Service unavailable: reactor is shutting down (error 108)
I20260709 13:10:33.699194 30748 leader_election.cc:304] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c45e1805f049467697a7358fb566de2f; no voters: 3b626c41600d4737a178ebab8a7ebfc2, 7aad11e70ca94fce8f672db05f1c28c3
W20260709 13:10:33.699640 30748 raft_consensus.cc:2686] T 135e2f7520a34b53aaf4ddb360a8a029 P c45e1805f049467697a7358fb566de2f: unable to run election callback: Service unavailable: The pool has been shut down.
I20260709 13:10:34.202230 21629 tablet_server.cc:196] TabletServer@127.21.31.68:0 shutdown complete.
I20260709 13:10:34.278669 21629 master.cc:562] Master@127.21.31.126:35001 shutting down...
W20260709 13:10:34.332052 22702 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:10:34.332427 22702 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:10:34.332654 22702 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:10:42.308187 22641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111) [suppressed 263 similar messages]
W20260709 13:10:42.312091 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4e4b3212e80646379ccf77b5952a4363: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
W20260709 13:10:47.302320 22641 catalog_manager.cc:4755] TS 6a5191aaff76469bbf6c4735dc9e6509 (127.21.31.65:37213): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 135e2f7520a34b53aaf4ddb360a8a029: Network error: Client connection negotiation failed: client connection to 127.21.31.65:37213: connect: Connection refused (error 111)
I20260709 13:10:47.950798 21629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:47.952035 21629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:47.952397 21629 tablet_replica.cc:333] T 00000000000000000000000000000000 P 359166ef74e042839c4e58133432df83: stopping tablet replica
I20260709 13:10:48.189672 21629 master.cc:584] Master@127.21.31.126:35001 shutdown complete.
I20260709 13:10:48.242259 21629 test_util.cc:182] -----------------------------------------------
I20260709 13:10:48.242468 21629 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602359137460-21629-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (265234 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:10:48.245750 21629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.31.126:37055
I20260709 13:10:48.246778 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:48.252180 31135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:48.252411 31136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:48.253636 31138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:48.254755 21629 server_base.cc:1061] running on GCE node
I20260709 13:10:48.258689 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:48.258878 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:48.258988 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602648258977 us; error 0 us; skew 500 ppm
I20260709 13:10:48.259429 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:48.261718 21629 webserver.cc:533] Webserver started at http://127.21.31.126:42383/ using document root <none> and password file <none>
I20260709 13:10:48.262146 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:48.262296 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:48.262496 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:48.263687 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/master-0-root/instance:
uuid: "9ca35edff3a347dc8711fd15b6385750"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.268303 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 13:10:48.272222 31143 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.273293 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:10:48.273640 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/master-0-root
uuid: "9ca35edff3a347dc8711fd15b6385750"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.273937 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:48.306406 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:48.307866 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:48.350344 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.126:37055
I20260709 13:10:48.350471 31194 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.126:37055 every 8 connection(s)
I20260709 13:10:48.354949 31195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:48.365597 31195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750: Bootstrap starting.
I20260709 13:10:48.370393 31195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.374347 31195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750: No bootstrap required, opened a new log
I20260709 13:10:48.376488 31195 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca35edff3a347dc8711fd15b6385750" member_type: VOTER }
I20260709 13:10:48.376885 31195 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.377066 31195 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ca35edff3a347dc8711fd15b6385750, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.377560 31195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca35edff3a347dc8711fd15b6385750" member_type: VOTER }
I20260709 13:10:48.377960 31195 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:48.378161 31195 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:48.378369 31195 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.382287 31195 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca35edff3a347dc8711fd15b6385750" member_type: VOTER }
I20260709 13:10:48.382710 31195 leader_election.cc:304] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ca35edff3a347dc8711fd15b6385750; no voters: 
I20260709 13:10:48.383796 31195 leader_election.cc:290] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:48.384215 31198 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.385555 31198 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 1 LEADER]: Becoming Leader. State: Replica: 9ca35edff3a347dc8711fd15b6385750, State: Running, Role: LEADER
I20260709 13:10:48.386260 31198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca35edff3a347dc8711fd15b6385750" member_type: VOTER }
I20260709 13:10:48.386700 31195 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:48.393728 31200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ca35edff3a347dc8711fd15b6385750. Latest consensus state: current_term: 1 leader_uuid: "9ca35edff3a347dc8711fd15b6385750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca35edff3a347dc8711fd15b6385750" member_type: VOTER } }
I20260709 13:10:48.393860 31199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ca35edff3a347dc8711fd15b6385750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca35edff3a347dc8711fd15b6385750" member_type: VOTER } }
I20260709 13:10:48.394680 31200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:48.394735 31199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:48.396420 31208 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:48.401129 31208 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:48.402247 21629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:48.410339 31208 catalog_manager.cc:1383] Generated new cluster ID: 7114584e3f0043d79e09eb5588672edc
I20260709 13:10:48.410645 31208 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:48.430255 31208 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:48.431474 31208 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:48.448029 31208 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750: Generated new TSK 0
I20260709 13:10:48.448750 31208 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:48.469169 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:48.476001 31216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:48.478257 31217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:48.478796 31219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:48.479702 21629 server_base.cc:1061] running on GCE node
I20260709 13:10:48.482007 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:48.482182 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:48.482398 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602648482359 us; error 0 us; skew 500 ppm
I20260709 13:10:48.483060 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:48.485466 21629 webserver.cc:533] Webserver started at http://127.21.31.65:33339/ using document root <none> and password file <none>
I20260709 13:10:48.485947 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:48.486107 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:48.486418 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:48.487557 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/instance:
uuid: "75b6f0017ce84a4c9822ffd27e053405"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.491902 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:10:48.495085 31224 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.495977 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:10:48.496328 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "75b6f0017ce84a4c9822ffd27e053405"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.496640 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:48.520448 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:48.521586 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:48.522981 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:48.525172 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:48.525403 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.525653 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:48.525830 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.566591 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:36133
I20260709 13:10:48.566737 31286 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:36133 every 8 connection(s)
I20260709 13:10:48.572831 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:48.580804 31291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:48.582484 31292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:48.583940 31294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:48.585779 21629 server_base.cc:1061] running on GCE node
I20260709 13:10:48.588663 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:48.588943 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:48.589144 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602648589129 us; error 0 us; skew 500 ppm
I20260709 13:10:48.589952 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:48.591723 31287 heartbeater.cc:344] Connected to a master server at 127.21.31.126:37055
I20260709 13:10:48.592156 31287 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:48.592690 21629 webserver.cc:533] Webserver started at http://127.21.31.66:39919/ using document root <none> and password file <none>
I20260709 13:10:48.593029 31287 heartbeater.cc:507] Master 127.21.31.126:37055 requested a full tablet report, sending...
I20260709 13:10:48.593146 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:48.593465 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:48.593673 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:48.595160 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/instance:
uuid: "07b814fcedea4142ac09385b680cae3a"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.595606 31160 ts_manager.cc:194] Registered new tserver with Master: 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:48.598656 31160 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60438
I20260709 13:10:48.601616 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:10:48.605844 31299 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.606798 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:10:48.607142 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-1-root
uuid: "07b814fcedea4142ac09385b680cae3a"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.607479 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:48.632371 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:48.633654 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:48.635097 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:48.637233 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:48.637511 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.637746 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:48.637923 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.682071 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.66:39075
I20260709 13:10:48.682183 31361 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.66:39075 every 8 connection(s)
I20260709 13:10:48.687059 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:48.694514 31365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:48.695510 31366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:48.697676 31368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:48.698237 21629 server_base.cc:1061] running on GCE node
I20260709 13:10:48.699394 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:48.699666 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:48.699878 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602648699836 us; error 0 us; skew 500 ppm
I20260709 13:10:48.700459 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:48.700848 31362 heartbeater.cc:344] Connected to a master server at 127.21.31.126:37055
I20260709 13:10:48.701211 31362 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:48.702108 31362 heartbeater.cc:507] Master 127.21.31.126:37055 requested a full tablet report, sending...
I20260709 13:10:48.703033 21629 webserver.cc:533] Webserver started at http://127.21.31.67:33155/ using document root <none> and password file <none>
I20260709 13:10:48.703613 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:48.703819 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:48.704104 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:48.704243 31160 ts_manager.cc:194] Registered new tserver with Master: 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:48.705430 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/instance:
uuid: "51b0533abe064293ac5047de0ac9bf25"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.705663 31160 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59678
I20260709 13:10:48.710435 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:10:48.714632 31373 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.715490 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:10:48.715811 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-2-root
uuid: "51b0533abe064293ac5047de0ac9bf25"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.716104 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:48.753521 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:48.754863 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:48.756333 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:48.758523 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:48.758755 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.758991 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:48.759161 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.811908 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.67:45117
I20260709 13:10:48.812021 31435 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.67:45117 every 8 connection(s)
I20260709 13:10:48.817364 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:48.824797 31439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:48.826256 31440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:48.828145 21629 server_base.cc:1061] running on GCE node
W20260709 13:10:48.828519 31442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:48.829584 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:48.829859 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:48.830137 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602648830100 us; error 0 us; skew 500 ppm
I20260709 13:10:48.830659 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:48.831092 31436 heartbeater.cc:344] Connected to a master server at 127.21.31.126:37055
I20260709 13:10:48.831470 31436 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:48.832222 31436 heartbeater.cc:507] Master 127.21.31.126:37055 requested a full tablet report, sending...
I20260709 13:10:48.833446 21629 webserver.cc:533] Webserver started at http://127.21.31.68:32961/ using document root <none> and password file <none>
I20260709 13:10:48.833973 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:48.834189 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:48.834105 31160 ts_manager.cc:194] Registered new tserver with Master: 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:48.834683 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:48.835685 31160 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59682
I20260709 13:10:48.835879 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-3-root/instance:
uuid: "001cac731610478a900ad62e830aaff6"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.840409 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.001s
I20260709 13:10:48.844769 31447 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.845842 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:10:48.846156 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-3-root
uuid: "001cac731610478a900ad62e830aaff6"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:10:48.846439 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:48.872705 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:48.874074 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:48.875376 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:48.877774 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:48.877950 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.878123 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:48.878257 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:48.923384 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.68:44895
I20260709 13:10:48.923556 31509 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.68:44895 every 8 connection(s)
I20260709 13:10:48.939240 31510 heartbeater.cc:344] Connected to a master server at 127.21.31.126:37055
I20260709 13:10:48.939805 31510 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:48.940825 31510 heartbeater.cc:507] Master 127.21.31.126:37055 requested a full tablet report, sending...
I20260709 13:10:48.943207 31160 ts_manager.cc:194] Registered new tserver with Master: 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:48.943425 21629 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016724509s
I20260709 13:10:48.944993 31160 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59690
I20260709 13:10:48.971088 31160 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59700:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:10:49.230197 31327 tablet_service.cc:1511] Processing CreateTablet for tablet 73bf9042b4104cb693d9daf0f280166c (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:49.231678 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73bf9042b4104cb693d9daf0f280166c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.232637 31399 tablet_service.cc:1511] Processing CreateTablet for tablet 914efa83b45e44dbbabeca35e75d0f3e (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:49.233665 31401 tablet_service.cc:1511] Processing CreateTablet for tablet 73bf9042b4104cb693d9daf0f280166c (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:49.234158 31399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 914efa83b45e44dbbabeca35e75d0f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.234973 31401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73bf9042b4104cb693d9daf0f280166c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.236498 31398 tablet_service.cc:1511] Processing CreateTablet for tablet e12592f1e5624454a04480b80ff628e2 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:49.237941 31398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12592f1e5624454a04480b80ff628e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.238592 31400 tablet_service.cc:1511] Processing CreateTablet for tablet bb349a8fff1b4d6f8dda0834dcec5b71 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:49.239977 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb349a8fff1b4d6f8dda0834dcec5b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.242624 31397 tablet_service.cc:1511] Processing CreateTablet for tablet dda3de9e2b1643049b35455ababf5d82 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:49.242857 31396 tablet_service.cc:1511] Processing CreateTablet for tablet 942702984f5b4342bdd5399274624f90 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:49.244050 31397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda3de9e2b1643049b35455ababf5d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.245959 31475 tablet_service.cc:1511] Processing CreateTablet for tablet 73bf9042b4104cb693d9daf0f280166c (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:49.247350 31475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73bf9042b4104cb693d9daf0f280166c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.248284 31396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 942702984f5b4342bdd5399274624f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.251205 31326 tablet_service.cc:1511] Processing CreateTablet for tablet bb349a8fff1b4d6f8dda0834dcec5b71 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:49.252593 31326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb349a8fff1b4d6f8dda0834dcec5b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.259627 31252 tablet_service.cc:1511] Processing CreateTablet for tablet bb349a8fff1b4d6f8dda0834dcec5b71 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:49.260998 31252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb349a8fff1b4d6f8dda0834dcec5b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.261595 31471 tablet_service.cc:1511] Processing CreateTablet for tablet dda3de9e2b1643049b35455ababf5d82 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:49.263490 31471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda3de9e2b1643049b35455ababf5d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.263872 31473 tablet_service.cc:1511] Processing CreateTablet for tablet 914efa83b45e44dbbabeca35e75d0f3e (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:49.261592 31474 tablet_service.cc:1511] Processing CreateTablet for tablet 5db13a06e22e411ba4c29d105c7e93f7 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:49.266547 31472 tablet_service.cc:1511] Processing CreateTablet for tablet a2ecee4c21f34ad0a27f7f36add491e9 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:49.265250 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 914efa83b45e44dbbabeca35e75d0f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.269642 31474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db13a06e22e411ba4c29d105c7e93f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.274195 31472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2ecee4c21f34ad0a27f7f36add491e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.276860 31323 tablet_service.cc:1511] Processing CreateTablet for tablet a2ecee4c21f34ad0a27f7f36add491e9 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:49.278226 31323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2ecee4c21f34ad0a27f7f36add491e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.281841 31395 tablet_service.cc:1511] Processing CreateTablet for tablet f12b34e35ea0429ea3c77b7d89fd748a (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:49.282500 31322 tablet_service.cc:1511] Processing CreateTablet for tablet 942702984f5b4342bdd5399274624f90 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:49.283218 31395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12b34e35ea0429ea3c77b7d89fd748a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.283834 31322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 942702984f5b4342bdd5399274624f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.286386 31325 tablet_service.cc:1511] Processing CreateTablet for tablet 5db13a06e22e411ba4c29d105c7e93f7 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:49.286897 31324 tablet_service.cc:1511] Processing CreateTablet for tablet e12592f1e5624454a04480b80ff628e2 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:49.287842 31325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db13a06e22e411ba4c29d105c7e93f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.288509 31324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12592f1e5624454a04480b80ff628e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.291481 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 5db13a06e22e411ba4c29d105c7e93f7 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:49.292820 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db13a06e22e411ba4c29d105c7e93f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.294493 31250 tablet_service.cc:1511] Processing CreateTablet for tablet 914efa83b45e44dbbabeca35e75d0f3e (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:49.295785 31250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 914efa83b45e44dbbabeca35e75d0f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.295501 31470 tablet_service.cc:1511] Processing CreateTablet for tablet 942702984f5b4342bdd5399274624f90 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:49.298051 31470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 942702984f5b4342bdd5399274624f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.298645 31248 tablet_service.cc:1511] Processing CreateTablet for tablet a2ecee4c21f34ad0a27f7f36add491e9 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:49.300022 31248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2ecee4c21f34ad0a27f7f36add491e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.302397 31249 tablet_service.cc:1511] Processing CreateTablet for tablet e12592f1e5624454a04480b80ff628e2 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:49.303752 31249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12592f1e5624454a04480b80ff628e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.307322 31247 tablet_service.cc:1511] Processing CreateTablet for tablet dda3de9e2b1643049b35455ababf5d82 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:49.308646 31247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda3de9e2b1643049b35455ababf5d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.310099 31530 tablet_bootstrap.cc:492] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:49.310530 31246 tablet_service.cc:1511] Processing CreateTablet for tablet f12b34e35ea0429ea3c77b7d89fd748a (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:49.311810 31246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12b34e35ea0429ea3c77b7d89fd748a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.315201 31321 tablet_service.cc:1511] Processing CreateTablet for tablet f12b34e35ea0429ea3c77b7d89fd748a (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:49.316565 31321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12b34e35ea0429ea3c77b7d89fd748a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.362313 31469 tablet_service.cc:1511] Processing CreateTablet for tablet c24424b197c343e1a57a4f490c67aaf3 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:49.363612 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24424b197c343e1a57a4f490c67aaf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.364413 31245 tablet_service.cc:1511] Processing CreateTablet for tablet c24424b197c343e1a57a4f490c67aaf3 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:49.365757 31245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24424b197c343e1a57a4f490c67aaf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.382266 31322 tablet_service.cc:1511] Processing CreateTablet for tablet c24424b197c343e1a57a4f490c67aaf3 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:49.394862 31473 tablet_service.cc:1511] Processing CreateTablet for tablet f70d36cace8741d7b866e776f6cc03c1 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:49.396229 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f70d36cace8741d7b866e776f6cc03c1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:49.397230 31207 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:49.398351 31322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24424b197c343e1a57a4f490c67aaf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.398969 31530 tablet_bootstrap.cc:654] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.409456 31399 tablet_service.cc:1511] Processing CreateTablet for tablet 35faa1d23acb4a6da70dc14e16ed6a69 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:49.410547 31327 tablet_service.cc:1511] Processing CreateTablet for tablet 35faa1d23acb4a6da70dc14e16ed6a69 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:49.411434 31399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35faa1d23acb4a6da70dc14e16ed6a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.412360 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35faa1d23acb4a6da70dc14e16ed6a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.412658 31471 tablet_service.cc:1511] Processing CreateTablet for tablet 35faa1d23acb4a6da70dc14e16ed6a69 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:49.413954 31471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35faa1d23acb4a6da70dc14e16ed6a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.415112 31531 tablet_bootstrap.cc:492] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:49.416356 31325 tablet_service.cc:1511] Processing CreateTablet for tablet b41a57045456429c857b8fb94b54c356 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:49.417639 31325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41a57045456429c857b8fb94b54c356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.421972 31396 tablet_service.cc:1511] Processing CreateTablet for tablet f70d36cace8741d7b866e776f6cc03c1 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:49.423199 31396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f70d36cace8741d7b866e776f6cc03c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.424391 31249 tablet_service.cc:1511] Processing CreateTablet for tablet b41a57045456429c857b8fb94b54c356 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:49.427799 31400 tablet_service.cc:1511] Processing CreateTablet for tablet b41a57045456429c857b8fb94b54c356 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:49.432550 31471 tablet_service.cc:1511] Processing CreateTablet for tablet fa7546f06a454f3899cb0bf7d0c0d084 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:49.433105 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41a57045456429c857b8fb94b54c356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.433768 31471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa7546f06a454f3899cb0bf7d0c0d084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.437127 31532 tablet_bootstrap.cc:492] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:49.443558 31250 tablet_service.cc:1511] Processing CreateTablet for tablet fa7546f06a454f3899cb0bf7d0c0d084 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:49.447359 31531 tablet_bootstrap.cc:654] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.447537 31251 tablet_service.cc:1511] Processing CreateTablet for tablet f70d36cace8741d7b866e776f6cc03c1 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:49.433110 31249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41a57045456429c857b8fb94b54c356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.450112 31533 tablet_bootstrap.cc:492] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:49.451886 31323 tablet_service.cc:1511] Processing CreateTablet for tablet fa7546f06a454f3899cb0bf7d0c0d084 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:49.470443 31250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa7546f06a454f3899cb0bf7d0c0d084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.471378 31395 tablet_service.cc:1511] Processing CreateTablet for tablet 24fe066d118d48c5939576417491e9ec (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:49.472784 31395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24fe066d118d48c5939576417491e9ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.481704 31323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa7546f06a454f3899cb0bf7d0c0d084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.482244 31245 tablet_service.cc:1511] Processing CreateTablet for tablet 24fe066d118d48c5939576417491e9ec (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:49.483858 31245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24fe066d118d48c5939576417491e9ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.484539 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f70d36cace8741d7b866e776f6cc03c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.490029 31324 tablet_service.cc:1511] Processing CreateTablet for tablet acf056205d504594827b6fda57d3cb62 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:49.491427 31324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acf056205d504594827b6fda57d3cb62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.486196 31532 tablet_bootstrap.cc:654] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.496590 31533 tablet_bootstrap.cc:654] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.512894 31531 tablet_bootstrap.cc:492] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:49.513427 31531 ts_tablet_manager.cc:1403] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.017s
I20260709 13:10:49.514298 31469 tablet_service.cc:1511] Processing CreateTablet for tablet 24fe066d118d48c5939576417491e9ec (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:49.515621 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24fe066d118d48c5939576417491e9ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.519246 31400 tablet_service.cc:1511] Processing CreateTablet for tablet acf056205d504594827b6fda57d3cb62 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:49.520792 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acf056205d504594827b6fda57d3cb62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.521376 31471 tablet_service.cc:1511] Processing CreateTablet for tablet acf056205d504594827b6fda57d3cb62 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:49.522575 31471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acf056205d504594827b6fda57d3cb62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.523409 31247 tablet_service.cc:1511] Processing CreateTablet for tablet 67b810039c1e415192775c9f611cd825 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:49.524601 31247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b810039c1e415192775c9f611cd825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.547261 31531 raft_consensus.cc:359] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.548085 31531 raft_consensus.cc:385] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.548382 31531 raft_consensus.cc:740] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.549176 31531 consensus_queue.cc:260] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.557273 31327 tablet_service.cc:1511] Processing CreateTablet for tablet 67b810039c1e415192775c9f611cd825 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:49.560478 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b810039c1e415192775c9f611cd825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.560926 31530 tablet_bootstrap.cc:492] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:49.561421 31530 ts_tablet_manager.cc:1403] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.252s	user 0.009s	sys 0.018s
I20260709 13:10:49.563144 31395 tablet_service.cc:1511] Processing CreateTablet for tablet 67b810039c1e415192775c9f611cd825 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:49.564399 31395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b810039c1e415192775c9f611cd825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.567646 31532 tablet_bootstrap.cc:492] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:49.568420 31532 ts_tablet_manager.cc:1403] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.132s	user 0.016s	sys 0.030s
I20260709 13:10:49.570986 31473 tablet_service.cc:1511] Processing CreateTablet for tablet 4b06bd7c5b2a4586b9caf425bc23720a (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:49.573468 31532 raft_consensus.cc:359] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:49.584373 31532 raft_consensus.cc:385] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.584795 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b06bd7c5b2a4586b9caf425bc23720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.584841 31532 raft_consensus.cc:740] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.586700 31532 consensus_queue.cc:260] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:49.564523 31530 raft_consensus.cc:359] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.579602 31245 tablet_service.cc:1511] Processing CreateTablet for tablet 4b06bd7c5b2a4586b9caf425bc23720a (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:49.588620 31530 raft_consensus.cc:385] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.590675 31530 raft_consensus.cc:740] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.594376 31530 consensus_queue.cc:260] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.594689 31400 tablet_service.cc:1511] Processing CreateTablet for tablet 181fa1bbe1e14e098cad0b23561576d2 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:49.595112 31533 tablet_bootstrap.cc:492] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:49.596486 31533 ts_tablet_manager.cc:1403] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.147s	user 0.010s	sys 0.006s
I20260709 13:10:49.596741 31362 heartbeater.cc:499] Master 127.21.31.126:37055 was elected leader, sending a full tablet report...
I20260709 13:10:49.597114 31245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b06bd7c5b2a4586b9caf425bc23720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.596763 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181fa1bbe1e14e098cad0b23561576d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.600318 31533 raft_consensus.cc:359] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:49.602141 31287 heartbeater.cc:499] Master 127.21.31.126:37055 was elected leader, sending a full tablet report...
I20260709 13:10:49.609916 31533 raft_consensus.cc:385] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.610759 31533 raft_consensus.cc:740] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.612371 31533 consensus_queue.cc:260] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:49.604743 31327 tablet_service.cc:1511] Processing CreateTablet for tablet 4b06bd7c5b2a4586b9caf425bc23720a (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:49.618181 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b06bd7c5b2a4586b9caf425bc23720a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.618527 31531 ts_tablet_manager.cc:1434] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.105s	user 0.018s	sys 0.007s
I20260709 13:10:49.618893 31469 tablet_service.cc:1511] Processing CreateTablet for tablet 181fa1bbe1e14e098cad0b23561576d2 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:49.619728 31531 tablet_bootstrap.cc:492] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:49.620484 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181fa1bbe1e14e098cad0b23561576d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.626483 31538 raft_consensus.cc:493] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.627334 31532 ts_tablet_manager.cc:1434] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.004s
I20260709 13:10:49.626984 31538 raft_consensus.cc:515] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.629705 31538 leader_election.cc:290] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:49.633956 31510 heartbeater.cc:499] Master 127.21.31.126:37055 was elected leader, sending a full tablet report...
I20260709 13:10:49.630236 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 181fa1bbe1e14e098cad0b23561576d2 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:49.634642 31533 ts_tablet_manager.cc:1434] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.038s	user 0.013s	sys 0.001s
I20260709 13:10:49.635073 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181fa1bbe1e14e098cad0b23561576d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.635700 31533 tablet_bootstrap.cc:492] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:49.639884 31436 heartbeater.cc:499] Master 127.21.31.126:37055 was elected leader, sending a full tablet report...
I20260709 13:10:49.645757 31533 tablet_bootstrap.cc:654] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.651753 31530 ts_tablet_manager.cc:1434] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.090s	user 0.012s	sys 0.007s
I20260709 13:10:49.665354 31532 tablet_bootstrap.cc:492] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:49.665849 31530 tablet_bootstrap.cc:492] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:49.672559 31530 tablet_bootstrap.cc:654] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:49.678073 31511 tablet.cc:2406] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:10:49.696511 31327 tablet_service.cc:1511] Processing CreateTablet for tablet 142741f7f41a4baa824a20e79ddf5cb6 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:49.697821 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142741f7f41a4baa824a20e79ddf5cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.694742 31395 tablet_service.cc:1511] Processing CreateTablet for tablet 142741f7f41a4baa824a20e79ddf5cb6 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:49.703066 31395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142741f7f41a4baa824a20e79ddf5cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.703583 31533 tablet_bootstrap.cc:492] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:49.704068 31533 ts_tablet_manager.cc:1403] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.069s	user 0.008s	sys 0.007s
I20260709 13:10:49.707082 31533 raft_consensus.cc:359] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.707803 31533 raft_consensus.cc:385] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.708094 31533 raft_consensus.cc:740] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.708894 31533 consensus_queue.cc:260] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.711090 31532 tablet_bootstrap.cc:654] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.720446 31251 tablet_service.cc:1511] Processing CreateTablet for tablet bfcdfdb50a8b463e851c89aa160fd949 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:49.721814 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfcdfdb50a8b463e851c89aa160fd949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.721726 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:49.711292 31533 ts_tablet_manager.cc:1434] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:49.725800 31533 tablet_bootstrap.cc:492] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:49.711855 31531 tablet_bootstrap.cc:654] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.714761 31469 tablet_service.cc:1511] Processing CreateTablet for tablet 142741f7f41a4baa824a20e79ddf5cb6 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:49.733057 31327 tablet_service.cc:1511] Processing CreateTablet for tablet bfcdfdb50a8b463e851c89aa160fd949 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:49.734728 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfcdfdb50a8b463e851c89aa160fd949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.745087 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142741f7f41a4baa824a20e79ddf5cb6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:49.745710 31301 leader_election.cc:343] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.754419 31533 tablet_bootstrap.cc:654] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.756695 31531 tablet_bootstrap.cc:492] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:49.756714 31532 tablet_bootstrap.cc:492] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:49.757436 31532 ts_tablet_manager.cc:1403] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.014s
I20260709 13:10:49.757436 31531 ts_tablet_manager.cc:1403] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.138s	user 0.022s	sys 0.000s
I20260709 13:10:49.761013 31532 raft_consensus.cc:359] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.761935 31532 raft_consensus.cc:385] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.762284 31532 raft_consensus.cc:740] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.762336 31530 tablet_bootstrap.cc:492] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:49.763157 31530 ts_tablet_manager.cc:1403] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.098s	user 0.020s	sys 0.000s
I20260709 13:10:49.763195 31532 consensus_queue.cc:260] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.764065 31531 raft_consensus.cc:359] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.764762 31531 raft_consensus.cc:385] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.765067 31531 raft_consensus.cc:740] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.766219 31532 ts_tablet_manager.cc:1434] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:10:49.765827 31531 consensus_queue.cc:260] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.767323 31532 tablet_bootstrap.cc:492] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:49.767032 31530 raft_consensus.cc:359] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:49.767825 31530 raft_consensus.cc:385] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.768110 31531 ts_tablet_manager.cc:1434] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:10:49.768127 31530 raft_consensus.cc:740] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.769035 31533 tablet_bootstrap.cc:492] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:49.769083 31531 tablet_bootstrap.cc:492] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:49.769798 31533 ts_tablet_manager.cc:1403] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.009s
I20260709 13:10:49.798945 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:49.804209 31530 consensus_queue.cc:260] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
W20260709 13:10:49.805095 31303 leader_election.cc:343] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.806334 31303 leader_election.cc:304] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:49.809039 31530 ts_tablet_manager.cc:1434] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:10:49.811830 31530 tablet_bootstrap.cc:492] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:49.818104 31538 raft_consensus.cc:2749] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.827009 31533 raft_consensus.cc:359] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:49.827929 31533 raft_consensus.cc:385] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.829624 31533 raft_consensus.cc:740] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.830858 31533 consensus_queue.cc:260] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:49.831761 31531 tablet_bootstrap.cc:654] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.835086 31533 ts_tablet_manager.cc:1434] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.006s
I20260709 13:10:49.836681 31532 tablet_bootstrap.cc:654] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.837378 31538 raft_consensus.cc:493] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.838229 31538 raft_consensus.cc:515] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.839301 31400 tablet_service.cc:1511] Processing CreateTablet for tablet bfcdfdb50a8b463e851c89aa160fd949 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:49.840895 31530 tablet_bootstrap.cc:654] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.841825 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
W20260709 13:10:49.843492 31303 leader_election.cc:343] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.846720 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
W20260709 13:10:49.849259 31301 leader_election.cc:343] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.849838 31301 leader_election.cc:304] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:49.851008 31538 leader_election.cc:290] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:49.837064 31473 tablet_service.cc:1511] Processing CreateTablet for tablet 663d2c98ef924d139ff1835938fef0df (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:49.851730 31538 raft_consensus.cc:2749] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.852866 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 663d2c98ef924d139ff1835938fef0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.841033 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfcdfdb50a8b463e851c89aa160fd949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:49.854696 31539 raft_consensus.cc:493] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.855794 31539 raft_consensus.cc:515] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.837067 31533 tablet_bootstrap.cc:492] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:49.882249 31539 leader_election.cc:290] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:49.893132 31533 tablet_bootstrap.cc:654] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.905632 31531 tablet_bootstrap.cc:492] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:49.906297 31531 ts_tablet_manager.cc:1403] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.137s	user 0.008s	sys 0.012s
I20260709 13:10:49.910326 31532 tablet_bootstrap.cc:492] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:49.910094 31531 raft_consensus.cc:359] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.910964 31531 raft_consensus.cc:385] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.911105 31532 ts_tablet_manager.cc:1403] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.144s	user 0.028s	sys 0.011s
I20260709 13:10:49.911334 31531 raft_consensus.cc:740] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.912464 31531 consensus_queue.cc:260] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.915526 31541 raft_consensus.cc:493] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.916216 31541 raft_consensus.cc:515] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:49.941527 31541 leader_election.cc:290] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:49.942083 31532 raft_consensus.cc:359] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.943020 31532 raft_consensus.cc:385] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.943849 31532 raft_consensus.cc:740] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.944761 31532 consensus_queue.cc:260] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.947337 31530 tablet_bootstrap.cc:492] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:49.948091 31530 ts_tablet_manager.cc:1403] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.137s	user 0.022s	sys 0.007s
I20260709 13:10:49.949024 31532 ts_tablet_manager.cc:1434] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.006s
I20260709 13:10:49.950803 31532 tablet_bootstrap.cc:492] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:49.951647 31533 tablet_bootstrap.cc:492] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:49.952175 31533 ts_tablet_manager.cc:1403] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.116s	user 0.020s	sys 0.014s
I20260709 13:10:49.952032 31530 raft_consensus.cc:359] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.952950 31530 raft_consensus.cc:385] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.953307 31530 raft_consensus.cc:740] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.954252 31530 consensus_queue.cc:260] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:49.956614 31530 ts_tablet_manager.cc:1434] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:49.957970 31530 tablet_bootstrap.cc:492] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:49.959610 31532 tablet_bootstrap.cc:654] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.961256 31533 raft_consensus.cc:359] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.962097 31533 raft_consensus.cc:385] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.962386 31533 raft_consensus.cc:740] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.966926 31540 raft_consensus.cc:493] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.943379 31531 ts_tablet_manager.cc:1434] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
I20260709 13:10:49.967991 31540 raft_consensus.cc:515] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:49.974432 31531 tablet_bootstrap.cc:492] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:49.978193 31533 consensus_queue.cc:260] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:49.980683 31533 ts_tablet_manager.cc:1434] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:10:49.982270 31533 tablet_bootstrap.cc:492] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:49.992547 31540 leader_election.cc:290] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:50.045567 31541 raft_consensus.cc:493] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.046651 31541 raft_consensus.cc:515] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.067305 31541 leader_election.cc:290] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:50.071547 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914efa83b45e44dbbabeca35e75d0f3e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:50.073704 31533 tablet_bootstrap.cc:654] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:50.074537 31226 leader_election.cc:343] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.081856 31530 tablet_bootstrap.cc:654] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.084977 31541 raft_consensus.cc:493] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.088352 31541 raft_consensus.cc:515] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.090319 31531 tablet_bootstrap.cc:654] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.090420 31540 raft_consensus.cc:493] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.091109 31540 raft_consensus.cc:515] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.091882 31541 leader_election.cc:290] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:50.108191 31540 leader_election.cc:290] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:50.117862 31531 tablet_bootstrap.cc:492] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:50.119910 31531 ts_tablet_manager.cc:1403] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.146s	user 0.010s	sys 0.012s
I20260709 13:10:50.120646 31532 tablet_bootstrap.cc:492] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:50.121153 31532 ts_tablet_manager.cc:1403] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.171s	user 0.053s	sys 0.017s
I20260709 13:10:50.122889 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda3de9e2b1643049b35455ababf5d82" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:50.123581 31411 raft_consensus.cc:2468] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:50.124491 31532 raft_consensus.cc:359] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.124835 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914efa83b45e44dbbabeca35e75d0f3e" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:50.125304 31532 raft_consensus.cc:385] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.125681 31532 raft_consensus.cc:740] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.126540 31532 consensus_queue.cc:260] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.126873 31449 leader_election.cc:304] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:50.128248 31541 raft_consensus.cc:2804] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:10:50.129168 31449 leader_election.cc:343] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.130643 31532 ts_tablet_manager.cc:1434] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:10:50.130860 31541 raft_consensus.cc:493] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.131253 31541 raft_consensus.cc:3060] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.131984 31532 tablet_bootstrap.cc:492] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:50.149296 31541 raft_consensus.cc:515] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:50.150338 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914efa83b45e44dbbabeca35e75d0f3e" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:50.150636 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda3de9e2b1643049b35455ababf5d82" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:50.151077 31261 raft_consensus.cc:2468] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:50.151268 31530 tablet_bootstrap.cc:492] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:50.151430 31485 raft_consensus.cc:2393] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 001cac731610478a900ad62e830aaff6 in this term.
I20260709 13:10:50.151829 31530 ts_tablet_manager.cc:1403] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.194s	user 0.007s	sys 0.012s
I20260709 13:10:50.152565 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db13a06e22e411ba4c29d105c7e93f7" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:50.152621 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda3de9e2b1643049b35455ababf5d82" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:50.152995 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bf9042b4104cb693d9daf0f280166c" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:50.153376 31261 raft_consensus.cc:2468] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:50.153420 31410 raft_consensus.cc:3060] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.155020 31531 raft_consensus.cc:359] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.155766 31531 raft_consensus.cc:385] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.155695 31451 leader_election.cc:304] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:50.156250 31531 raft_consensus.cc:740] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.153764 31484 raft_consensus.cc:2468] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:50.164580 31530 raft_consensus.cc:359] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.167099 31530 raft_consensus.cc:385] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.302101 31570 raft_consensus.cc:2804] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.302840 31570 raft_consensus.cc:493] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.303252 31570 raft_consensus.cc:3060] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.317956 31570 raft_consensus.cc:515] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.320889 31570 leader_election.cc:290] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:50.376659 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda3de9e2b1643049b35455ababf5d82" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:50.384922 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db13a06e22e411ba4c29d105c7e93f7" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
W20260709 13:10:50.386482 31448 leader_election.cc:343] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.387545 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bf9042b4104cb693d9daf0f280166c" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:50.388123 31337 raft_consensus.cc:2468] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:50.389703 31374 leader_election.cc:304] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:50.390451 31473 tablet_service.cc:1511] Processing CreateTablet for tablet 1b9a979d5b8b4d02b133ffc4078d9cbf (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:50.391849 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b9a979d5b8b4d02b133ffc4078d9cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.167483 31530 raft_consensus.cc:740] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.395406 31327 tablet_service.cc:1511] Processing CreateTablet for tablet 663d2c98ef924d139ff1835938fef0df (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:50.396844 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 663d2c98ef924d139ff1835938fef0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.157450 31531 consensus_queue.cc:260] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.404376 31531 ts_tablet_manager.cc:1434] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.284s	user 0.012s	sys 0.000s
I20260709 13:10:50.400023 31572 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.404332 31539 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.396184 31530 consensus_queue.cc:260] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.405386 31539 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.406059 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914efa83b45e44dbbabeca35e75d0f3e" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:50.406430 31531 tablet_bootstrap.cc:492] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:50.406698 31411 raft_consensus.cc:3060] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.407928 31539 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:50.409525 31572 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.413589 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
W20260709 13:10:50.415539 31374 leader_election.cc:343] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.416339 31572 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:50.416932 31572 raft_consensus.cc:2804] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.417333 31572 raft_consensus.cc:493] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.417649 31532 tablet_bootstrap.cc:654] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.417722 31572 raft_consensus.cc:3060] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.423349 31539 raft_consensus.cc:493] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.418228 31411 raft_consensus.cc:2468] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:50.418506 31530 ts_tablet_manager.cc:1434] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.266s	user 0.015s	sys 0.003s
I20260709 13:10:50.425756 31530 tablet_bootstrap.cc:492] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:50.426632 31572 raft_consensus.cc:515] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.428756 31572 leader_election.cc:290] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:50.418778 31531 tablet_bootstrap.cc:654] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.432786 31532 tablet_bootstrap.cc:492] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:50.433395 31532 ts_tablet_manager.cc:1403] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.302s	user 0.019s	sys 0.006s
I20260709 13:10:50.436045 31530 tablet_bootstrap.cc:654] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:50.436671 31451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:36133, user_credentials={real_user=slave}} blocked reactor thread for 281178us
I20260709 13:10:50.423957 31539 raft_consensus.cc:515] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.448701 31539 leader_election.cc:290] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:50.436618 31532 raft_consensus.cc:359] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.438386 31449 leader_election.cc:304] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:50.449797 31532 raft_consensus.cc:385] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.450170 31532 raft_consensus.cc:740] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.450423 31570 raft_consensus.cc:2804] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.450841 31532 consensus_queue.cc:260] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.440618 31541 leader_election.cc:290] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:50.441893 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bf9042b4104cb693d9daf0f280166c" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:50.454099 31451 leader_election.cc:304] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:50.454473 31337 raft_consensus.cc:3060] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.455466 31541 raft_consensus.cc:2804] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.455928 31541 raft_consensus.cc:493] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.456313 31541 raft_consensus.cc:3060] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.458833 31400 tablet_service.cc:1511] Processing CreateTablet for tablet 1b9a979d5b8b4d02b133ffc4078d9cbf (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:50.460515 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b9a979d5b8b4d02b133ffc4078d9cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.464838 31410 raft_consensus.cc:2468] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:50.466109 31532 ts_tablet_manager.cc:1434] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.004s
I20260709 13:10:50.465185 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda3de9e2b1643049b35455ababf5d82" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:50.467281 31570 raft_consensus.cc:697] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:10:50.467607 31532 tablet_bootstrap.cc:492] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
W20260709 13:10:50.468276 31377 leader_election.cc:343] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.468251 31570 consensus_queue.cc:237] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
W20260709 13:10:50.465677 31451 leader_election.cc:343] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.465817 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:50.470386 31261 raft_consensus.cc:2468] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:50.471455 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 663d2c98ef924d139ff1835938fef0df (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:50.472113 31449 leader_election.cc:304] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:50.473089 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 663d2c98ef924d139ff1835938fef0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.477984 31541 raft_consensus.cc:515] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.481287 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db13a06e22e411ba4c29d105c7e93f7" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
W20260709 13:10:50.482707 31448 leader_election.cc:343] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.486102 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73bf9042b4104cb693d9daf0f280166c" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:50.486678 31484 raft_consensus.cc:3060] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.488940 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db13a06e22e411ba4c29d105c7e93f7" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:50.489763 31260 raft_consensus.cc:3060] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.469061 31377 leader_election.cc:304] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:50.494973 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914efa83b45e44dbbabeca35e75d0f3e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:50.496193 31377 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:50.497162 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:50.498148 31533 tablet_bootstrap.cc:492] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:50.499558 31228 leader_election.cc:304] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:50.500727 31539 raft_consensus.cc:2749] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.499588 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda3de9e2b1643049b35455ababf5d82" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:50.502521 31245 tablet_service.cc:1511] Processing CreateTablet for tablet 1b9a979d5b8b4d02b133ffc4078d9cbf (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:50.503470 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:50.504215 31540 raft_consensus.cc:2749] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.504356 31409 raft_consensus.cc:2468] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
I20260709 13:10:50.505160 31540 raft_consensus.cc:2804] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.505646 31540 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.506065 31540 raft_consensus.cc:3060] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.485915 31578 raft_consensus.cc:2804] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.503856 31245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b9a979d5b8b4d02b133ffc4078d9cbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.511636 31484 raft_consensus.cc:2468] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:50.507959 31533 ts_tablet_manager.cc:1403] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.526s	user 0.011s	sys 0.015s
I20260709 13:10:50.498826 31541 leader_election.cc:290] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:50.501056 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914efa83b45e44dbbabeca35e75d0f3e" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:50.517663 31552 raft_consensus.cc:493] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.518946 31261 raft_consensus.cc:3060] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.518935 31552 raft_consensus.cc:515] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.522130 31552 leader_election.cc:290] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
W20260709 13:10:50.523891 31207 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:50.525058 31226 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
W20260709 13:10:50.518024 31228 leader_election.cc:343] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.526213 31539 raft_consensus.cc:2804] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.526611 31539 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.526949 31539 raft_consensus.cc:3060] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:50.532691 31451 leader_election.cc:343] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.536295 31261 raft_consensus.cc:2468] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:50.544740 31531 tablet_bootstrap.cc:492] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:50.545368 31531 ts_tablet_manager.cc:1403] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.139s	user 0.039s	sys 0.041s
I20260709 13:10:50.549480 31395 tablet_service.cc:1511] Processing CreateTablet for tablet f4aaf32f729d46578d080ffe9a172eda (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:50.551160 31395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4aaf32f729d46578d080ffe9a172eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.554909 31532 tablet_bootstrap.cc:654] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.559621 31530 tablet_bootstrap.cc:492] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:50.560133 31530 ts_tablet_manager.cc:1403] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.135s	user 0.009s	sys 0.002s
I20260709 13:10:50.562610 31533 raft_consensus.cc:359] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.563261 31533 raft_consensus.cc:385] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.563530 31533 raft_consensus.cc:740] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.564210 31533 consensus_queue.cc:260] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.593551 31377 leader_election.cc:304] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:50.593954 31473 tablet_service.cc:1511] Processing CreateTablet for tablet f4aaf32f729d46578d080ffe9a172eda (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:50.594513 31572 raft_consensus.cc:2804] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.595003 31572 raft_consensus.cc:697] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:50.595242 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4aaf32f729d46578d080ffe9a172eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.595997 31572 consensus_queue.cc:237] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.600363 31533 ts_tablet_manager.cc:1434] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.070s	user 0.004s	sys 0.002s
I20260709 13:10:50.600128 31396 tablet_service.cc:1511] Processing CreateTablet for tablet 92573de449ff4002a92b5ffcecfb8521 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:50.602739 31531 raft_consensus.cc:359] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.603627 31531 raft_consensus.cc:385] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.603761 31396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92573de449ff4002a92b5ffcecfb8521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.604060 31531 raft_consensus.cc:740] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.608814 31582 raft_consensus.cc:493] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.610509 31582 raft_consensus.cc:515] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.576522 31539 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.610849 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 92573de449ff4002a92b5ffcecfb8521 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:50.612236 31578 raft_consensus.cc:697] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:10:50.619042 31530 raft_consensus.cc:359] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.618899 31578 consensus_queue.cc:237] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:50.605087 31531 consensus_queue.cc:260] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.628123 31533 tablet_bootstrap.cc:492] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:50.629088 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92573de449ff4002a92b5ffcecfb8521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.629235 31531 ts_tablet_manager.cc:1434] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.001s
I20260709 13:10:50.630869 31531 tablet_bootstrap.cc:492] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:50.633992 31532 tablet_bootstrap.cc:492] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:50.634598 31532 ts_tablet_manager.cc:1403] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.167s	user 0.008s	sys 0.008s
I20260709 13:10:50.635888 31539 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:50.637521 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:50.622248 31530 raft_consensus.cc:385] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.644260 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:50.645053 31261 raft_consensus.cc:2468] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:50.647832 31337 raft_consensus.cc:2468] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:50.642802 31530 raft_consensus.cc:740] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.651273 31531 tablet_bootstrap.cc:654] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.652343 31530 consensus_queue.cc:260] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.670462 31327 tablet_service.cc:1511] Processing CreateTablet for tablet f4aaf32f729d46578d080ffe9a172eda (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:50.671213 31323 tablet_service.cc:1511] Processing CreateTablet for tablet 92573de449ff4002a92b5ffcecfb8521 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:50.671763 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4aaf32f729d46578d080ffe9a172eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.672742 31323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92573de449ff4002a92b5ffcecfb8521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.676541 31303 leader_election.cc:304] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:50.667119 31530 ts_tablet_manager.cc:1434] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.106s	user 0.009s	sys 0.003s
I20260709 13:10:50.665207 31540 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.682725 31552 raft_consensus.cc:2804] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.687549 31552 raft_consensus.cc:493] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.687934 31552 raft_consensus.cc:3060] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.688293 31409 raft_consensus.cc:2393] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 75b6f0017ce84a4c9822ffd27e053405 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:50.688673 31473 tablet_service.cc:1511] Processing CreateTablet for tablet 077ed73927a64748baae88b0b8dc4a13 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:50.689996 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 077ed73927a64748baae88b0b8dc4a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.693817 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:50.694592 31261 raft_consensus.cc:2393] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 75b6f0017ce84a4c9822ffd27e053405 in this term.
I20260709 13:10:50.694900 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:50.695492 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:50.694700 31540 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:50.697861 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
W20260709 13:10:50.699110 31225 leader_election.cc:343] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:50.702700 31374 leader_election.cc:343] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.703167 31374 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:50.702786 31225 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:50.704397 31540 raft_consensus.cc:2749] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.706431 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:50.707024 31334 raft_consensus.cc:2468] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
I20260709 13:10:50.701535 31582 leader_election.cc:290] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:50.708305 31582 raft_consensus.cc:2749] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.709136 31225 leader_election.cc:304] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:50.709977 31539 raft_consensus.cc:2804] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.710332 31539 raft_consensus.cc:493] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.710672 31539 raft_consensus.cc:3060] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.714425 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 077ed73927a64748baae88b0b8dc4a13 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:50.715747 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 077ed73927a64748baae88b0b8dc4a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.717721 31325 tablet_service.cc:1511] Processing CreateTablet for tablet 077ed73927a64748baae88b0b8dc4a13 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:50.718993 31325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 077ed73927a64748baae88b0b8dc4a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.720973 31469 tablet_service.cc:1511] Processing CreateTablet for tablet c90bfbd1ec9d458faadce4b9a625024b (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:50.721935 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:50.723780 31399 tablet_service.cc:1511] Processing CreateTablet for tablet c90bfbd1ec9d458faadce4b9a625024b (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:50.722290 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c90bfbd1ec9d458faadce4b9a625024b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.726387 31533 tablet_bootstrap.cc:654] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:50.723807 31225 leader_election.cc:343] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.731271 31245 tablet_service.cc:1511] Processing CreateTablet for tablet c90bfbd1ec9d458faadce4b9a625024b (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:50.731604 31532 raft_consensus.cc:359] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.732384 31532 raft_consensus.cc:385] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.732713 31532 raft_consensus.cc:740] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.732816 31245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c90bfbd1ec9d458faadce4b9a625024b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.734709 31260 raft_consensus.cc:2468] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:50.738958 31451 leader_election.cc:304] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:50.740195 31580 raft_consensus.cc:2804] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.740687 31580 raft_consensus.cc:697] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:10:50.741561 31580 consensus_queue.cc:237] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.758603 31530 tablet_bootstrap.cc:492] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:50.762143 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:50.762631 31399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c90bfbd1ec9d458faadce4b9a625024b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.741135 31552 raft_consensus.cc:515] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.763716 31336 raft_consensus.cc:2393] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75b6f0017ce84a4c9822ffd27e053405 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:50.765288 31552 leader_election.cc:290] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:50.765897 31552 raft_consensus.cc:493] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:50.766027 31226 leader_election.cc:343] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.766333 31552 raft_consensus.cc:515] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.764883 31158 catalog_manager.cc:5697] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:50.768972 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942702984f5b4342bdd5399274624f90" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:50.769604 31409 raft_consensus.cc:2468] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:50.770875 31301 leader_election.cc:304] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:50.783450 31531 tablet_bootstrap.cc:492] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:50.783978 31531 ts_tablet_manager.cc:1403] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.153s	user 0.009s	sys 0.027s
I20260709 13:10:50.780723 31532 consensus_queue.cc:260] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.772003 31225 leader_election.cc:304] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:50.786274 31582 raft_consensus.cc:3060] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.786530 31532 ts_tablet_manager.cc:1434] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.152s	user 0.005s	sys 0.002s
I20260709 13:10:50.787019 31531 raft_consensus.cc:359] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.787715 31532 tablet_bootstrap.cc:492] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:50.787761 31531 raft_consensus.cc:385] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.788211 31531 raft_consensus.cc:740] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.792703 31530 tablet_bootstrap.cc:654] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.797883 31539 raft_consensus.cc:515] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.800871 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:50.801458 31336 raft_consensus.cc:3060] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.809697 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:50.809899 31582 raft_consensus.cc:2749] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:50.811455 31226 leader_election.cc:343] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.813977 31469 tablet_service.cc:1511] Processing CreateTablet for tablet d4be50f8b2fe4ae4a303702831a3f006 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:50.815114 31401 tablet_service.cc:1511] Processing CreateTablet for tablet d4be50f8b2fe4ae4a303702831a3f006 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:50.815651 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4be50f8b2fe4ae4a303702831a3f006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.816375 31401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4be50f8b2fe4ae4a303702831a3f006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.817703 31336 raft_consensus.cc:2468] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:50.818890 31225 leader_election.cc:304] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:50.822536 31582 raft_consensus.cc:2804] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.805899 31539 leader_election.cc:290] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:50.833077 31589 raft_consensus.cc:2804] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.833199 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:50.833660 31589 raft_consensus.cc:493] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.834043 31589 raft_consensus.cc:3060] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.836079 31249 tablet_service.cc:1511] Processing CreateTablet for tablet 29510c165a1f48b1949c6ee31e136064 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:50.837445 31249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29510c165a1f48b1949c6ee31e136064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.841805 31260 raft_consensus.cc:2468] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:50.843264 31552 leader_election.cc:290] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:50.827486 31531 consensus_queue.cc:260] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:50.841818 31589 raft_consensus.cc:515] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.858336 31533 tablet_bootstrap.cc:492] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:50.859007 31533 ts_tablet_manager.cc:1403] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.237s	user 0.012s	sys 0.016s
I20260709 13:10:50.850932 31531 ts_tablet_manager.cc:1434] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.005s
I20260709 13:10:50.852279 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942702984f5b4342bdd5399274624f90" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:50.864616 31409 raft_consensus.cc:3060] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.864609 31531 tablet_bootstrap.cc:492] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:50.872066 31582 raft_consensus.cc:697] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:10:50.876536 31582 consensus_queue.cc:237] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:50.872195 31589 leader_election.cc:290] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:50.883291 31409 raft_consensus.cc:2468] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:50.886668 31301 leader_election.cc:304] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:50.887593 31589 raft_consensus.cc:2804] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.887717 31533 raft_consensus.cc:359] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.888332 31589 raft_consensus.cc:697] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:50.888741 31533 raft_consensus.cc:385] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.889391 31589 consensus_queue.cc:237] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:50.891515 31399 tablet_service.cc:1511] Processing CreateTablet for tablet 29510c165a1f48b1949c6ee31e136064 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:50.892771 31399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29510c165a1f48b1949c6ee31e136064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.893178 31473 tablet_service.cc:1511] Processing CreateTablet for tablet d913d7bf7d154005a431dc28e2bd91f6 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:50.894392 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d913d7bf7d154005a431dc28e2bd91f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.895066 31327 tablet_service.cc:1511] Processing CreateTablet for tablet d4be50f8b2fe4ae4a303702831a3f006 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:50.895390 31247 tablet_service.cc:1511] Processing CreateTablet for tablet d913d7bf7d154005a431dc28e2bd91f6 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:50.896528 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4be50f8b2fe4ae4a303702831a3f006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.896708 31247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d913d7bf7d154005a431dc28e2bd91f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.921196 31580 consensus_queue.cc:1048] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:50.889201 31533 raft_consensus.cc:740] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.939049 31532 tablet_bootstrap.cc:654] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.958525 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:50.959008 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:50.959940 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942702984f5b4342bdd5399274624f90" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:50.960618 31482 raft_consensus.cc:3060] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.960783 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "942702984f5b4342bdd5399274624f90" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:50.962641 31303 leader_election.cc:304] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:50.964323 31531 tablet_bootstrap.cc:654] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:50.964632 31303 leader_election.cc:343] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.964898 31325 tablet_service.cc:1511] Processing CreateTablet for tablet 29510c165a1f48b1949c6ee31e136064 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:50.965552 31552 raft_consensus.cc:2804] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.966414 31325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29510c165a1f48b1949c6ee31e136064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.938678 31580 consensus_queue.cc:1048] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:10:50.967376 31323 tablet_service.cc:1511] Processing CreateTablet for tablet d913d7bf7d154005a431dc28e2bd91f6 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:50.967574 31401 tablet_service.cc:1511] Processing CreateTablet for tablet d6498a8fe8ab41d19cfe4afd8ccfb904 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:50.977737 31401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6498a8fe8ab41d19cfe4afd8ccfb904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.979110 31323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d913d7bf7d154005a431dc28e2bd91f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.992452 31473 tablet_service.cc:1511] Processing CreateTablet for tablet d6498a8fe8ab41d19cfe4afd8ccfb904 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:50.992551 31471 tablet_service.cc:1511] Processing CreateTablet for tablet a283218fb40d45d0a877dcd23bfc51d5 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:50.986347 31395 tablet_service.cc:1511] Processing CreateTablet for tablet a283218fb40d45d0a877dcd23bfc51d5 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:50.993762 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6498a8fe8ab41d19cfe4afd8ccfb904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.994012 31395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a283218fb40d45d0a877dcd23bfc51d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.990675 31400 tablet_service.cc:1511] Processing CreateTablet for tablet f81f5f6441044395977dbda8239d7289 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:50.997997 31471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a283218fb40d45d0a877dcd23bfc51d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:50.966421 31159 catalog_manager.cc:5697] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:50.998059 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f81f5f6441044395977dbda8239d7289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.003572 31250 tablet_service.cc:1511] Processing CreateTablet for tablet d6498a8fe8ab41d19cfe4afd8ccfb904 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:50.968232 31533 consensus_queue.cc:260] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:51.005280 31250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6498a8fe8ab41d19cfe4afd8ccfb904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.008970 31533 ts_tablet_manager.cc:1434] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.150s	user 0.010s	sys 0.000s
I20260709 13:10:51.015923 31552 raft_consensus.cc:697] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:51.018570 31472 tablet_service.cc:1511] Processing CreateTablet for tablet 1d10f77e65c14720a8312b1dd44d3199 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:51.005303 31540 consensus_queue.cc:1048] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:10:51.024039 31472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d10f77e65c14720a8312b1dd44d3199. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:50.982326 31303 leader_election.cc:343] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.028119 31533 tablet_bootstrap.cc:492] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:51.032547 31530 tablet_bootstrap.cc:492] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:51.033102 31530 ts_tablet_manager.cc:1403] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.275s	user 0.031s	sys 0.062s
I20260709 13:10:51.033787 31582 raft_consensus.cc:493] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.034341 31582 raft_consensus.cc:515] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:51.036408 31530 raft_consensus.cc:359] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:51.038074 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:51.038551 31582 leader_election.cc:290] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:51.040254 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:51.006628 31248 tablet_service.cc:1511] Processing CreateTablet for tablet f81f5f6441044395977dbda8239d7289 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:51.046790 31248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f81f5f6441044395977dbda8239d7289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.045499 31530 raft_consensus.cc:385] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.050909 31530 raft_consensus.cc:740] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.051102 31396 tablet_service.cc:1511] Processing CreateTablet for tablet a6c9185858d64cfca85bfe148ea6bef0 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:51.052366 31396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c9185858d64cfca85bfe148ea6bef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.058194 31249 tablet_service.cc:1511] Processing CreateTablet for tablet 1d10f77e65c14720a8312b1dd44d3199 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:51.054872 31336 raft_consensus.cc:2468] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
W20260709 13:10:51.041474 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.68:44895, user_credentials={real_user=slave}} blocked reactor thread for 61066.8us
I20260709 13:10:51.078290 31225 leader_election.cc:304] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:51.079348 31582 raft_consensus.cc:2804] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.079756 31582 raft_consensus.cc:493] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.080119 31582 raft_consensus.cc:3060] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.051954 31530 consensus_queue.cc:260] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:51.089285 31249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d10f77e65c14720a8312b1dd44d3199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.092491 31410 raft_consensus.cc:2468] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
I20260709 13:10:51.085036 31580 raft_consensus.cc:493] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.126144 31247 tablet_service.cc:1511] Processing CreateTablet for tablet a6c9185858d64cfca85bfe148ea6bef0 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:51.127549 31247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c9185858d64cfca85bfe148ea6bef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.129501 31580 raft_consensus.cc:515] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:51.132121 31469 tablet_service.cc:1511] Processing CreateTablet for tablet a6c9185858d64cfca85bfe148ea6bef0 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:51.132710 31580 leader_election.cc:290] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:51.133387 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35faa1d23acb4a6da70dc14e16ed6a69" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:51.141980 31540 consensus_queue.cc:1048] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:51.142724 31448 leader_election.cc:343] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.133854 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c9185858d64cfca85bfe148ea6bef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.135562 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35faa1d23acb4a6da70dc14e16ed6a69" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
W20260709 13:10:51.144743 31449 leader_election.cc:343] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.145252 31449 leader_election.cc:304] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:51.147099 31397 tablet_service.cc:1511] Processing CreateTablet for tablet c5846d47686948bcb0d921291b4bc5c9 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:51.148566 31397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5846d47686948bcb0d921291b4bc5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.148383 31245 tablet_service.cc:1511] Processing CreateTablet for tablet e69c3326d8874749a268ae8dfdca3b3c (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:51.149696 31245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e69c3326d8874749a268ae8dfdca3b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.162048 31582 raft_consensus.cc:515] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:51.162175 31473 tablet_service.cc:1511] Processing CreateTablet for tablet c5846d47686948bcb0d921291b4bc5c9 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:51.163523 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5846d47686948bcb0d921291b4bc5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.069329 31552 consensus_queue.cc:237] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.163072 31589 raft_consensus.cc:493] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.163959 31580 raft_consensus.cc:2749] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.187386 31400 tablet_service.cc:1511] Processing CreateTablet for tablet e69c3326d8874749a268ae8dfdca3b3c (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:51.187896 31324 tablet_service.cc:1511] Processing CreateTablet for tablet f81f5f6441044395977dbda8239d7289 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
W20260709 13:10:51.188382 31358 debug-util.cc:398] Leaking SignalData structure 0x7b080025f5e0 after lost signal to thread 21632
I20260709 13:10:51.189208 31324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f81f5f6441044395977dbda8239d7289. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:51.189504 31358 debug-util.cc:398] Leaking SignalData structure 0x7b080026d460 after lost signal to thread 31194
I20260709 13:10:51.189126 31532 tablet_bootstrap.cc:492] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:51.195523 31532 ts_tablet_manager.cc:1403] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.408s	user 0.006s	sys 0.017s
I20260709 13:10:51.196367 31398 tablet_service.cc:1511] Processing CreateTablet for tablet 862d691f6d9e4d9fb185be53b562b8ff (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
W20260709 13:10:51.192559 31358 debug-util.cc:398] Leaking SignalData structure 0x7b08001316c0 after lost signal to thread 31435
I20260709 13:10:51.201073 31582 leader_election.cc:290] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:51.202651 31327 tablet_service.cc:1511] Processing CreateTablet for tablet a283218fb40d45d0a877dcd23bfc51d5 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:51.204514 31322 tablet_service.cc:1511] Processing CreateTablet for tablet 1d10f77e65c14720a8312b1dd44d3199 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:51.205266 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 1b5b2bc298df4947b958fefdc9c2846e (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:51.205825 31326 tablet_service.cc:1511] Processing CreateTablet for tablet c5846d47686948bcb0d921291b4bc5c9 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:51.206614 31475 tablet_service.cc:1511] Processing CreateTablet for tablet 1b5b2bc298df4947b958fefdc9c2846e (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:51.206848 31320 tablet_service.cc:1511] Processing CreateTablet for tablet 1b5b2bc298df4947b958fefdc9c2846e (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:51.207260 31321 tablet_service.cc:1511] Processing CreateTablet for tablet e69c3326d8874749a268ae8dfdca3b3c (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:51.201772 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:51.208482 31531 tablet_bootstrap.cc:492] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:51.262923 31337 raft_consensus.cc:3060] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.180336 31589 raft_consensus.cc:515] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.225538 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:51.262925 31475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b5b2bc298df4947b958fefdc9c2846e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.265933 31410 raft_consensus.cc:3060] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.266937 31589 leader_election.cc:290] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:51.267501 31334 raft_consensus.cc:3060] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.231727 31532 raft_consensus.cc:359] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:51.269122 31532 raft_consensus.cc:385] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.269471 31532 raft_consensus.cc:740] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.270243 31532 consensus_queue.cc:260] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:51.270802 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db13a06e22e411ba4c29d105c7e93f7" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:51.272593 31532 ts_tablet_manager.cc:1434] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.000s
I20260709 13:10:51.234377 31471 tablet_service.cc:1511] Processing CreateTablet for tablet 862d691f6d9e4d9fb185be53b562b8ff (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:51.240415 31246 tablet_service.cc:1511] Processing CreateTablet for tablet 862d691f6d9e4d9fb185be53b562b8ff (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:51.249389 31398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 862d691f6d9e4d9fb185be53b562b8ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.252820 31530 ts_tablet_manager.cc:1434] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.219s	user 0.006s	sys 0.004s
I20260709 13:10:51.182484 31482 raft_consensus.cc:2468] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:51.253907 31533 tablet_bootstrap.cc:654] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.253226 31160 catalog_manager.cc:5697] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 1 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.258823 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a283218fb40d45d0a877dcd23bfc51d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.263288 31531 ts_tablet_manager.cc:1403] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.399s	user 0.011s	sys 0.007s
I20260709 13:10:51.275786 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b5b2bc298df4947b958fefdc9c2846e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.298699 31326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5846d47686948bcb0d921291b4bc5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.304085 31321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e69c3326d8874749a268ae8dfdca3b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.305505 31410 raft_consensus.cc:2468] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:51.307137 31226 leader_election.cc:304] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:51.308203 31320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b5b2bc298df4947b958fefdc9c2846e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.307893 31396 tablet_service.cc:1511] Processing CreateTablet for tablet f564510ff30e42ec890531c82405946c (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:51.309203 31582 raft_consensus.cc:2804] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.309943 31582 raft_consensus.cc:697] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:10:51.310837 31582 consensus_queue.cc:237] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:51.312058 31531 raft_consensus.cc:359] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:51.312800 31531 raft_consensus.cc:385] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.313621 31322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d10f77e65c14720a8312b1dd44d3199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.318904 31607 consensus_queue.cc:1048] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:51.321642 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e69c3326d8874749a268ae8dfdca3b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.299530 31160 catalog_manager.cc:5697] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 1 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:51.329114 31530 tablet_bootstrap.cc:492] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:51.330305 31472 tablet_service.cc:1511] Processing CreateTablet for tablet f564510ff30e42ec890531c82405946c (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:51.324635 31532 tablet_bootstrap.cc:492] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:51.313091 31531 raft_consensus.cc:740] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.332278 31531 consensus_queue.cc:260] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:51.333791 31578 consensus_queue.cc:1048] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:51.336292 31260 raft_consensus.cc:3060] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.336237 31578 consensus_queue.cc:1048] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:51.337553 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db13a06e22e411ba4c29d105c7e93f7" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:51.337574 31396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f564510ff30e42ec890531c82405946c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.356701 31303 leader_election.cc:304] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:51.359337 31552 raft_consensus.cc:2749] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.357757 31246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 862d691f6d9e4d9fb185be53b562b8ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.360743 31471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 862d691f6d9e4d9fb185be53b562b8ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.273247 31337 raft_consensus.cc:2468] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:51.360749 31250 tablet_service.cc:1511] Processing CreateTablet for tablet 9b3eb1a472a546fdb51a2cc295716492 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:51.372834 31250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3eb1a472a546fdb51a2cc295716492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.373409 31397 tablet_service.cc:1511] Processing CreateTablet for tablet 9b3eb1a472a546fdb51a2cc295716492 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:51.375346 31323 tablet_service.cc:1511] Processing CreateTablet for tablet f564510ff30e42ec890531c82405946c (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:51.376638 31323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f564510ff30e42ec890531c82405946c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.372476 31324 tablet_service.cc:1511] Processing CreateTablet for tablet 9b3eb1a472a546fdb51a2cc295716492 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:51.392783 31472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f564510ff30e42ec890531c82405946c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.405927 31324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3eb1a472a546fdb51a2cc295716492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.415225 31325 tablet_service.cc:1511] Processing CreateTablet for tablet f9baba29a028498288a27243e555fb27 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:51.416553 31325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9baba29a028498288a27243e555fb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.420351 31248 tablet_service.cc:1511] Processing CreateTablet for tablet f9baba29a028498288a27243e555fb27 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:51.399010 31397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3eb1a472a546fdb51a2cc295716492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.421572 31248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9baba29a028498288a27243e555fb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.426124 31532 tablet_bootstrap.cc:654] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.431377 31469 tablet_service.cc:1511] Processing CreateTablet for tablet f9baba29a028498288a27243e555fb27 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:51.432592 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9baba29a028498288a27243e555fb27. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:51.391625 31374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39075, user_credentials={real_user=slave}} blocked reactor thread for 72326.7us
I20260709 13:10:51.450887 31607 consensus_queue.cc:1048] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:51.454212 31531 ts_tablet_manager.cc:1434] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.158s	user 0.007s	sys 0.000s
I20260709 13:10:51.455324 31531 tablet_bootstrap.cc:492] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:51.471216 31398 tablet_service.cc:1511] Processing CreateTablet for tablet 271f7357440943008e55795b0d6cbbda (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:51.472904 31398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271f7357440943008e55795b0d6cbbda. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:51.481756 31226 consensus_peers.cc:597] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:51.481884 31592 consensus_queue.cc:1048] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:51.513149 31552 consensus_queue.cc:1048] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:51.531790 31469 tablet_service.cc:1511] Processing CreateTablet for tablet 271f7357440943008e55795b0d6cbbda (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:51.532955 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271f7357440943008e55795b0d6cbbda. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:51.536900 31207 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:51.547116 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 271f7357440943008e55795b0d6cbbda (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:51.547690 31250 tablet_service.cc:1511] Processing CreateTablet for tablet c415b45c0cf0456bae70f23527c3488e (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:51.548291 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271f7357440943008e55795b0d6cbbda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.548821 31250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c415b45c0cf0456bae70f23527c3488e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.540907 31327 tablet_service.cc:1511] Processing CreateTablet for tablet e0bc6ebfdfef4b7a81d4cbc17a86d70f (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:51.550063 31475 tablet_service.cc:1511] Processing CreateTablet for tablet e0bc6ebfdfef4b7a81d4cbc17a86d70f (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:51.550168 31320 tablet_service.cc:1511] Processing CreateTablet for tablet c415b45c0cf0456bae70f23527c3488e (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:51.541594 31530 tablet_bootstrap.cc:654] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.543143 31400 tablet_service.cc:1511] Processing CreateTablet for tablet e0bc6ebfdfef4b7a81d4cbc17a86d70f (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:51.562291 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bc6ebfdfef4b7a81d4cbc17a86d70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.551265 31475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bc6ebfdfef4b7a81d4cbc17a86d70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.563898 31327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bc6ebfdfef4b7a81d4cbc17a86d70f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.565452 31531 tablet_bootstrap.cc:654] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.573864 31552 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.574338 31552 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:51.576397 31552 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:51.577112 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:51.577884 31260 raft_consensus.cc:2393] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07b814fcedea4142ac09385b680cae3a in current term 1: Already voted for candidate 75b6f0017ce84a4c9822ffd27e053405 in this term.
I20260709 13:10:51.578068 31473 tablet_service.cc:1511] Processing CreateTablet for tablet d905b9b08a224fe09df30744558819e3 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:51.587149 31320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c415b45c0cf0456bae70f23527c3488e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.586934 31552 consensus_queue.cc:1048] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:51.589869 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:51.588845 31251 tablet_service.cc:1511] Processing CreateTablet for tablet d905b9b08a224fe09df30744558819e3 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:51.590626 31409 raft_consensus.cc:2393] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07b814fcedea4142ac09385b680cae3a in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:51.590116 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d905b9b08a224fe09df30744558819e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.594324 31324 tablet_service.cc:1511] Processing CreateTablet for tablet d905b9b08a224fe09df30744558819e3 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:51.595494 31324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d905b9b08a224fe09df30744558819e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.598304 31301 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:51.591158 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d905b9b08a224fe09df30744558819e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.630463 31159 catalog_manager.cc:5697] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 1 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } }
I20260709 13:10:51.617564 31552 raft_consensus.cc:3060] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.639639 31398 tablet_service.cc:1511] Processing CreateTablet for tablet c415b45c0cf0456bae70f23527c3488e (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:51.640941 31398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c415b45c0cf0456bae70f23527c3488e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.645066 31532 tablet_bootstrap.cc:492] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:51.645690 31532 ts_tablet_manager.cc:1403] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.352s	user 0.030s	sys 0.020s
I20260709 13:10:51.656335 31533 tablet_bootstrap.cc:492] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:51.657060 31533 ts_tablet_manager.cc:1403] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.629s	user 0.018s	sys 0.018s
I20260709 13:10:51.632256 31159 catalog_manager.cc:5697] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.648512 31532 raft_consensus.cc:359] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.662164 31532 raft_consensus.cc:385] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.662554 31532 raft_consensus.cc:740] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.669976 31532 consensus_queue.cc:260] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.679131 31552 raft_consensus.cc:2749] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.682551 31532 ts_tablet_manager.cc:1434] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.003s
I20260709 13:10:51.682469 31473 tablet_service.cc:1511] Processing CreateTablet for tablet 0ad34ee917be4455bbe71da15c523cee (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:51.683530 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad34ee917be4455bbe71da15c523cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.694561 31400 tablet_service.cc:1511] Processing CreateTablet for tablet 0ad34ee917be4455bbe71da15c523cee (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:51.695827 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad34ee917be4455bbe71da15c523cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.706413 31159 catalog_manager.cc:5697] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:51.707717 31323 tablet_service.cc:1511] Processing CreateTablet for tablet f213e6b4ef544937bbd9c8e49c672693 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:51.709144 31323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f213e6b4ef544937bbd9c8e49c672693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.711793 31532 tablet_bootstrap.cc:492] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:51.720690 31582 consensus_queue.cc:1048] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:51.721521 31533 raft_consensus.cc:359] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.722414 31533 raft_consensus.cc:385] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.722716 31533 raft_consensus.cc:740] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.723624 31533 consensus_queue.cc:260] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.733721 31396 tablet_service.cc:1511] Processing CreateTablet for tablet f213e6b4ef544937bbd9c8e49c672693 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:51.736019 31396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f213e6b4ef544937bbd9c8e49c672693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.739347 31473 tablet_service.cc:1511] Processing CreateTablet for tablet f213e6b4ef544937bbd9c8e49c672693 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:51.740556 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f213e6b4ef544937bbd9c8e49c672693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.740648 31607 raft_consensus.cc:493] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.741416 31607 raft_consensus.cc:515] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.745190 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:51.748162 31320 tablet_service.cc:1511] Processing CreateTablet for tablet 7c760b7263fd41fea6c0a6dfd9dc5d85 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:51.749231 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:51.749987 31607 leader_election.cc:290] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:51.757565 31261 raft_consensus.cc:2393] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:51.759047 31320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c760b7263fd41fea6c0a6dfd9dc5d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.759114 31451 leader_election.cc:304] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:51.760684 31607 raft_consensus.cc:3060] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.771637 31530 tablet_bootstrap.cc:492] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:51.772158 31530 ts_tablet_manager.cc:1403] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.443s	user 0.023s	sys 0.022s
I20260709 13:10:51.772614 31589 consensus_queue.cc:1048] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:51.775322 31530 raft_consensus.cc:359] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:51.776036 31530 raft_consensus.cc:385] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.776346 31530 raft_consensus.cc:740] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.777209 31530 consensus_queue.cc:260] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:51.778715 31532 tablet_bootstrap.cc:654] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.780063 31530 ts_tablet_manager.cc:1434] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:10:51.781417 31530 tablet_bootstrap.cc:492] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:51.781486 31607 raft_consensus.cc:2749] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.808400 31589 consensus_queue.cc:1048] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:51.809993 31156 catalog_manager.cc:5697] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.814344 31248 tablet_service.cc:1511] Processing CreateTablet for tablet 0ad34ee917be4455bbe71da15c523cee (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:51.815097 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 7c760b7263fd41fea6c0a6dfd9dc5d85 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:51.815616 31248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad34ee917be4455bbe71da15c523cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.816521 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c760b7263fd41fea6c0a6dfd9dc5d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.818661 31582 consensus_queue.cc:1048] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:51.827360 31400 tablet_service.cc:1511] Processing CreateTablet for tablet 7c760b7263fd41fea6c0a6dfd9dc5d85 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:51.828524 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c760b7263fd41fea6c0a6dfd9dc5d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.829501 31533 ts_tablet_manager.cc:1434] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.169s	user 0.008s	sys 0.000s
I20260709 13:10:51.832959 31533 tablet_bootstrap.cc:492] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:51.835124 31530 tablet_bootstrap.cc:654] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.849330 31531 tablet_bootstrap.cc:492] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:51.849819 31531 ts_tablet_manager.cc:1403] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.395s	user 0.018s	sys 0.012s
I20260709 13:10:51.864102 31626 raft_consensus.cc:493] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.872241 31473 tablet_service.cc:1511] Processing CreateTablet for tablet d23babc199fb49fe9e6423834c97d1f7 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:51.873736 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d23babc199fb49fe9e6423834c97d1f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.876331 31533 tablet_bootstrap.cc:654] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.879873 31626 raft_consensus.cc:515] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.882191 31626 leader_election.cc:290] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:51.896581 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:51.899459 31228 leader_election.cc:343] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.907223 31531 raft_consensus.cc:359] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:51.907825 31531 raft_consensus.cc:385] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.908056 31531 raft_consensus.cc:740] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.911079 31531 consensus_queue.cc:260] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:51.920238 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:51.921707 31251 tablet_service.cc:1511] Processing CreateTablet for tablet d23babc199fb49fe9e6423834c97d1f7 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:10:51.921877 31225 leader_election.cc:343] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.922225 31532 tablet_bootstrap.cc:492] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:51.919828 31323 tablet_service.cc:1511] Processing CreateTablet for tablet d23babc199fb49fe9e6423834c97d1f7 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:51.922495 31225 leader_election.cc:304] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:51.922693 31532 ts_tablet_manager.cc:1403] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.213s	user 0.013s	sys 0.001s
I20260709 13:10:51.922851 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d23babc199fb49fe9e6423834c97d1f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.923403 31323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d23babc199fb49fe9e6423834c97d1f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.925518 31626 raft_consensus.cc:2749] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.925549 31532 raft_consensus.cc:359] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.926445 31532 raft_consensus.cc:385] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.926734 31532 raft_consensus.cc:740] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.927371 31532 consensus_queue.cc:260] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:51.929615 31532 ts_tablet_manager.cc:1434] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:51.944540 31400 tablet_service.cc:1511] Processing CreateTablet for tablet 9725e6248e7b4e67b23a3b99967003bf (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:51.946826 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9725e6248e7b4e67b23a3b99967003bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:51.952347 31532 tablet_bootstrap.cc:492] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:51.966144 31589 raft_consensus.cc:493] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.966658 31589 raft_consensus.cc:515] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:51.968889 31589 leader_election.cc:290] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:51.975720 31532 tablet_bootstrap.cc:654] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.971010 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35faa1d23acb4a6da70dc14e16ed6a69" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:51.991333 31531 ts_tablet_manager.cc:1434] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.141s	user 0.020s	sys 0.018s
I20260709 13:10:51.989243 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35faa1d23acb4a6da70dc14e16ed6a69" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:51.994400 31411 raft_consensus.cc:2468] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:52.000108 31396 tablet_service.cc:1511] Processing CreateTablet for tablet bbfb2b2292c140bbaa93438cce1746cb (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:51.990432 31485 raft_consensus.cc:2468] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:52.001422 31396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfb2b2292c140bbaa93438cce1746cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.001852 31324 tablet_service.cc:1511] Processing CreateTablet for tablet bbfb2b2292c140bbaa93438cce1746cb (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:52.002418 31303 leader_election.cc:304] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:52.003582 31324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfb2b2292c140bbaa93438cce1746cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.011041 31473 tablet_service.cc:1511] Processing CreateTablet for tablet 9725e6248e7b4e67b23a3b99967003bf (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:52.013598 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9725e6248e7b4e67b23a3b99967003bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.023191 31475 tablet_service.cc:1511] Processing CreateTablet for tablet bbfb2b2292c140bbaa93438cce1746cb (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:52.024430 31475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfb2b2292c140bbaa93438cce1746cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.025055 31631 raft_consensus.cc:2804] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.025466 31631 raft_consensus.cc:493] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.025803 31631 raft_consensus.cc:3060] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.027251 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 9725e6248e7b4e67b23a3b99967003bf (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:52.028640 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9725e6248e7b4e67b23a3b99967003bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.036270 31248 tablet_service.cc:1511] Processing CreateTablet for tablet 229af634ffc1437fb9dea4a95652ab5b (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:52.038096 31531 tablet_bootstrap.cc:492] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:52.045907 31530 tablet_bootstrap.cc:492] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:52.046551 31530 ts_tablet_manager.cc:1403] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.265s	user 0.015s	sys 0.012s
I20260709 13:10:52.048779 31533 tablet_bootstrap.cc:492] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:52.049288 31533 ts_tablet_manager.cc:1403] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.217s	user 0.009s	sys 0.019s
I20260709 13:10:52.049633 31320 tablet_service.cc:1511] Processing CreateTablet for tablet 229af634ffc1437fb9dea4a95652ab5b (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:52.051100 31320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 229af634ffc1437fb9dea4a95652ab5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.051915 31631 raft_consensus.cc:515] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.052510 31533 raft_consensus.cc:359] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.054111 31631 leader_election.cc:290] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:52.054930 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35faa1d23acb4a6da70dc14e16ed6a69" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:52.055197 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35faa1d23acb4a6da70dc14e16ed6a69" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:52.055512 31411 raft_consensus.cc:3060] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.055732 31485 raft_consensus.cc:3060] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.065691 31411 raft_consensus.cc:2468] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:52.067183 31485 raft_consensus.cc:2468] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:52.053179 31533 raft_consensus.cc:385] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.079751 31533 raft_consensus.cc:740] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.072254 31531 tablet_bootstrap.cc:654] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.074067 31469 tablet_service.cc:1511] Processing CreateTablet for tablet 8e1eac6b3d8a47b59679e4aee4293326 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:52.080921 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e1eac6b3d8a47b59679e4aee4293326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.080777 31400 tablet_service.cc:1511] Processing CreateTablet for tablet 229af634ffc1437fb9dea4a95652ab5b (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:52.081761 31248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 229af634ffc1437fb9dea4a95652ab5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.083149 31533 consensus_queue.cc:260] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.086659 31532 tablet_bootstrap.cc:492] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:52.094882 31533 ts_tablet_manager.cc:1434] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.045s	user 0.003s	sys 0.002s
I20260709 13:10:52.095158 31532 ts_tablet_manager.cc:1403] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.165s	user 0.027s	sys 0.022s
I20260709 13:10:52.095836 31533 tablet_bootstrap.cc:492] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:52.095570 31250 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb1e844232d4ed0b8752e51e31204b8 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:52.096808 31250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb1e844232d4ed0b8752e51e31204b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.088524 31323 tablet_service.cc:1511] Processing CreateTablet for tablet 8e1eac6b3d8a47b59679e4aee4293326 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:52.097767 31532 raft_consensus.cc:359] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.098433 31532 raft_consensus.cc:385] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.098538 31323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e1eac6b3d8a47b59679e4aee4293326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.098753 31532 raft_consensus.cc:740] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.099522 31532 consensus_queue.cc:260] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.089766 31303 leader_election.cc:304] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:52.101372 31533 tablet_bootstrap.cc:654] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.092549 31398 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb1e844232d4ed0b8752e51e31204b8 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:52.107023 31626 raft_consensus.cc:493] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.107470 31626 raft_consensus.cc:515] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.109745 31530 raft_consensus.cc:359] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.110613 31530 raft_consensus.cc:385] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.110936 31530 raft_consensus.cc:740] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.111465 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 229af634ffc1437fb9dea4a95652ab5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.111527 31530 consensus_queue.cc:260] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.113543 31398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb1e844232d4ed0b8752e51e31204b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.117717 31582 raft_consensus.cc:493] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.118193 31582 raft_consensus.cc:515] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.120220 31582 leader_election.cc:290] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:52.122555 31637 raft_consensus.cc:2804] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.122639 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa7546f06a454f3899cb0bf7d0c0d084" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
W20260709 13:10:52.125293 31225 leader_election.cc:343] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.126586 31626 leader_election.cc:290] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:52.127197 31320 tablet_service.cc:1511] Processing CreateTablet for tablet 1dfef8e9e4964ab3b8b7c10e84a9d293 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:52.128345 31320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfef8e9e4964ab3b8b7c10e84a9d293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.128178 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24fe066d118d48c5939576417491e9ec" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:52.129601 31228 leader_election.cc:343] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.124953 31251 tablet_service.cc:1511] Processing CreateTablet for tablet 8e1eac6b3d8a47b59679e4aee4293326 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:52.131335 31532 ts_tablet_manager.cc:1434] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:10:52.132064 31531 tablet_bootstrap.cc:492] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:52.132675 31531 ts_tablet_manager.cc:1403] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.095s	user 0.003s	sys 0.011s
I20260709 13:10:52.134801 31615 raft_consensus.cc:493] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.135259 31615 raft_consensus.cc:515] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.136373 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa7546f06a454f3899cb0bf7d0c0d084" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:52.136504 31531 raft_consensus.cc:359] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.137229 31531 raft_consensus.cc:385] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.159689 31531 raft_consensus.cc:740] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.140552 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24fe066d118d48c5939576417491e9ec" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:52.160806 31531 consensus_queue.cc:260] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
W20260709 13:10:52.161880 31226 leader_election.cc:343] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.162395 31226 leader_election.cc:304] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:52.163674 31592 raft_consensus.cc:2749] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.147450 31473 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb1e844232d4ed0b8752e51e31204b8 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
W20260709 13:10:52.137928 31228 leader_election.cc:343] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.165752 31228 leader_election.cc:304] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:52.166224 31324 tablet_service.cc:1511] Processing CreateTablet for tablet 77fb3ec3412341df88b759485f90f108 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:52.167466 31324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fb3ec3412341df88b759485f90f108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.167977 31640 raft_consensus.cc:493] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.168138 31475 tablet_service.cc:1511] Processing CreateTablet for tablet 1dfef8e9e4964ab3b8b7c10e84a9d293 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:52.168606 31531 ts_tablet_manager.cc:1434] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 13:10:52.168475 31640 raft_consensus.cc:515] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.170738 31640 leader_election.cc:290] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:52.171196 31246 tablet_service.cc:1511] Processing CreateTablet for tablet 77fb3ec3412341df88b759485f90f108 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:52.172851 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:52.169701 31531 tablet_bootstrap.cc:492] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:52.173561 31485 raft_consensus.cc:2468] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:52.174400 31396 tablet_service.cc:1511] Processing CreateTablet for tablet 77fb3ec3412341df88b759485f90f108 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:52.174836 31303 leader_election.cc:304] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:52.175624 31396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fb3ec3412341df88b759485f90f108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.175774 31640 raft_consensus.cc:2804] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.176136 31640 raft_consensus.cc:493] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.176448 31640 raft_consensus.cc:3060] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.139899 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:52.153360 31530 ts_tablet_manager.cc:1434] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.107s	user 0.007s	sys 0.001s
I20260709 13:10:52.179376 31530 tablet_bootstrap.cc:492] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:52.181882 31245 tablet_service.cc:1511] Processing CreateTablet for tablet d9645a5374ed42a79a83e6af2d395bcc (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:52.123149 31637 raft_consensus.cc:697] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:52.137179 31615 leader_election.cc:290] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:52.167958 31592 raft_consensus.cc:2749] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.186864 31325 tablet_service.cc:1511] Processing CreateTablet for tablet d9645a5374ed42a79a83e6af2d395bcc (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:52.187156 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:52.187801 31260 raft_consensus.cc:2468] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:52.188097 31325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9645a5374ed42a79a83e6af2d395bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.189415 31246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fb3ec3412341df88b759485f90f108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.186887 31637 consensus_queue.cc:237] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.192869 31251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e1eac6b3d8a47b59679e4aee4293326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.187918 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:52.193397 31245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9645a5374ed42a79a83e6af2d395bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.199621 31473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb1e844232d4ed0b8752e51e31204b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.202677 31475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfef8e9e4964ab3b8b7c10e84a9d293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.202733 31469 tablet_service.cc:1511] Processing CreateTablet for tablet d9645a5374ed42a79a83e6af2d395bcc (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:52.204313 31377 leader_election.cc:304] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:52.205367 31615 raft_consensus.cc:2749] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.183822 31640 raft_consensus.cc:515] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.213681 31640 leader_election.cc:290] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:52.220458 31532 tablet_bootstrap.cc:492] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:52.233383 31400 tablet_service.cc:1511] Processing CreateTablet for tablet 1dfef8e9e4964ab3b8b7c10e84a9d293 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:52.234716 31400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfef8e9e4964ab3b8b7c10e84a9d293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.235520 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:52.236107 31485 raft_consensus.cc:3060] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.243233 31531 tablet_bootstrap.cc:654] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.244238 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:52.244462 31485 raft_consensus.cc:2468] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:52.244906 31261 raft_consensus.cc:3060] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.251014 31530 tablet_bootstrap.cc:654] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.253998 31303 leader_election.cc:304] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:52.260722 31397 tablet_service.cc:1511] Processing CreateTablet for tablet cfee9be6a99740c4ab0759431ad94310 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:52.261947 31397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfee9be6a99740c4ab0759431ad94310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.265926 31533 tablet_bootstrap.cc:492] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:52.266397 31533 ts_tablet_manager.cc:1403] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.171s	user 0.021s	sys 0.001s
I20260709 13:10:52.267797 31637 raft_consensus.cc:2804] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.268325 31637 raft_consensus.cc:697] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:52.269761 31469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9645a5374ed42a79a83e6af2d395bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.269975 31530 tablet_bootstrap.cc:492] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:52.270468 31530 ts_tablet_manager.cc:1403] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.091s	user 0.012s	sys 0.000s
I20260709 13:10:52.269095 31533 raft_consensus.cc:359] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.274161 31533 raft_consensus.cc:385] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.274495 31533 raft_consensus.cc:740] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.275507 31533 consensus_queue.cc:260] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.253000 31261 raft_consensus.cc:2468] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:52.278146 31533 ts_tablet_manager.cc:1434] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:10:52.280030 31530 raft_consensus.cc:359] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.280709 31530 raft_consensus.cc:385] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.281113 31530 raft_consensus.cc:740] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.281764 31530 consensus_queue.cc:260] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.281953 31472 tablet_service.cc:1511] Processing CreateTablet for tablet cfee9be6a99740c4ab0759431ad94310 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:52.283468 31248 tablet_service.cc:1511] Processing CreateTablet for tablet cfee9be6a99740c4ab0759431ad94310 (DEFAULT_TABLE table=test-workload [id=e482d805368c498fa4d0d46ae8539652]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:52.284042 31530 ts_tablet_manager.cc:1434] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:10:52.284782 31248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfee9be6a99740c4ab0759431ad94310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.285120 31530 tablet_bootstrap.cc:492] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:52.264744 31157 catalog_manager.cc:5697] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.269130 31637 consensus_queue.cc:237] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.297575 31532 tablet_bootstrap.cc:654] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.299817 31530 tablet_bootstrap.cc:654] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.301945 31472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfee9be6a99740c4ab0759431ad94310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:52.326467 31533 tablet_bootstrap.cc:492] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:52.335446 31531 tablet_bootstrap.cc:492] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:52.335944 31531 ts_tablet_manager.cc:1403] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.166s	user 0.009s	sys 0.010s
I20260709 13:10:52.331220 31157 catalog_manager.cc:5697] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:52.338672 31531 raft_consensus.cc:359] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.339349 31531 raft_consensus.cc:385] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.339605 31531 raft_consensus.cc:740] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.340215 31531 consensus_queue.cc:260] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.340893 31532 tablet_bootstrap.cc:492] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:52.341367 31532 ts_tablet_manager.cc:1403] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.121s	user 0.007s	sys 0.006s
I20260709 13:10:52.340215 31530 tablet_bootstrap.cc:492] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:52.345077 31530 ts_tablet_manager.cc:1403] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.060s	user 0.007s	sys 0.011s
I20260709 13:10:52.348022 31531 ts_tablet_manager.cc:1434] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:10:52.349022 31531 tablet_bootstrap.cc:492] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:52.355293 31531 tablet_bootstrap.cc:654] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.359652 31533 tablet_bootstrap.cc:654] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.366221 31533 tablet_bootstrap.cc:492] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:52.366659 31533 ts_tablet_manager.cc:1403] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 13:10:52.368947 31530 raft_consensus.cc:359] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.369331 31533 raft_consensus.cc:359] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.369973 31533 raft_consensus.cc:385] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.348659 31532 raft_consensus.cc:359] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.370266 31533 raft_consensus.cc:740] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.370512 31532 raft_consensus.cc:385] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.370826 31532 raft_consensus.cc:740] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.369678 31530 raft_consensus.cc:385] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.371912 31532 consensus_queue.cc:260] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.371495 31530 raft_consensus.cc:740] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.371945 31533 consensus_queue.cc:260] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.379845 31530 consensus_queue.cc:260] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.381211 31531 tablet_bootstrap.cc:492] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:52.381929 31531 ts_tablet_manager.cc:1403] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.005s
I20260709 13:10:52.384016 31532 ts_tablet_manager.cc:1434] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.042s	user 0.002s	sys 0.004s
I20260709 13:10:52.385288 31532 tablet_bootstrap.cc:492] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:52.387614 31533 ts_tablet_manager.cc:1434] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:10:52.388602 31533 tablet_bootstrap.cc:492] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:52.388357 31531 raft_consensus.cc:359] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.389070 31531 raft_consensus.cc:385] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.389379 31531 raft_consensus.cc:740] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.390062 31531 consensus_queue.cc:260] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.392010 31531 ts_tablet_manager.cc:1434] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:10:52.392869 31531 tablet_bootstrap.cc:492] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:52.397004 31530 ts_tablet_manager.cc:1434] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 13:10:52.398046 31530 tablet_bootstrap.cc:492] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:52.402385 31532 tablet_bootstrap.cc:654] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.406451 31531 tablet_bootstrap.cc:654] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.407975 31533 tablet_bootstrap.cc:654] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.410355 31530 tablet_bootstrap.cc:654] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.420925 31411 raft_consensus.cc:3060] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.441691 31532 tablet_bootstrap.cc:492] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:52.442390 31532 ts_tablet_manager.cc:1403] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.018s
I20260709 13:10:52.446919 31532 raft_consensus.cc:359] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.449579 31532 raft_consensus.cc:385] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.449928 31532 raft_consensus.cc:740] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.450776 31532 consensus_queue.cc:260] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.455696 31532 ts_tablet_manager.cc:1434] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:10:52.458700 31530 tablet_bootstrap.cc:492] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:52.458807 31531 tablet_bootstrap.cc:492] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:52.459633 31531 ts_tablet_manager.cc:1403] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.067s	user 0.008s	sys 0.009s
I20260709 13:10:52.462164 31532 tablet_bootstrap.cc:492] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:52.462482 31531 raft_consensus.cc:359] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.463200 31531 raft_consensus.cc:385] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.463523 31531 raft_consensus.cc:740] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.464320 31531 consensus_queue.cc:260] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.466591 31531 ts_tablet_manager.cc:1434] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:10:52.462397 31530 ts_tablet_manager.cc:1403] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.009s
I20260709 13:10:52.468498 31532 tablet_bootstrap.cc:654] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.469604 31533 tablet_bootstrap.cc:492] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:52.470046 31533 ts_tablet_manager.cc:1403] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.021s
I20260709 13:10:52.470954 31530 raft_consensus.cc:359] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.471652 31530 raft_consensus.cc:385] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.471923 31530 raft_consensus.cc:740] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.472549 31533 raft_consensus.cc:359] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.473217 31533 raft_consensus.cc:385] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.472577 31530 consensus_queue.cc:260] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.473520 31533 raft_consensus.cc:740] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.474220 31533 consensus_queue.cc:260] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.475397 31530 ts_tablet_manager.cc:1434] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:52.476168 31533 ts_tablet_manager.cc:1434] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:52.476545 31530 tablet_bootstrap.cc:492] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:52.477185 31533 tablet_bootstrap.cc:492] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:52.480585 31607 raft_consensus.cc:493] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.481077 31607 raft_consensus.cc:515] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.482688 31530 tablet_bootstrap.cc:654] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.483093 31607 leader_election.cc:290] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:52.484687 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa7546f06a454f3899cb0bf7d0c0d084" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:52.485383 31261 raft_consensus.cc:2468] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:52.486241 31533 tablet_bootstrap.cc:654] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.488991 31451 leader_election.cc:304] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:52.490931 31532 tablet_bootstrap.cc:492] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:52.491351 31532 ts_tablet_manager.cc:1403] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.000s
I20260709 13:10:52.493960 31607 raft_consensus.cc:2804] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.493880 31532 raft_consensus.cc:359] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.494369 31607 raft_consensus.cc:493] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.494557 31532 raft_consensus.cc:385] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.494769 31607 raft_consensus.cc:3060] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.494874 31532 raft_consensus.cc:740] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.502125 31532 consensus_queue.cc:260] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.504532 31532 ts_tablet_manager.cc:1434] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:10:52.505574 31532 tablet_bootstrap.cc:492] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:52.496680 31530 tablet_bootstrap.cc:492] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:52.506867 31530 ts_tablet_manager.cc:1403] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.003s
I20260709 13:10:52.509707 31530 raft_consensus.cc:359] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.510370 31530 raft_consensus.cc:385] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.510630 31530 raft_consensus.cc:740] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.511267 31530 consensus_queue.cc:260] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.519820 31532 tablet_bootstrap.cc:654] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.521764 31531 tablet_bootstrap.cc:492] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:52.523629 31607 raft_consensus.cc:515] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.528321 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa7546f06a454f3899cb0bf7d0c0d084" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:52.528712 31607 leader_election.cc:290] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:52.528990 31261 raft_consensus.cc:3060] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.529304 31607 raft_consensus.cc:493] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.530128 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa7546f06a454f3899cb0bf7d0c0d084" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:52.521034 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa7546f06a454f3899cb0bf7d0c0d084" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:52.532333 31592 raft_consensus.cc:493] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.532915 31531 tablet_bootstrap.cc:654] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:52.532374 31448 leader_election.cc:343] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:52.534340 31448 leader_election.cc:343] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.532823 31592 raft_consensus.cc:515] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.536615 31592 leader_election.cc:290] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:52.538882 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b810039c1e415192775c9f611cd825" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:52.539217 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b810039c1e415192775c9f611cd825" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:52.539613 31615 raft_consensus.cc:493] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.539912 31334 raft_consensus.cc:2468] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
I20260709 13:10:52.540115 31615 raft_consensus.cc:515] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.542133 31225 leader_election.cc:304] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:52.542375 31261 raft_consensus.cc:2468] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
W20260709 13:10:52.543045 31226 leader_election.cc:343] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.544394 31451 leader_election.cc:304] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:52.546248 31615 leader_election.cc:290] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:52.557650 31626 raft_consensus.cc:2804] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.558598 31626 raft_consensus.cc:493] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.558780 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:52.559059 31626 raft_consensus.cc:3060] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.559485 31334 raft_consensus.cc:2468] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:52.560642 31374 leader_election.cc:304] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:52.557978 31592 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.562273 31615 raft_consensus.cc:2804] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.562660 31615 raft_consensus.cc:493] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.561816 31592 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:10:52.564743 31207 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:52.563037 31615 raft_consensus.cc:3060] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.571173 31582 raft_consensus.cc:493] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.571667 31582 raft_consensus.cc:515] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.574040 31582 leader_election.cc:290] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:52.575210 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:52.576983 31640 consensus_queue.cc:1048] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.577924 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:52.578629 31336 raft_consensus.cc:2468] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
I20260709 13:10:52.578778 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:52.579540 31334 raft_consensus.cc:2468] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:52.579996 31225 leader_election.cc:304] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:52.581566 31582 raft_consensus.cc:2804] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.582091 31582 raft_consensus.cc:493] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.582638 31225 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:52.583204 31582 raft_consensus.cc:3060] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.584633 31664 raft_consensus.cc:2804] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:52.587476 31664 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.588781 31664 raft_consensus.cc:3060] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:52.590344 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:52.591301 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:52.592010 31410 raft_consensus.cc:2468] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:52.593493 31592 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:52.529752 31607 raft_consensus.cc:515] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.608146 31533 tablet_bootstrap.cc:492] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:52.608537 31662 raft_consensus.cc:2804] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.611649 31615 raft_consensus.cc:515] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.612861 31409 raft_consensus.cc:2393] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75b6f0017ce84a4c9822ffd27e053405 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:52.613013 31640 consensus_queue.cc:1048] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.614975 31607 leader_election.cc:290] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:52.615598 31582 raft_consensus.cc:515] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.617146 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70d36cace8741d7b866e776f6cc03c1" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:52.617919 31410 raft_consensus.cc:2468] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:52.619656 31261 raft_consensus.cc:2393] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 75b6f0017ce84a4c9822ffd27e053405 in this term.
I20260709 13:10:52.619382 31449 leader_election.cc:304] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:52.623102 31607 raft_consensus.cc:2804] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.623011 31626 raft_consensus.cc:515] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.623617 31607 raft_consensus.cc:493] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.624033 31607 raft_consensus.cc:3060] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.626391 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b810039c1e415192775c9f611cd825" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:52.627085 31530 ts_tablet_manager.cc:1434] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.120s	user 0.006s	sys 0.001s
I20260709 13:10:52.628298 31530 tablet_bootstrap.cc:492] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
W20260709 13:10:52.628182 31226 leader_election.cc:343] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.628980 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:52.629796 31409 raft_consensus.cc:2393] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 75b6f0017ce84a4c9822ffd27e053405 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:52.631294 31607 raft_consensus.cc:515] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.633339 31607 leader_election.cc:290] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:52.634670 31662 raft_consensus.cc:697] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:10:52.635234 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70d36cace8741d7b866e776f6cc03c1" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:52.635787 31409 raft_consensus.cc:3060] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.635717 31662 consensus_queue.cc:237] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.644740 31409 raft_consensus.cc:2468] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:52.608671 31533 ts_tablet_manager.cc:1403] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.132s	user 0.017s	sys 0.058s
I20260709 13:10:52.650285 31533 raft_consensus.cc:359] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.650921 31533 raft_consensus.cc:385] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.651191 31533 raft_consensus.cc:740] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.651818 31533 consensus_queue.cc:260] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.655023 31531 tablet_bootstrap.cc:492] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:52.654966 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b810039c1e415192775c9f611cd825" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:52.655877 31336 raft_consensus.cc:3060] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.657138 31640 raft_consensus.cc:493] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.657423 31615 leader_election.cc:290] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:52.657670 31640 raft_consensus.cc:515] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.658509 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:52.659072 31337 raft_consensus.cc:3060] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.659683 31640 leader_election.cc:290] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:52.660861 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:52.661242 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf056205d504594827b6fda57d3cb62" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:52.661782 31261 raft_consensus.cc:2393] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 75b6f0017ce84a4c9822ffd27e053405 in this term.
I20260709 13:10:52.661947 31483 raft_consensus.cc:2468] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:52.660653 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:52.662552 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf056205d504594827b6fda57d3cb62" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:52.663116 31410 raft_consensus.cc:2468] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:52.663233 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70d36cace8741d7b866e776f6cc03c1" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:52.664017 31261 raft_consensus.cc:2468] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:52.664502 31301 leader_election.cc:304] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:52.665410 31640 raft_consensus.cc:2804] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.665759 31640 raft_consensus.cc:493] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.666076 31640 raft_consensus.cc:3060] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.666258 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70d36cace8741d7b866e776f6cc03c1" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:52.666848 31532 tablet_bootstrap.cc:492] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:52.667392 31532 ts_tablet_manager.cc:1403] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.162s	user 0.019s	sys 0.016s
I20260709 13:10:52.666795 31261 raft_consensus.cc:3060] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.671679 31664 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.667402 31336 raft_consensus.cc:2468] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:52.677394 31626 leader_election.cc:290] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:52.677229 31449 leader_election.cc:304] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:52.678061 31582 leader_election.cc:290] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:52.677743 31532 raft_consensus.cc:359] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.680496 31531 ts_tablet_manager.cc:1403] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.159s	user 0.012s	sys 0.022s
I20260709 13:10:52.681345 31530 tablet_bootstrap.cc:654] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.680473 31532 raft_consensus.cc:385] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.681779 31532 raft_consensus.cc:740] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.682835 31532 consensus_queue.cc:260] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.685640 31225 leader_election.cc:304] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:52.686808 31225 leader_election.cc:304] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:52.687541 31337 raft_consensus.cc:2468] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:52.689548 31626 raft_consensus.cc:2749] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.689816 31659 raft_consensus.cc:2804] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.690203 31626 raft_consensus.cc:2804] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.690673 31626 raft_consensus.cc:697] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:10:52.690687 31659 raft_consensus.cc:697] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:10:52.691689 31626 consensus_queue.cc:237] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.692024 31659 consensus_queue.cc:237] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.696600 31640 raft_consensus.cc:515] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.698626 31640 leader_election.cc:290] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:52.698407 31532 ts_tablet_manager.cc:1434] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:10:52.700011 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf056205d504594827b6fda57d3cb62" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:52.700371 31531 raft_consensus.cc:359] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.702450 31485 raft_consensus.cc:3060] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.702513 31532 tablet_bootstrap.cc:492] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:52.700906 31533 ts_tablet_manager.cc:1434] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 13:10:52.702826 31531 raft_consensus.cc:385] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.706838 31531 raft_consensus.cc:740] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.704360 31261 raft_consensus.cc:2468] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:52.712066 31533 tablet_bootstrap.cc:492] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:52.713760 31532 tablet_bootstrap.cc:654] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.713414 31158 catalog_manager.cc:5697] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 1 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.714720 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:52.715301 31336 raft_consensus.cc:3060] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:52.707350 31531 consensus_queue.cc:260] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.720521 31531 ts_tablet_manager.cc:1434] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.002s
I20260709 13:10:52.718158 31664 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:52.731885 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:52.732676 31409 raft_consensus.cc:3060] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:52.734489 31374 leader_election.cc:304] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:52.737856 31615 raft_consensus.cc:2804] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.738171 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acf056205d504594827b6fda57d3cb62" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:52.738385 31615 raft_consensus.cc:697] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:52.738821 31411 raft_consensus.cc:3060] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.738986 31336 raft_consensus.cc:2468] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:10:52.739293 31615 consensus_queue.cc:237] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.744148 31159 catalog_manager.cc:5697] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.732910 31531 tablet_bootstrap.cc:492] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:52.735909 31485 raft_consensus.cc:2468] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:52.748173 31225 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:52.750676 31533 tablet_bootstrap.cc:654] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.779168 31303 leader_election.cc:304] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:52.781391 31664 raft_consensus.cc:2804] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:52.781906 31664 raft_consensus.cc:697] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:10:52.782784 31664 consensus_queue.cc:237] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.785470 31531 tablet_bootstrap.cc:654] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.761056 31160 catalog_manager.cc:5697] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 1 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.755640 31530 tablet_bootstrap.cc:492] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:52.775951 31409 raft_consensus.cc:2468] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:10:52.777601 31411 raft_consensus.cc:2468] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:52.783250 31640 raft_consensus.cc:2804] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.789407 31640 raft_consensus.cc:697] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:52.786973 31530 ts_tablet_manager.cc:1403] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.159s	user 0.016s	sys 0.008s
I20260709 13:10:52.790191 31640 consensus_queue.cc:237] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:52.765053 31659 raft_consensus.cc:493] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.792790 31530 raft_consensus.cc:359] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.793924 31530 raft_consensus.cc:385] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.792333 31159 catalog_manager.cc:5697] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:52.793753 31659 raft_consensus.cc:515] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.794298 31530 raft_consensus.cc:740] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.796669 31530 consensus_queue.cc:260] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.798632 31659 leader_election.cc:290] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:52.799403 31533 tablet_bootstrap.cc:492] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:52.804917 31533 ts_tablet_manager.cc:1403] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.001s
I20260709 13:10:52.800067 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24fe066d118d48c5939576417491e9ec" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:52.805953 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24fe066d118d48c5939576417491e9ec" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:52.806429 31530 ts_tablet_manager.cc:1434] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:10:52.809329 31261 raft_consensus.cc:2468] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:52.809604 31411 raft_consensus.cc:2468] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:52.810230 31530 tablet_bootstrap.cc:492] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:52.811002 31451 leader_election.cc:304] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:52.812080 31662 raft_consensus.cc:2804] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.812445 31662 raft_consensus.cc:493] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.812407 31533 raft_consensus.cc:359] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.812922 31662 raft_consensus.cc:3060] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.813139 31533 raft_consensus.cc:385] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.813443 31533 raft_consensus.cc:740] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.814098 31533 consensus_queue.cc:260] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.815371 31157 catalog_manager.cc:5697] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.821611 31662 raft_consensus.cc:515] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.806557 31640 consensus_queue.cc:1048] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.823698 31533 ts_tablet_manager.cc:1434] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:10:52.825563 31533 tablet_bootstrap.cc:492] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:52.829841 31530 tablet_bootstrap.cc:654] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.835239 31532 tablet_bootstrap.cc:492] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:52.835705 31532 ts_tablet_manager.cc:1403] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.134s	user 0.021s	sys 0.016s
I20260709 13:10:52.840561 31532 raft_consensus.cc:359] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.841218 31532 raft_consensus.cc:385] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.841526 31532 raft_consensus.cc:740] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.839301 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24fe066d118d48c5939576417491e9ec" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:52.842396 31532 consensus_queue.cc:260] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.843947 31533 tablet_bootstrap.cc:654] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.847126 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24fe066d118d48c5939576417491e9ec" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:52.847926 31260 raft_consensus.cc:3060] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.849001 31532 ts_tablet_manager.cc:1434] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.002s
I20260709 13:10:52.851651 31531 tablet_bootstrap.cc:492] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:52.854212 31411 raft_consensus.cc:3060] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.863750 31532 tablet_bootstrap.cc:492] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:52.866489 31531 ts_tablet_manager.cc:1403] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.134s	user 0.009s	sys 0.005s
I20260709 13:10:52.873405 31662 leader_election.cc:290] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:52.876386 31531 raft_consensus.cc:359] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.882081 31531 raft_consensus.cc:385] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.882372 31531 raft_consensus.cc:740] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.883985 31530 tablet_bootstrap.cc:492] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:52.884511 31530 ts_tablet_manager.cc:1403] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.074s	user 0.022s	sys 0.025s
I20260709 13:10:52.886430 31533 tablet_bootstrap.cc:492] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:52.883946 31532 tablet_bootstrap.cc:654] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.886911 31533 ts_tablet_manager.cc:1403] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.062s	user 0.007s	sys 0.012s
I20260709 13:10:52.889662 31530 raft_consensus.cc:359] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.890333 31530 raft_consensus.cc:385] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.890702 31530 raft_consensus.cc:740] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.890743 31156 catalog_manager.cc:5697] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 0 to 2, leader changed from <none> to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.895329 31532 tablet_bootstrap.cc:492] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:52.895833 31532 ts_tablet_manager.cc:1403] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.000s
I20260709 13:10:52.890058 31637 consensus_queue.cc:1048] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.898443 31532 raft_consensus.cc:359] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.899098 31532 raft_consensus.cc:385] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.899417 31532 raft_consensus.cc:740] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.900089 31532 consensus_queue.cc:260] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.902122 31532 ts_tablet_manager.cc:1434] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:10:52.881116 31260 raft_consensus.cc:2468] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:52.903085 31532 tablet_bootstrap.cc:492] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:52.891639 31530 consensus_queue.cc:260] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.903504 31451 leader_election.cc:304] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:52.904578 31659 raft_consensus.cc:2804] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.905022 31659 raft_consensus.cc:697] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:10:52.905805 31659 consensus_queue.cc:237] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.906730 31530 ts_tablet_manager.cc:1434] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:10:52.907655 31530 tablet_bootstrap.cc:492] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:52.909379 31532 tablet_bootstrap.cc:654] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.919005 31530 tablet_bootstrap.cc:654] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.892206 31533 raft_consensus.cc:359] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.922010 31533 raft_consensus.cc:385] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.922320 31533 raft_consensus.cc:740] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.877285 31411 raft_consensus.cc:2468] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:52.889496 31531 consensus_queue.cc:260] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.924549 31531 ts_tablet_manager.cc:1434] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.002s
I20260709 13:10:52.924937 31532 tablet_bootstrap.cc:492] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:52.924232 31533 consensus_queue.cc:260] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.925411 31532 ts_tablet_manager.cc:1403] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.023s	user 0.005s	sys 0.012s
I20260709 13:10:52.925985 31531 tablet_bootstrap.cc:492] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:52.927760 31533 ts_tablet_manager.cc:1434] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:10:52.928011 31532 raft_consensus.cc:359] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.928682 31532 raft_consensus.cc:385] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.928951 31532 raft_consensus.cc:740] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.929708 31532 consensus_queue.cc:260] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:52.935699 31532 ts_tablet_manager.cc:1434] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:10:52.936556 31615 raft_consensus.cc:493] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.937045 31615 raft_consensus.cc:515] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:52.939096 31615 leader_election.cc:290] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:52.940843 31532 tablet_bootstrap.cc:492] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:52.940568 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b810039c1e415192775c9f611cd825" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:52.946547 31532 tablet_bootstrap.cc:654] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.940529 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b810039c1e415192775c9f611cd825" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:52.949028 31336 raft_consensus.cc:2393] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 75b6f0017ce84a4c9822ffd27e053405 in this term.
I20260709 13:10:52.950623 31374 leader_election.cc:304] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:52.951623 31615 raft_consensus.cc:3060] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.928696 31533 tablet_bootstrap.cc:492] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:52.958597 31663 raft_consensus.cc:493] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.959100 31659 raft_consensus.cc:493] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.959069 31663 raft_consensus.cc:515] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.959707 31659 raft_consensus.cc:515] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.961301 31663 leader_election.cc:290] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:52.965199 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181fa1bbe1e14e098cad0b23561576d2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:52.964094 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181fa1bbe1e14e098cad0b23561576d2" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:52.969812 31485 raft_consensus.cc:2468] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:52.970052 31411 raft_consensus.cc:2468] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:52.970367 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181fa1bbe1e14e098cad0b23561576d2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:52.966414 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181fa1bbe1e14e098cad0b23561576d2" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:52.967403 31531 tablet_bootstrap.cc:654] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.961771 31659 leader_election.cc:290] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:52.974669 31449 leader_election.cc:304] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:52.974926 31377 leader_election.cc:304] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:52.975525 31659 raft_consensus.cc:2804] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.975888 31659 raft_consensus.cc:493] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.976230 31659 raft_consensus.cc:3060] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.970985 31261 raft_consensus.cc:2468] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:52.981714 31533 tablet_bootstrap.cc:654] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.985163 31663 raft_consensus.cc:2804] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.985687 31663 raft_consensus.cc:493] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.986123 31663 raft_consensus.cc:3060] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.987107 31659 raft_consensus.cc:515] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:52.989516 31158 catalog_manager.cc:5697] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 1 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.990782 31659 leader_election.cc:290] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:52.992187 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181fa1bbe1e14e098cad0b23561576d2" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:52.993271 31261 raft_consensus.cc:3060] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.994230 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181fa1bbe1e14e098cad0b23561576d2" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:52.995985 31615 raft_consensus.cc:2749] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.004812 31531 tablet_bootstrap.cc:492] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:53.005282 31531 ts_tablet_manager.cc:1403] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.001s
I20260709 13:10:53.008047 31532 tablet_bootstrap.cc:492] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:53.008527 31532 ts_tablet_manager.cc:1403] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.068s	user 0.005s	sys 0.008s
I20260709 13:10:53.011044 31531 raft_consensus.cc:359] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.011893 31531 raft_consensus.cc:385] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.012210 31531 raft_consensus.cc:740] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.011394 31673 raft_consensus.cc:493] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.015175 31530 tablet_bootstrap.cc:492] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:53.015794 31530 ts_tablet_manager.cc:1403] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.108s	user 0.021s	sys 0.049s
I20260709 13:10:53.019152 31530 raft_consensus.cc:359] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.020076 31530 raft_consensus.cc:385] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.014233 31531 consensus_queue.cc:260] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.020368 31530 raft_consensus.cc:740] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.021891 31530 consensus_queue.cc:260] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.023922 31530 ts_tablet_manager.cc:1434] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:53.024778 31530 tablet_bootstrap.cc:492] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:53.018977 31663 raft_consensus.cc:515] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.014081 31673 raft_consensus.cc:515] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.029022 31673 leader_election.cc:290] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:53.029683 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b06bd7c5b2a4586b9caf425bc23720a" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:53.030300 31485 raft_consensus.cc:2468] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:53.030730 31411 raft_consensus.cc:2393] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:53.032112 31261 raft_consensus.cc:2468] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:53.032909 31532 raft_consensus.cc:359] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.033653 31532 raft_consensus.cc:385] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.033886 31532 raft_consensus.cc:740] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.033998 31531 ts_tablet_manager.cc:1434] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.004s
I20260709 13:10:53.034724 31532 consensus_queue.cc:260] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.035714 31451 leader_election.cc:304] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:53.037350 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181fa1bbe1e14e098cad0b23561576d2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:53.037703 31303 leader_election.cc:304] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:53.038677 31673 raft_consensus.cc:2804] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.038877 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b06bd7c5b2a4586b9caf425bc23720a" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:53.037855 31663 leader_election.cc:290] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:53.039495 31530 tablet_bootstrap.cc:654] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.039054 31673 raft_consensus.cc:493] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.040035 31261 raft_consensus.cc:2393] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 001cac731610478a900ad62e830aaff6 in this term.
I20260709 13:10:53.040249 31673 raft_consensus.cc:3060] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.040738 31531 tablet_bootstrap.cc:492] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:53.043033 31659 raft_consensus.cc:2804] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.044111 31659 raft_consensus.cc:697] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:10:53.044816 31659 consensus_queue.cc:237] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.045706 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181fa1bbe1e14e098cad0b23561576d2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:53.046937 31531 tablet_bootstrap.cc:654] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.046886 31582 raft_consensus.cc:493] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.047570 31582 raft_consensus.cc:515] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.048532 31532 ts_tablet_manager.cc:1434] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.001s
I20260709 13:10:53.049460 31532 tablet_bootstrap.cc:492] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:53.049606 31582 leader_election.cc:290] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:53.051220 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:53.052392 31377 leader_election.cc:304] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:53.039547 31260 raft_consensus.cc:2468] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:53.054508 31637 raft_consensus.cc:493] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:53.055042 31226 leader_election.cc:343] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.057997 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:53.055111 31637 raft_consensus.cc:515] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.063886 31533 tablet_bootstrap.cc:492] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:53.064527 31533 ts_tablet_manager.cc:1403] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.136s	user 0.014s	sys 0.004s
I20260709 13:10:53.068186 31532 tablet_bootstrap.cc:654] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.062197 31663 raft_consensus.cc:2749] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:10:53.067094 31225 leader_election.cc:343] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.073482 31225 leader_election.cc:304] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:53.075127 31582 raft_consensus.cc:2749] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.084497 31673 raft_consensus.cc:515] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.085813 31530 tablet_bootstrap.cc:492] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:53.086421 31530 ts_tablet_manager.cc:1403] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.006s
I20260709 13:10:53.086486 31673 leader_election.cc:290] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:53.089394 31533 raft_consensus.cc:359] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.090070 31533 raft_consensus.cc:385] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.089658 31530 raft_consensus.cc:359] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.090356 31533 raft_consensus.cc:740] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.090528 31530 raft_consensus.cc:385] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.090988 31530 raft_consensus.cc:740] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.091092 31533 consensus_queue.cc:260] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.091907 31530 consensus_queue.cc:260] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.094285 31530 ts_tablet_manager.cc:1434] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:53.093811 31157 catalog_manager.cc:5697] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 1 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.095332 31530 tablet_bootstrap.cc:492] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:53.095605 31666 consensus_queue.cc:1048] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.106400 31531 tablet_bootstrap.cc:492] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:53.106905 31531 ts_tablet_manager.cc:1403] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.066s	user 0.007s	sys 0.014s
I20260709 13:10:53.115744 31637 leader_election.cc:290] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:53.109797 31531 raft_consensus.cc:359] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.125744 31259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa7546f06a454f3899cb0bf7d0c0d084" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:53.155961 31531 raft_consensus.cc:385] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.156270 31531 raft_consensus.cc:740] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.128187 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa7546f06a454f3899cb0bf7d0c0d084" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:53.157007 31531 consensus_queue.cc:260] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.128494 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b06bd7c5b2a4586b9caf425bc23720a" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:53.158210 31483 raft_consensus.cc:3060] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.160375 31531 ts_tablet_manager.cc:1434] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.004s
I20260709 13:10:53.128947 31582 consensus_queue.cc:1048] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 13:10:53.161397 31531 tablet_bootstrap.cc:492] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:53.132922 31659 consensus_queue.cc:1048] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:10:53.167595 31531 tablet_bootstrap.cc:654] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.151772 31532 tablet_bootstrap.cc:492] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:53.168651 31532 ts_tablet_manager.cc:1403] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.119s	user 0.030s	sys 0.005s
I20260709 13:10:53.117241 31533 ts_tablet_manager.cc:1434] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 13:10:53.169835 31533 tablet_bootstrap.cc:492] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:53.171633 31532 raft_consensus.cc:359] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.172330 31532 raft_consensus.cc:385] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.172624 31532 raft_consensus.cc:740] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.112749 31530 tablet_bootstrap.cc:654] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.144164 31669 consensus_queue.cc:1048] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:10:53.176064 31532 consensus_queue.cc:260] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.178627 31532 ts_tablet_manager.cc:1434] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:10:53.141898 31626 raft_consensus.cc:493] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.181099 31626 raft_consensus.cc:515] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.183421 31626 leader_election.cc:290] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:53.124172 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b06bd7c5b2a4586b9caf425bc23720a" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:53.184864 31261 raft_consensus.cc:3060] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.185829 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
W20260709 13:10:53.187299 31225 leader_election.cc:343] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.188656 31662 raft_consensus.cc:493] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.188805 31663 raft_consensus.cc:493] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.189162 31662 raft_consensus.cc:515] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.189283 31663 raft_consensus.cc:515] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.190380 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:53.191202 31663 leader_election.cc:290] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
W20260709 13:10:53.192065 31228 leader_election.cc:343] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.192180 31662 leader_election.cc:290] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:53.192613 31228 leader_election.cc:304] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:53.192955 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:53.193598 31337 raft_consensus.cc:2468] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:53.193794 31626 raft_consensus.cc:2749] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.195600 31261 raft_consensus.cc:2468] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:53.197429 31532 tablet_bootstrap.cc:492] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:53.202440 31257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:53.203276 31257 raft_consensus.cc:2468] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:53.209932 31303 leader_election.cc:304] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:53.212069 31673 raft_consensus.cc:2804] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.212679 31673 raft_consensus.cc:697] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:53.156280 31259 raft_consensus.cc:2393] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07b814fcedea4142ac09385b680cae3a in current term 1: Already voted for candidate 001cac731610478a900ad62e830aaff6 in this term.
I20260709 13:10:53.214545 31532 tablet_bootstrap.cc:654] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.217175 31374 leader_election.cc:304] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:53.218771 31615 raft_consensus.cc:2804] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.219003 31530 tablet_bootstrap.cc:492] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:53.219173 31615 raft_consensus.cc:493] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.219548 31530 ts_tablet_manager.cc:1403] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.124s	user 0.013s	sys 0.004s
I20260709 13:10:53.219624 31615 raft_consensus.cc:3060] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.217203 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142741f7f41a4baa824a20e79ddf5cb6" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:53.222010 31409 raft_consensus.cc:2468] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:53.222407 31530 raft_consensus.cc:359] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.222997 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142741f7f41a4baa824a20e79ddf5cb6" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:53.223161 31530 raft_consensus.cc:385] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.223266 31334 raft_consensus.cc:3060] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.223675 31530 raft_consensus.cc:740] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.223773 31337 raft_consensus.cc:2468] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:53.225157 31448 leader_election.cc:304] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:53.225863 31530 consensus_queue.cc:260] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.227481 31530 ts_tablet_manager.cc:1434] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:10:53.228329 31530 tablet_bootstrap.cc:492] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:53.230291 31659 raft_consensus.cc:2804] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.230827 31659 raft_consensus.cc:493] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.231287 31659 raft_consensus.cc:3060] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.233722 31615 raft_consensus.cc:515] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.235787 31615 leader_election.cc:290] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:53.236837 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:53.237412 31335 raft_consensus.cc:3060] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.241936 31531 tablet_bootstrap.cc:492] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:53.242054 31259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:53.242446 31531 ts_tablet_manager.cc:1403] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.081s	user 0.016s	sys 0.023s
I20260709 13:10:53.242655 31259 raft_consensus.cc:3060] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.246237 31532 tablet_bootstrap.cc:492] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:53.246739 31532 ts_tablet_manager.cc:1403] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.000s
I20260709 13:10:53.247726 31615 consensus_queue.cc:1048] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.249065 31673 consensus_queue.cc:237] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.249541 31532 raft_consensus.cc:359] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.250285 31532 raft_consensus.cc:385] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.250576 31532 raft_consensus.cc:740] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.251276 31532 consensus_queue.cc:260] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.252468 31531 raft_consensus.cc:359] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.253178 31531 raft_consensus.cc:385] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.253307 31532 ts_tablet_manager.cc:1434] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:10:53.257081 31532 tablet_bootstrap.cc:492] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:53.254431 31303 leader_election.cc:304] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:53.255760 31335 raft_consensus.cc:2468] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:53.259871 31640 consensus_queue.cc:1048] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:53.260614 31374 leader_election.cc:304] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:53.255983 31530 tablet_bootstrap.cc:654] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.263886 31615 raft_consensus.cc:2804] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.264385 31615 raft_consensus.cc:697] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:53.265236 31615 consensus_queue.cc:237] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.253471 31531 raft_consensus.cc:740] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.270306 31531 consensus_queue.cc:260] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.272080 31626 consensus_queue.cc:1048] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.273062 31410 raft_consensus.cc:1275] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:53.275022 31532 tablet_bootstrap.cc:654] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.277611 31626 consensus_queue.cc:1048] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.281591 31663 raft_consensus.cc:493] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.282088 31663 raft_consensus.cc:515] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.285604 31663 leader_election.cc:290] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:53.294200 31532 tablet_bootstrap.cc:492] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:53.304402 31532 ts_tablet_manager.cc:1403] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.000s
I20260709 13:10:53.297349 31533 tablet_bootstrap.cc:654] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.301367 31257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92573de449ff4002a92b5ffcecfb8521" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:53.306033 31257 raft_consensus.cc:2468] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:53.308614 31532 raft_consensus.cc:359] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.309680 31532 raft_consensus.cc:385] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.308496 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92573de449ff4002a92b5ffcecfb8521" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:53.308554 31377 leader_election.cc:304] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:53.261037 31483 raft_consensus.cc:2468] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:53.310424 31532 raft_consensus.cc:740] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.315621 31532 consensus_queue.cc:260] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.316915 31663 raft_consensus.cc:2804] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.338374 31531 ts_tablet_manager.cc:1434] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 13:10:53.340137 31531 tablet_bootstrap.cc:492] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:53.311429 31659 raft_consensus.cc:515] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.344414 31532 ts_tablet_manager.cc:1434] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.001s
I20260709 13:10:53.345819 31532 tablet_bootstrap.cc:492] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:53.351473 31259 raft_consensus.cc:2468] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:53.338915 31663 raft_consensus.cc:493] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:53.348834 31374 leader_election.cc:343] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.348435 31637 raft_consensus.cc:2749] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.366075 31662 consensus_queue.cc:1048] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:53.375769 31530 tablet_bootstrap.cc:492] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:53.358502 31663 raft_consensus.cc:3060] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.386303 31530 ts_tablet_manager.cc:1403] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.158s	user 0.020s	sys 0.029s
I20260709 13:10:53.388520 31666 consensus_queue.cc:1048] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.393157 31659 leader_election.cc:290] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:53.393950 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142741f7f41a4baa824a20e79ddf5cb6" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:53.394527 31337 raft_consensus.cc:3060] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.398613 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142741f7f41a4baa824a20e79ddf5cb6" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:53.399442 31410 raft_consensus.cc:3060] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.399080 31530 raft_consensus.cc:359] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.400089 31530 raft_consensus.cc:385] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.402468 31337 raft_consensus.cc:2468] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
W20260709 13:10:53.403399 31374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39075, user_credentials={real_user=slave}} blocked reactor thread for 55945.6us
I20260709 13:10:53.403913 31530 raft_consensus.cc:740] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.405066 31531 tablet_bootstrap.cc:654] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.405133 31532 tablet_bootstrap.cc:654] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.417472 31530 consensus_queue.cc:260] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.439080 31448 leader_election.cc:304] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:53.443004 31711 consensus_queue.cc:1048] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.453296 31530 ts_tablet_manager.cc:1434] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.000s
I20260709 13:10:53.455996 31530 tablet_bootstrap.cc:492] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
W20260709 13:10:53.445415 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:44895, user_credentials={real_user=slave}} blocked reactor thread for 53547.4us
I20260709 13:10:53.447604 31666 raft_consensus.cc:2804] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.417821 31157 catalog_manager.cc:5697] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:53.467746 31336 raft_consensus.cc:1275] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:53.471235 31666 raft_consensus.cc:697] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:10:53.485538 31715 consensus_queue.cc:1048] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:10:53.474895 31410 raft_consensus.cc:2468] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:53.492379 31533 tablet_bootstrap.cc:492] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:53.492996 31533 ts_tablet_manager.cc:1403] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.323s	user 0.012s	sys 0.005s
I20260709 13:10:53.494951 31659 consensus_queue.cc:1048] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.495695 31711 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.496256 31711 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.498832 31711 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:53.500188 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:53.484388 31626 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.509967 31664 consensus_queue.cc:1048] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:10:53.511873 31626 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.525919 31637 raft_consensus.cc:493] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.562896 31637 raft_consensus.cc:515] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.563485 31582 raft_consensus.cc:493] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.563983 31582 raft_consensus.cc:515] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.564702 31530 tablet_bootstrap.cc:654] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.564978 31637 leader_election.cc:290] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:53.558943 31532 tablet_bootstrap.cc:492] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:53.561928 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:53.529927 31666 consensus_queue.cc:237] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.560461 31533 raft_consensus.cc:359] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.566587 31532 ts_tablet_manager.cc:1403] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.221s	user 0.029s	sys 0.009s
I20260709 13:10:53.571785 31533 raft_consensus.cc:385] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.572182 31533 raft_consensus.cc:740] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.581707 31260 raft_consensus.cc:2468] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:53.581660 31663 raft_consensus.cc:515] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.584188 31626 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:53.574350 31533 consensus_queue.cc:260] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.585410 31663 leader_election.cc:290] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:53.587980 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:53.588714 31258 raft_consensus.cc:2468] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:53.591023 31533 ts_tablet_manager.cc:1434] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.098s	user 0.007s	sys 0.000s
I20260709 13:10:53.592603 31533 tablet_bootstrap.cc:492] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:53.594381 31582 leader_election.cc:290] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:53.599220 31531 tablet_bootstrap.cc:492] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:53.599757 31531 ts_tablet_manager.cc:1403] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.260s	user 0.018s	sys 0.020s
I20260709 13:10:53.601262 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:53.601349 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92573de449ff4002a92b5ffcecfb8521" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:53.601764 31408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:53.602137 31260 raft_consensus.cc:3060] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.606369 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:53.611680 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92573de449ff4002a92b5ffcecfb8521" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:53.612000 31411 raft_consensus.cc:2468] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
I20260709 13:10:53.613197 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:53.614398 31482 raft_consensus.cc:2468] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:53.619294 31533 tablet_bootstrap.cc:654] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.619971 31303 leader_election.cc:304] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
W20260709 13:10:53.622949 31374 leader_election.cc:343] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.624831 31715 raft_consensus.cc:2804] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.625331 31715 raft_consensus.cc:493] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.625751 31715 raft_consensus.cc:3060] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:53.622112 31377 leader_election.cc:343] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.627049 31377 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:53.629338 31615 raft_consensus.cc:2804] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.629698 31615 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.630008 31615 raft_consensus.cc:3060] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:53.638423 31228 leader_election.cc:343] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.641638 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:53.644780 31226 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:45117, user_credentials={real_user=slave}} blocked reactor thread for 43797.9us
W20260709 13:10:53.646000 31226 leader_election.cc:343] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.646914 31226 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
W20260709 13:10:53.647918 31228 leader_election.cc:343] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.648650 31728 raft_consensus.cc:2804] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.649068 31228 leader_election.cc:304] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:53.649245 31728 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.650097 31728 raft_consensus.cc:3060] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.650403 31582 raft_consensus.cc:2749] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.679255 31532 raft_consensus.cc:359] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.680294 31532 raft_consensus.cc:385] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.680646 31532 raft_consensus.cc:740] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.682170 31532 consensus_queue.cc:260] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.687999 31532 ts_tablet_manager.cc:1434] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.116s	user 0.007s	sys 0.002s
I20260709 13:10:53.694111 31531 raft_consensus.cc:359] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.694950 31531 raft_consensus.cc:385] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.695324 31531 raft_consensus.cc:740] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.696425 31532 tablet_bootstrap.cc:492] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:53.696152 31531 consensus_queue.cc:260] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.699872 31664 raft_consensus.cc:493] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:53.694475 31449 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:45117, user_credentials={real_user=slave}} blocked reactor thread for 106651us
I20260709 13:10:53.734287 31664 raft_consensus.cc:515] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.735745 31734 consensus_queue.cc:1048] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.736394 31664 leader_election.cc:290] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:53.737049 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077ed73927a64748baae88b0b8dc4a13" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:53.738564 31228 leader_election.cc:343] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.722114 31530 tablet_bootstrap.cc:492] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:53.739338 31724 consensus_queue.cc:1048] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.740165 31530 ts_tablet_manager.cc:1403] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.286s	user 0.017s	sys 0.013s
I20260709 13:10:53.741138 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077ed73927a64748baae88b0b8dc4a13" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:53.723086 31637 raft_consensus.cc:493] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.741808 31637 raft_consensus.cc:515] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.743872 31637 leader_election.cc:290] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:53.745354 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92573de449ff4002a92b5ffcecfb8521" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:53.746153 31409 raft_consensus.cc:2393] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07b814fcedea4142ac09385b680cae3a in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:53.712260 31531 ts_tablet_manager.cc:1434] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.112s	user 0.007s	sys 0.003s
I20260709 13:10:53.747991 31531 tablet_bootstrap.cc:492] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
W20260709 13:10:53.752115 31225 leader_election.cc:343] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.754863 31666 consensus_queue.cc:1048] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.754927 31225 leader_election.cc:304] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:53.756063 31664 raft_consensus.cc:2749] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.757144 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92573de449ff4002a92b5ffcecfb8521" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:53.759382 31303 leader_election.cc:304] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:53.769527 31531 tablet_bootstrap.cc:654] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.768752 31637 raft_consensus.cc:3060] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:53.775906 31207 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:53.784747 31664 raft_consensus.cc:493] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.785223 31664 raft_consensus.cc:515] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:53.787637 31664 leader_election.cc:290] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:53.787912 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29510c165a1f48b1949c6ee31e136064" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:53.790833 31260 raft_consensus.cc:2468] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
W20260709 13:10:53.799548 31225 leader_election.cc:343] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.800293 31377 leader_election.cc:304] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:53.800316 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29510c165a1f48b1949c6ee31e136064" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:53.791388 31615 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.796129 31532 tablet_bootstrap.cc:654] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.795781 31530 raft_consensus.cc:359] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.802525 31530 raft_consensus.cc:385] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.802909 31530 raft_consensus.cc:740] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.801447 31724 raft_consensus.cc:2804] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.805757 31724 raft_consensus.cc:697] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:53.806417 31530 consensus_queue.cc:260] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.806599 31724 consensus_queue.cc:237] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:10:53.807350 31226 leader_election.cc:343] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.720794 31715 raft_consensus.cc:515] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.801486 31156 catalog_manager.cc:5697] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:53.809628 31226 leader_election.cc:304] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:53.817624 31664 raft_consensus.cc:2749] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.809046 31637 raft_consensus.cc:2749] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.803346 31728 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.810604 31530 ts_tablet_manager.cc:1434] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.000s
I20260709 13:10:53.822245 31530 tablet_bootstrap.cc:492] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:53.822953 31728 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:53.824753 31158 catalog_manager.cc:5697] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 1 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:53.827952 31530 tablet_bootstrap.cc:654] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.836652 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:53.837453 31718 consensus_queue.cc:1048] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.838605 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:53.839363 31411 raft_consensus.cc:2393] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 75b6f0017ce84a4c9822ffd27e053405 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:53.840090 31663 consensus_queue.cc:1048] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:53.847430 31228 leader_election.cc:343] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.848558 31533 tablet_bootstrap.cc:492] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:53.848986 31533 ts_tablet_manager.cc:1403] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.257s	user 0.027s	sys 0.015s
I20260709 13:10:53.855015 31715 leader_election.cc:290] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:53.856269 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:53.856837 31482 raft_consensus.cc:3060] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.850996 31226 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:53.868932 31664 raft_consensus.cc:2749] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.874408 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:53.886404 31260 raft_consensus.cc:3060] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.888854 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:53.888892 31718 consensus_queue.cc:1048] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.890022 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:53.890776 31262 raft_consensus.cc:2393] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 75b6f0017ce84a4c9822ffd27e053405 in this term.
I20260709 13:10:53.877688 31482 raft_consensus.cc:2468] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:53.892680 31303 leader_election.cc:304] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:53.893522 31715 raft_consensus.cc:2804] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.893971 31715 raft_consensus.cc:697] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:53.894809 31715 consensus_queue.cc:237] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.899680 31615 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:53.908305 31531 tablet_bootstrap.cc:492] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:53.908775 31531 ts_tablet_manager.cc:1403] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.161s	user 0.007s	sys 0.019s
I20260709 13:10:53.911509 31531 raft_consensus.cc:359] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.912130 31531 raft_consensus.cc:385] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.912366 31531 raft_consensus.cc:740] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.913013 31531 consensus_queue.cc:260] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:53.916242 31531 ts_tablet_manager.cc:1434] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:53.921443 31530 tablet_bootstrap.cc:492] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:53.921862 31530 ts_tablet_manager.cc:1403] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.100s	user 0.006s	sys 0.014s
I20260709 13:10:53.928694 31531 tablet_bootstrap.cc:492] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
W20260709 13:10:53.934731 31377 leader_election.cc:343] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.935420 31377 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:53.937098 31615 raft_consensus.cc:2749] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.939046 31530 raft_consensus.cc:359] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.939571 31531 tablet_bootstrap.cc:654] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.939781 31530 raft_consensus.cc:385] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.940311 31530 raft_consensus.cc:740] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.939656 31533 raft_consensus.cc:359] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.941236 31533 raft_consensus.cc:385] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.941562 31533 raft_consensus.cc:740] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.942045 31530 consensus_queue.cc:260] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.944377 31530 ts_tablet_manager.cc:1434] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.002s
I20260709 13:10:53.944876 31532 tablet_bootstrap.cc:492] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:53.945356 31532 ts_tablet_manager.cc:1403] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.249s	user 0.021s	sys 0.020s
I20260709 13:10:53.945350 31530 tablet_bootstrap.cc:492] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:53.945809 31260 raft_consensus.cc:2468] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:53.946483 31533 consensus_queue.cc:260] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:53.948905 31532 raft_consensus.cc:359] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.949713 31532 raft_consensus.cc:385] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.949992 31532 raft_consensus.cc:740] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.950657 31532 consensus_queue.cc:260] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.951835 31533 ts_tablet_manager.cc:1434] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.103s	user 0.007s	sys 0.001s
I20260709 13:10:53.959620 31531 tablet_bootstrap.cc:492] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:53.952045 31530 tablet_bootstrap.cc:654] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.952900 31532 ts_tablet_manager.cc:1434] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:53.960103 31531 ts_tablet_manager.cc:1403] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.043s	user 0.007s	sys 0.002s
I20260709 13:10:53.959959 31159 catalog_manager.cc:5697] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:53.960832 31532 tablet_bootstrap.cc:492] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:53.962685 31531 raft_consensus.cc:359] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.963366 31531 raft_consensus.cc:385] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.963660 31531 raft_consensus.cc:740] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.964277 31531 consensus_queue.cc:260] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.966670 31531 ts_tablet_manager.cc:1434] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:10:53.967509 31531 tablet_bootstrap.cc:492] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:53.968151 31533 tablet_bootstrap.cc:492] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:53.973060 31531 tablet_bootstrap.cc:654] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.974813 31532 tablet_bootstrap.cc:654] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.978605 31741 consensus_queue.cc:1048] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.986740 31533 tablet_bootstrap.cc:654] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.991840 31664 raft_consensus.cc:493] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.992453 31664 raft_consensus.cc:515] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:53.994956 31664 leader_election.cc:290] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:54.003700 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.008549 31662 consensus_queue.cc:1048] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:10:54.014528 31532 tablet_bootstrap.cc:492] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:54.015043 31532 ts_tablet_manager.cc:1403] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.054s	user 0.008s	sys 0.008s
W20260709 13:10:54.017540 31228 leader_election.cc:343] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.004737 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
W20260709 13:10:54.028527 31225 leader_election.cc:343] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.028986 31225 leader_election.cc:304] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:54.030066 31664 raft_consensus.cc:2749] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.036379 31533 tablet_bootstrap.cc:492] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:54.036885 31533 ts_tablet_manager.cc:1403] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.003s
I20260709 13:10:54.036721 31532 raft_consensus.cc:359] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.037436 31532 raft_consensus.cc:385] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.037735 31532 raft_consensus.cc:740] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.038756 31531 tablet_bootstrap.cc:492] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:54.038465 31532 consensus_queue.cc:260] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.039233 31531 ts_tablet_manager.cc:1403] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.072s	user 0.031s	sys 0.019s
I20260709 13:10:54.039436 31533 raft_consensus.cc:359] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.040105 31533 raft_consensus.cc:385] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.041033 31532 ts_tablet_manager.cc:1434] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.002s
I20260709 13:10:54.041939 31532 tablet_bootstrap.cc:492] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:54.040349 31533 raft_consensus.cc:740] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.042059 31531 raft_consensus.cc:359] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.042783 31531 raft_consensus.cc:385] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.043071 31531 raft_consensus.cc:740] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.043829 31530 tablet_bootstrap.cc:492] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:54.044263 31530 ts_tablet_manager.cc:1403] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.099s	user 0.023s	sys 0.003s
I20260709 13:10:54.045219 31531 consensus_queue.cc:260] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.046679 31530 raft_consensus.cc:359] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.047307 31530 raft_consensus.cc:385] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.047556 31530 raft_consensus.cc:740] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.048007 31531 ts_tablet_manager.cc:1434] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:54.048509 31533 consensus_queue.cc:260] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.049206 31531 tablet_bootstrap.cc:492] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:54.050768 31533 ts_tablet_manager.cc:1434] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:10:54.048450 31157 catalog_manager.cc:5697] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:54.055837 31531 tablet_bootstrap.cc:654] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.056141 31530 consensus_queue.cc:260] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.060737 31532 tablet_bootstrap.cc:654] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.063637 31531 tablet_bootstrap.cc:492] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:54.064054 31531 ts_tablet_manager.cc:1403] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:10:54.066386 31531 raft_consensus.cc:359] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.067034 31531 raft_consensus.cc:385] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.067236 31533 tablet_bootstrap.cc:492] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:54.067308 31531 raft_consensus.cc:740] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.068409 31531 consensus_queue.cc:260] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.070513 31531 ts_tablet_manager.cc:1434] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:10:54.071362 31531 tablet_bootstrap.cc:492] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:54.066892 31530 ts_tablet_manager.cc:1434] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:10:54.074268 31530 tablet_bootstrap.cc:492] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:54.075343 31532 tablet_bootstrap.cc:492] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:54.075773 31532 ts_tablet_manager.cc:1403] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.003s
I20260709 13:10:54.077342 31531 tablet_bootstrap.cc:654] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.078769 31532 raft_consensus.cc:359] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.079411 31532 raft_consensus.cc:385] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.079674 31530 tablet_bootstrap.cc:654] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.079682 31532 raft_consensus.cc:740] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.080701 31532 consensus_queue.cc:260] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.082819 31533 tablet_bootstrap.cc:654] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.083499 31532 ts_tablet_manager.cc:1434] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:54.084784 31532 tablet_bootstrap.cc:492] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:54.089223 31531 tablet_bootstrap.cc:492] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:54.089692 31531 ts_tablet_manager.cc:1403] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 13:10:54.090786 31532 tablet_bootstrap.cc:654] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.092296 31531 raft_consensus.cc:359] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.092958 31531 raft_consensus.cc:385] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.093240 31531 raft_consensus.cc:740] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.093940 31531 consensus_queue.cc:260] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.096190 31531 ts_tablet_manager.cc:1434] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:54.097021 31531 tablet_bootstrap.cc:492] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:54.098775 31615 raft_consensus.cc:493] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.099395 31615 raft_consensus.cc:515] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.101529 31615 leader_election.cc:290] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.101788 31532 tablet_bootstrap.cc:492] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:54.102236 31532 ts_tablet_manager.cc:1403] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.008s
I20260709 13:10:54.102278 31531 tablet_bootstrap.cc:654] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.103094 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4be50f8b2fe4ae4a303702831a3f006" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:54.105337 31377 leader_election.cc:343] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.104960 31532 raft_consensus.cc:359] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.105854 31532 raft_consensus.cc:385] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.106139 31532 raft_consensus.cc:740] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.106590 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4be50f8b2fe4ae4a303702831a3f006" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:54.106921 31532 consensus_queue.cc:260] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.107424 31336 raft_consensus.cc:2468] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:54.109396 31532 ts_tablet_manager.cc:1434] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:54.109529 31374 leader_election.cc:304] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:54.110260 31532 tablet_bootstrap.cc:492] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:54.111081 31615 raft_consensus.cc:2804] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.111541 31615 raft_consensus.cc:493] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.112274 31615 raft_consensus.cc:3060] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.114658 31531 tablet_bootstrap.cc:492] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:54.115291 31531 ts_tablet_manager.cc:1403] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:10:54.116631 31533 tablet_bootstrap.cc:492] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:54.117123 31533 ts_tablet_manager.cc:1403] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.004s
I20260709 13:10:54.118942 31531 raft_consensus.cc:359] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.119849 31531 raft_consensus.cc:385] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.119769 31533 raft_consensus.cc:359] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.120334 31531 raft_consensus.cc:740] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.120589 31533 raft_consensus.cc:385] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.120939 31533 raft_consensus.cc:740] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.121466 31531 consensus_queue.cc:260] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.122025 31732 raft_consensus.cc:493] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.122578 31732 raft_consensus.cc:515] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.123526 31530 tablet_bootstrap.cc:492] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:54.123983 31530 ts_tablet_manager.cc:1403] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.050s	user 0.005s	sys 0.005s
I20260709 13:10:54.124452 31716 raft_consensus.cc:493] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.125008 31716 raft_consensus.cc:515] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.126310 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6498a8fe8ab41d19cfe4afd8ccfb904" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.126725 31530 raft_consensus.cc:359] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.127382 31530 raft_consensus.cc:385] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.127650 31724 raft_consensus.cc:493] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.128196 31724 raft_consensus.cc:515] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.128546 31716 leader_election.cc:290] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:54.127676 31530 raft_consensus.cc:740] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.129546 31531 ts_tablet_manager.cc:1434] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:10:54.130342 31724 leader_election.cc:290] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.130900 31531 tablet_bootstrap.cc:492] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:54.131357 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4aaf32f729d46578d080ffe9a172eda" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.131671 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81f5f6441044395977dbda8239d7289" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.132061 31483 raft_consensus.cc:2468] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:54.132283 31260 raft_consensus.cc:2468] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
W20260709 13:10:54.131357 31228 leader_election.cc:343] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.133284 31377 leader_election.cc:304] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:54.134054 31303 leader_election.cc:304] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:54.130149 31530 consensus_queue.cc:260] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.134876 31716 raft_consensus.cc:2804] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.135042 31663 raft_consensus.cc:2804] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.135280 31716 raft_consensus.cc:493] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.135563 31663 raft_consensus.cc:493] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.135684 31716 raft_consensus.cc:3060] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.136011 31663 raft_consensus.cc:3060] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.138203 31531 tablet_bootstrap.cc:654] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.142556 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4aaf32f729d46578d080ffe9a172eda" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:54.142774 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81f5f6441044395977dbda8239d7289" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.143222 31336 raft_consensus.cc:2468] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:54.124719 31732 leader_election.cc:290] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.143344 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6498a8fe8ab41d19cfe4afd8ccfb904" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.143323 31411 raft_consensus.cc:2468] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
W20260709 13:10:54.145052 31226 leader_election.cc:343] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.145191 31716 raft_consensus.cc:515] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.145560 31226 leader_election.cc:304] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:54.146626 31732 raft_consensus.cc:2749] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.148844 31663 raft_consensus.cc:515] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.150894 31532 tablet_bootstrap.cc:654] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.151382 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4aaf32f729d46578d080ffe9a172eda" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:54.152027 31336 raft_consensus.cc:3060] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.154639 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81f5f6441044395977dbda8239d7289" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:54.155218 31260 raft_consensus.cc:3060] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.162823 31260 raft_consensus.cc:2468] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:54.164160 31303 leader_election.cc:304] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:54.165089 31727 raft_consensus.cc:2804] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.165791 31727 raft_consensus.cc:697] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:54.166344 31336 raft_consensus.cc:2468] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:54.166683 31727 consensus_queue.cc:237] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.167611 31374 leader_election.cc:304] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:54.168493 31724 raft_consensus.cc:2804] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.169453 31724 raft_consensus.cc:697] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:54.170328 31724 consensus_queue.cc:237] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.171473 31723 raft_consensus.cc:493] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.172327 31723 raft_consensus.cc:515] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.174288 31716 leader_election.cc:290] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:54.175309 31723 leader_election.cc:290] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:54.176981 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81f5f6441044395977dbda8239d7289" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:54.177639 31411 raft_consensus.cc:3060] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.178511 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.179040 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.179198 31262 raft_consensus.cc:2468] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
W20260709 13:10:54.180511 31377 leader_election.cc:343] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.181408 31377 leader_election.cc:304] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:54.182591 31723 raft_consensus.cc:2804] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.183002 31723 raft_consensus.cc:493] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.183364 31723 raft_consensus.cc:3060] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.185890 31411 raft_consensus.cc:2468] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:54.190348 31723 raft_consensus.cc:515] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.194175 31530 ts_tablet_manager.cc:1434] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.001s
I20260709 13:10:54.195340 31530 tablet_bootstrap.cc:492] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:54.197050 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:54.197487 31723 leader_election.cc:290] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:54.197777 31262 raft_consensus.cc:3060] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.198616 31663 leader_election.cc:290] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.197108 31533 consensus_queue.cc:260] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.197050 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:54.201829 31663 consensus_queue.cc:1048] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:54.203626 31377 leader_election.cc:343] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.204386 31533 ts_tablet_manager.cc:1434] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.087s	user 0.007s	sys 0.000s
I20260709 13:10:54.214061 31533 tablet_bootstrap.cc:492] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:54.217859 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4aaf32f729d46578d080ffe9a172eda" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:54.218642 31483 raft_consensus.cc:3060] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.226747 31262 raft_consensus.cc:2468] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:54.229043 31377 leader_election.cc:304] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:54.230854 31724 raft_consensus.cc:2804] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.231431 31724 raft_consensus.cc:697] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:54.232219 31724 consensus_queue.cc:237] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.243727 31615 raft_consensus.cc:515] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.249172 31662 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.254356 31662 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.257449 31615 leader_election.cc:290] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.258774 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4be50f8b2fe4ae4a303702831a3f006" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:54.259313 31732 raft_consensus.cc:493] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.259795 31732 raft_consensus.cc:515] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.261530 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.262481 31262 raft_consensus.cc:2393] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 75b6f0017ce84a4c9822ffd27e053405 in this term.
I20260709 13:10:54.261965 31160 catalog_manager.cc:5697] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.264008 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4be50f8b2fe4ae4a303702831a3f006" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:54.261716 31732 leader_election.cc:290] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:54.265074 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.259341 31336 raft_consensus.cc:3060] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:54.266117 31377 leader_election.cc:343] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.265440 31483 raft_consensus.cc:2468] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:54.267835 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d10f77e65c14720a8312b1dd44d3199" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.261487 31533 tablet_bootstrap.cc:654] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:54.269389 31228 leader_election.cc:343] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.265851 31411 raft_consensus.cc:2393] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:54.271035 31449 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:54.271831 31742 raft_consensus.cc:3060] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.265635 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d10f77e65c14720a8312b1dd44d3199" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:54.273666 31723 consensus_queue.cc:1048] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:54.273928 31225 leader_election.cc:343] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.274394 31225 leader_election.cc:304] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:54.275409 31732 raft_consensus.cc:2749] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.279582 31530 tablet_bootstrap.cc:654] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.286054 31336 raft_consensus.cc:2468] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:54.257251 31662 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:54.287688 31374 leader_election.cc:304] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:54.290063 31723 raft_consensus.cc:2804] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.290711 31532 tablet_bootstrap.cc:492] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:54.290647 31723 raft_consensus.cc:697] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:54.291404 31532 ts_tablet_manager.cc:1403] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.181s	user 0.015s	sys 0.006s
I20260709 13:10:54.292217 31723 consensus_queue.cc:237] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.292819 31662 raft_consensus.cc:493] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.294111 31662 raft_consensus.cc:515] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.295360 31742 raft_consensus.cc:2749] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.297681 31531 tablet_bootstrap.cc:492] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:54.298135 31531 ts_tablet_manager.cc:1403] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.168s	user 0.043s	sys 0.045s
I20260709 13:10:54.299676 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077ed73927a64748baae88b0b8dc4a13" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.300439 31260 raft_consensus.cc:2468] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:54.301496 31662 leader_election.cc:290] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:54.306186 31451 leader_election.cc:304] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:54.308450 31662 raft_consensus.cc:2804] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.308791 31662 raft_consensus.cc:493] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.309110 31662 raft_consensus.cc:3060] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.313249 31532 raft_consensus.cc:359] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.313915 31532 raft_consensus.cc:385] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.314183 31532 raft_consensus.cc:740] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.314930 31532 consensus_queue.cc:260] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.317058 31532 ts_tablet_manager.cc:1434] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:10:54.303169 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077ed73927a64748baae88b0b8dc4a13" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:54.319017 31336 raft_consensus.cc:2468] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:54.319465 31160 catalog_manager.cc:5697] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.321933 31531 raft_consensus.cc:359] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.322377 31532 tablet_bootstrap.cc:492] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:54.338650 31160 catalog_manager.cc:5697] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.339672 31532 tablet_bootstrap.cc:654] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.341024 31716 consensus_queue.cc:1048] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.343472 31533 tablet_bootstrap.cc:492] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:54.343979 31533 ts_tablet_manager.cc:1403] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.138s	user 0.012s	sys 0.008s
I20260709 13:10:54.317031 31662 raft_consensus.cc:515] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.334121 31531 raft_consensus.cc:385] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.358800 31531 raft_consensus.cc:740] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.359810 31531 consensus_queue.cc:260] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.362067 31531 ts_tablet_manager.cc:1434] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.064s	user 0.003s	sys 0.003s
I20260709 13:10:54.346689 31533 raft_consensus.cc:359] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.364760 31533 raft_consensus.cc:385] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.366636 31533 raft_consensus.cc:740] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.366351 31530 tablet_bootstrap.cc:492] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:54.367626 31530 ts_tablet_manager.cc:1403] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.173s	user 0.010s	sys 0.010s
I20260709 13:10:54.367774 31156 catalog_manager.cc:5697] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.369354 31531 tablet_bootstrap.cc:492] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:54.369930 31662 leader_election.cc:290] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:54.370883 31533 consensus_queue.cc:260] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.379892 31533 ts_tablet_manager.cc:1434] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.000s
I20260709 13:10:54.377384 31530 raft_consensus.cc:359] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.372448 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077ed73927a64748baae88b0b8dc4a13" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:54.378655 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077ed73927a64748baae88b0b8dc4a13" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:54.381132 31335 raft_consensus.cc:3060] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.381417 31727 raft_consensus.cc:493] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.381958 31727 raft_consensus.cc:515] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.383055 31732 raft_consensus.cc:493] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.383692 31732 raft_consensus.cc:515] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.384701 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.385236 31530 raft_consensus.cc:385] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.388799 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c9185858d64cfca85bfe148ea6bef0" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.389021 31530 raft_consensus.cc:740] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.389904 31530 consensus_queue.cc:260] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.386126 31727 leader_election.cc:290] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:54.386922 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.387466 31732 leader_election.cc:290] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.388932 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c9185858d64cfca85bfe148ea6bef0" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:54.390290 31226 leader_election.cc:343] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:54.395859 31228 leader_election.cc:343] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.396173 31532 tablet_bootstrap.cc:492] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:54.396780 31532 ts_tablet_manager.cc:1403] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.000s
I20260709 13:10:54.397894 31228 leader_election.cc:304] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:54.381402 31260 raft_consensus.cc:3060] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.399341 31732 raft_consensus.cc:2749] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.391021 31262 raft_consensus.cc:2468] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:54.391285 31533 tablet_bootstrap.cc:492] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
W20260709 13:10:54.393234 31303 leader_election.cc:343] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.405725 31531 tablet_bootstrap.cc:654] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.400180 31532 raft_consensus.cc:359] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.405993 31303 leader_election.cc:304] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:54.406349 31532 raft_consensus.cc:385] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.406692 31532 raft_consensus.cc:740] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.408092 31716 raft_consensus.cc:2804] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.408488 31716 raft_consensus.cc:493] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.408699 31533 tablet_bootstrap.cc:654] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.408866 31716 raft_consensus.cc:3060] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.412093 31532 consensus_queue.cc:260] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.414417 31532 ts_tablet_manager.cc:1434] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.004s
I20260709 13:10:54.415998 31532 tablet_bootstrap.cc:492] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:54.417830 31530 ts_tablet_manager.cc:1434] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 13:10:54.431536 31530 tablet_bootstrap.cc:492] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:54.422205 31531 tablet_bootstrap.cc:492] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:54.432142 31531 ts_tablet_manager.cc:1403] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.063s	user 0.014s	sys 0.000s
I20260709 13:10:54.433943 31782 raft_consensus.cc:493] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.434407 31782 raft_consensus.cc:515] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.434836 31531 raft_consensus.cc:359] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.435499 31531 raft_consensus.cc:385] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.435753 31531 raft_consensus.cc:740] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.436461 31782 leader_election.cc:290] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:54.436355 31531 consensus_queue.cc:260] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.438915 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.439635 31262 raft_consensus.cc:2393] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:54.441309 31531 ts_tablet_manager.cc:1434] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:10:54.441480 31723 raft_consensus.cc:493] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.442027 31533 tablet_bootstrap.cc:492] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:54.442226 31531 tablet_bootstrap.cc:492] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:54.441958 31723 raft_consensus.cc:515] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.442473 31533 ts_tablet_manager.cc:1403] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.000s
I20260709 13:10:54.444131 31723 leader_election.cc:290] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.427173 31335 raft_consensus.cc:2468] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:54.419402 31716 raft_consensus.cc:515] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.444741 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a283218fb40d45d0a877dcd23bfc51d5" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:54.445153 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a283218fb40d45d0a877dcd23bfc51d5" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:54.446045 31374 leader_election.cc:343] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:54.446553 31377 leader_election.cc:343] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.447041 31377 leader_election.cc:304] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:54.447871 31723 raft_consensus.cc:2749] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.448951 31448 leader_election.cc:304] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:54.450417 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:54.450982 31262 raft_consensus.cc:3060] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.446503 31716 leader_election.cc:290] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:54.451597 31715 consensus_queue.cc:1048] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.451936 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:54.452975 31782 raft_consensus.cc:2804] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.453536 31782 raft_consensus.cc:697] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:10:54.454324 31782 consensus_queue.cc:237] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.455358 31533 raft_consensus.cc:359] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.456086 31530 tablet_bootstrap.cc:654] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.457552 31532 tablet_bootstrap.cc:654] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.456038 31533 raft_consensus.cc:385] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.459434 31533 raft_consensus.cc:740] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
W20260709 13:10:54.459478 31303 leader_election.cc:343] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.460321 31533 consensus_queue.cc:260] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.462623 31533 ts_tablet_manager.cc:1434] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.003s
I20260709 13:10:54.463853 31533 tablet_bootstrap.cc:492] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:54.463936 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.468176 31531 tablet_bootstrap.cc:654] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.470067 31715 raft_consensus.cc:493] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.470542 31715 raft_consensus.cc:515] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.477114 31257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29510c165a1f48b1949c6ee31e136064" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.476364 31715 leader_election.cc:290] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:54.478950 31260 raft_consensus.cc:2468] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:10:54.475597 31449 leader_election.cc:304] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:54.479856 31257 raft_consensus.cc:2468] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:54.487103 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29510c165a1f48b1949c6ee31e136064" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.492250 31741 raft_consensus.cc:3060] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:54.491767 31301 leader_election.cc:343] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.499374 31532 tablet_bootstrap.cc:492] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:54.500614 31532 ts_tablet_manager.cc:1403] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.085s	user 0.010s	sys 0.005s
I20260709 13:10:54.502228 31301 leader_election.cc:304] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:54.504073 31530 tablet_bootstrap.cc:492] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:54.504624 31715 raft_consensus.cc:2804] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.504987 31715 raft_consensus.cc:493] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.505616 31715 raft_consensus.cc:3060] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.509825 31262 raft_consensus.cc:2468] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:54.513566 31530 ts_tablet_manager.cc:1403] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.082s	user 0.017s	sys 0.005s
I20260709 13:10:54.515982 31532 raft_consensus.cc:359] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.516660 31532 raft_consensus.cc:385] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.516939 31532 raft_consensus.cc:740] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.538717 31532 consensus_queue.cc:260] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.540930 31531 tablet_bootstrap.cc:492] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:54.541550 31531 ts_tablet_manager.cc:1403] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.100s	user 0.014s	sys 0.003s
I20260709 13:10:54.549084 31533 tablet_bootstrap.cc:654] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.550120 31531 raft_consensus.cc:359] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.550773 31531 raft_consensus.cc:385] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.551007 31531 raft_consensus.cc:740] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.554641 31530 raft_consensus.cc:359] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.555745 31530 raft_consensus.cc:385] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.553869 31531 consensus_queue.cc:260] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.556140 31532 ts_tablet_manager.cc:1434] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.055s	user 0.010s	sys 0.005s
I20260709 13:10:54.556041 31530 raft_consensus.cc:740] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.557878 31532 tablet_bootstrap.cc:492] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:54.560117 31530 consensus_queue.cc:260] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.562577 31531 ts_tablet_manager.cc:1434] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.021s	user 0.002s	sys 0.005s
I20260709 13:10:54.563964 31531 tablet_bootstrap.cc:492] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:54.568619 31530 ts_tablet_manager.cc:1434] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 13:10:54.570305 31530 tablet_bootstrap.cc:492] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:54.574889 31531 tablet_bootstrap.cc:654] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.576880 31530 tablet_bootstrap.cc:654] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.591627 31715 raft_consensus.cc:515] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.597797 31532 tablet_bootstrap.cc:654] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.598690 31779 raft_consensus.cc:493] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.599203 31779 raft_consensus.cc:515] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.603755 31779 leader_election.cc:290] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:54.604287 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.605028 31262 raft_consensus.cc:2393] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:54.606868 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:54.607904 31723 consensus_queue.cc:1048] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.631177 31727 raft_consensus.cc:493] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.631641 31727 raft_consensus.cc:515] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.607921 31335 raft_consensus.cc:2393] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:54.634537 31448 leader_election.cc:304] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:54.636096 31782 raft_consensus.cc:3060] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.639446 31723 consensus_queue.cc:1048] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.644341 31793 raft_consensus.cc:493] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.644960 31793 raft_consensus.cc:515] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.668555 31716 raft_consensus.cc:493] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.676347 31716 raft_consensus.cc:515] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.678444 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.679373 31716 leader_election.cc:290] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
W20260709 13:10:54.681046 31301 leader_election.cc:343] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.674743 31727 leader_election.cc:290] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:54.685483 31531 tablet_bootstrap.cc:492] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:54.681492 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5846d47686948bcb0d921291b4bc5c9" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.686168 31531 ts_tablet_manager.cc:1403] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.122s	user 0.009s	sys 0.004s
I20260709 13:10:54.677670 31303 leader_election.cc:304] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
W20260709 13:10:54.687427 31301 leader_election.cc:343] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.686196 31793 leader_election.cc:290] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:54.689914 31727 raft_consensus.cc:2804] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.690514 31727 raft_consensus.cc:697] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:54.691568 31741 raft_consensus.cc:2749] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.691326 31727 consensus_queue.cc:237] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.694732 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5846d47686948bcb0d921291b4bc5c9" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.695245 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.694519 31715 leader_election.cc:290] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:54.697703 31530 tablet_bootstrap.cc:492] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:54.698285 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.698935 31260 raft_consensus.cc:2468] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:54.701740 31530 ts_tablet_manager.cc:1403] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.132s	user 0.028s	sys 0.045s
I20260709 13:10:54.704275 31782 raft_consensus.cc:2749] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.704501 31257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
W20260709 13:10:54.705587 31303 leader_election.cc:343] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.706040 31303 leader_election.cc:304] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
W20260709 13:10:54.707445 31303 leader_election.cc:343] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.708644 31257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29510c165a1f48b1949c6ee31e136064" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:54.709226 31257 raft_consensus.cc:3060] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.709528 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29510c165a1f48b1949c6ee31e136064" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:54.710103 31410 raft_consensus.cc:3060] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.713436 31802 raft_consensus.cc:2749] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.714387 31530 raft_consensus.cc:359] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.715044 31530 raft_consensus.cc:385] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.715304 31530 raft_consensus.cc:740] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.715960 31530 consensus_queue.cc:260] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.717175 31728 raft_consensus.cc:493] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.717689 31728 raft_consensus.cc:515] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.719818 31728 leader_election.cc:290] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:54.721127 31533 tablet_bootstrap.cc:492] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:54.721601 31533 ts_tablet_manager.cc:1403] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.258s	user 0.035s	sys 0.017s
I20260709 13:10:54.721956 31257 raft_consensus.cc:2468] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:54.724182 31533 raft_consensus.cc:359] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.725119 31533 raft_consensus.cc:385] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.725196 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b3eb1a472a546fdb51a2cc295716492" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:54.725425 31533 raft_consensus.cc:740] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.725557 31531 raft_consensus.cc:359] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
W20260709 13:10:54.726677 31225 leader_election.cc:343] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.728574 31532 tablet_bootstrap.cc:492] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:54.716452 31156 catalog_manager.cc:5697] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 1 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.731204 31303 leader_election.cc:304] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:54.733271 31716 raft_consensus.cc:2804] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.733781 31716 raft_consensus.cc:493] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.734243 31716 raft_consensus.cc:3060] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.735301 31410 raft_consensus.cc:2468] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:54.726250 31531 raft_consensus.cc:385] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.736625 31531 raft_consensus.cc:740] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.737255 31531 consensus_queue.cc:260] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.739215 31531 ts_tablet_manager.cc:1434] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.000s
I20260709 13:10:54.740091 31531 tablet_bootstrap.cc:492] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:54.743275 31716 raft_consensus.cc:515] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:10:54.745920 31303 leader_election.cc:343] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.746338 31303 leader_election.cc:304] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:54.746999 31801 raft_consensus.cc:2749] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.748325 31801 consensus_queue.cc:1048] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.726236 31533 consensus_queue.cc:260] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.729123 31532 ts_tablet_manager.cc:1403] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.172s	user 0.025s	sys 0.003s
I20260709 13:10:54.751178 31533 ts_tablet_manager.cc:1434] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.002s
I20260709 13:10:54.752039 31532 raft_consensus.cc:359] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.753520 31716 leader_election.cc:290] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:54.752233 31303 leader_election.cc:304] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:54.754657 31723 consensus_queue.cc:1048] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.756313 31723 raft_consensus.cc:493] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.756704 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:54.757133 31801 raft_consensus.cc:2804] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.761885 31801 raft_consensus.cc:697] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:54.757906 31531 tablet_bootstrap.cc:654] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.758587 31530 ts_tablet_manager.cc:1434] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 13:10:54.753707 31532 raft_consensus.cc:385] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.765472 31532 raft_consensus.cc:740] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.766122 31532 consensus_queue.cc:260] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:54.759092 31533 tablet_bootstrap.cc:492] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:54.759021 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b3eb1a472a546fdb51a2cc295716492" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.764694 31801 consensus_queue.cc:237] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.770998 31156 catalog_manager.cc:5697] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:54.773665 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
W20260709 13:10:54.775683 31301 leader_election.cc:343] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.764667 31530 tablet_bootstrap.cc:492] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:54.777616 31532 ts_tablet_manager.cc:1434] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.002s
I20260709 13:10:54.791378 31532 tablet_bootstrap.cc:492] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:54.781008 31533 tablet_bootstrap.cc:654] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:54.787163 31226 leader_election.cc:343] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.756915 31723 raft_consensus.cc:515] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.802695 31723 leader_election.cc:290] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:54.805135 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6498a8fe8ab41d19cfe4afd8ccfb904" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.805945 31485 raft_consensus.cc:2468] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:54.795428 31531 tablet_bootstrap.cc:492] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:54.789734 31262 raft_consensus.cc:3060] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.797195 31226 leader_election.cc:304] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:54.816453 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6498a8fe8ab41d19cfe4afd8ccfb904" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.817507 31728 raft_consensus.cc:2749] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.817683 31711 consensus_queue.cc:1048] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.799670 31724 consensus_queue.cc:1048] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:54.818166 31530 tablet_bootstrap.cc:654] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.809386 31531 ts_tablet_manager.cc:1403] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.069s	user 0.017s	sys 0.005s
I20260709 13:10:54.810052 31377 leader_election.cc:304] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:54.811005 31727 consensus_queue.cc:1048] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:54.817041 31258 raft_consensus.cc:2468] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:54.824805 31723 raft_consensus.cc:2804] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.825182 31723 raft_consensus.cc:493] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.825523 31723 raft_consensus.cc:3060] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.830937 31532 tablet_bootstrap.cc:654] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.841741 31262 raft_consensus.cc:2468] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:54.843146 31303 leader_election.cc:304] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:54.844007 31727 raft_consensus.cc:2804] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.844493 31727 raft_consensus.cc:697] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:54.845381 31727 consensus_queue.cc:237] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.850133 31802 raft_consensus.cc:493] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.850615 31802 raft_consensus.cc:515] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.852622 31802 leader_election.cc:290] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.854050 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a283218fb40d45d0a877dcd23bfc51d5" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:54.854746 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a283218fb40d45d0a877dcd23bfc51d5" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:54.856109 31303 leader_election.cc:343] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.854660 31411 raft_consensus.cc:2468] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:54.857664 31301 leader_election.cc:304] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:54.857762 31531 raft_consensus.cc:359] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.858807 31531 raft_consensus.cc:385] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.859025 31802 raft_consensus.cc:2804] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.859148 31531 raft_consensus.cc:740] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.859681 31802 raft_consensus.cc:493] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.860271 31802 raft_consensus.cc:3060] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.860309 31531 consensus_queue.cc:260] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.855907 31723 raft_consensus.cc:515] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.869447 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6498a8fe8ab41d19cfe4afd8ccfb904" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:54.870090 31484 raft_consensus.cc:3060] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.871503 31531 ts_tablet_manager.cc:1434] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 13:10:54.872438 31531 tablet_bootstrap.cc:492] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:54.878234 31484 raft_consensus.cc:2468] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:54.878978 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6498a8fe8ab41d19cfe4afd8ccfb904" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:54.879549 31260 raft_consensus.cc:3060] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.879428 31377 leader_election.cc:304] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:54.880311 31711 raft_consensus.cc:2804] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.881099 31711 raft_consensus.cc:697] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:54.882030 31711 consensus_queue.cc:237] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.888248 31532 tablet_bootstrap.cc:492] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:54.893879 31532 ts_tablet_manager.cc:1403] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.103s	user 0.006s	sys 0.004s
I20260709 13:10:54.899142 31532 raft_consensus.cc:359] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.903199 31532 raft_consensus.cc:385] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.890765 31723 leader_election.cc:290] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:54.892431 31533 tablet_bootstrap.cc:492] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:54.868402 31802 raft_consensus.cc:515] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.904247 31533 ts_tablet_manager.cc:1403] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.002s
I20260709 13:10:54.904433 31532 raft_consensus.cc:740] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
W20260709 13:10:54.906458 31377 consensus_peers.cc:597] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Couldn't send request to peer 001cac731610478a900ad62e830aaff6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:54.919525 31801 raft_consensus.cc:493] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.920019 31801 raft_consensus.cc:515] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:54.923144 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b3eb1a472a546fdb51a2cc295716492" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:54.923768 31262 raft_consensus.cc:2468] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:54.926913 31303 leader_election.cc:304] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:54.928735 31533 raft_consensus.cc:359] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.929440 31533 raft_consensus.cc:385] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.929683 31533 raft_consensus.cc:740] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
W20260709 13:10:54.910725 31207 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:54.929898 31727 raft_consensus.cc:2804] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.932449 31727 raft_consensus.cc:493] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.933794 31727 raft_consensus.cc:3060] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.933404 31801 leader_election.cc:290] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:54.933080 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b3eb1a472a546fdb51a2cc295716492" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
W20260709 13:10:54.938720 31301 leader_election.cc:343] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.941192 31741 consensus_queue.cc:1048] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:54.945086 31531 tablet_bootstrap.cc:654] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.946805 31532 consensus_queue.cc:260] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.948007 31533 consensus_queue.cc:260] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:54.955142 31532 ts_tablet_manager.cc:1434] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 13:10:54.955842 31532 tablet_bootstrap.cc:492] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:54.956840 31533 ts_tablet_manager.cc:1434] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.001s
I20260709 13:10:54.957720 31533 tablet_bootstrap.cc:492] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:54.961018 31156 catalog_manager.cc:5697] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.963532 31728 raft_consensus.cc:493] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.976681 31530 tablet_bootstrap.cc:492] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:54.978752 31530 ts_tablet_manager.cc:1403] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.214s	user 0.023s	sys 0.016s
I20260709 13:10:54.979982 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a283218fb40d45d0a877dcd23bfc51d5" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:54.980556 31410 raft_consensus.cc:3060] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.981338 31530 raft_consensus.cc:359] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.981997 31530 raft_consensus.cc:385] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.982251 31530 raft_consensus.cc:740] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.982510 31533 tablet_bootstrap.cc:654] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.982877 31530 consensus_queue.cc:260] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.985468 31530 ts_tablet_manager.cc:1434] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:54.975332 31728 raft_consensus.cc:515] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:54.986315 31530 tablet_bootstrap.cc:492] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:54.988533 31728 leader_election.cc:290] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.989705 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271f7357440943008e55795b0d6cbbda" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:54.990774 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271f7357440943008e55795b0d6cbbda" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
W20260709 13:10:54.991066 31228 leader_election.cc:343] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.999241 31530 tablet_bootstrap.cc:654] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.011425 31727 raft_consensus.cc:515] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.016623 31727 leader_election.cc:290] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:55.012604 31802 leader_election.cc:290] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:54.995018 31260 raft_consensus.cc:2468] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:55.021310 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b3eb1a472a546fdb51a2cc295716492" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:55.022301 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b3eb1a472a546fdb51a2cc295716492" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:55.030341 31258 raft_consensus.cc:3060] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:55.031147 31301 leader_election.cc:343] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:55.010106 31226 leader_election.cc:343] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.028612 31410 raft_consensus.cc:2468] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:55.018265 31157 catalog_manager.cc:5697] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:55.045022 31532 tablet_bootstrap.cc:654] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.044307 31779 raft_consensus.cc:493] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.045888 31779 raft_consensus.cc:515] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.048022 31779 leader_election.cc:290] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:55.049017 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4be50f8b2fe4ae4a303702831a3f006" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:55.054301 31301 leader_election.cc:304] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:55.055105 31530 tablet_bootstrap.cc:492] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:55.054898 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a283218fb40d45d0a877dcd23bfc51d5" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:55.058969 31727 raft_consensus.cc:2804] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.059047 31530 ts_tablet_manager.cc:1403] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.073s	user 0.029s	sys 0.007s
I20260709 13:10:55.059506 31727 raft_consensus.cc:697] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:55.060360 31727 consensus_queue.cc:237] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.042073 31226 leader_election.cc:304] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
W20260709 13:10:55.062052 31226 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:45117, user_credentials={real_user=slave}} blocked reactor thread for 56362.8us
I20260709 13:10:55.062397 31728 raft_consensus.cc:2749] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.065299 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4be50f8b2fe4ae4a303702831a3f006" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:55.069136 31448 leader_election.cc:304] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:55.069195 31741 consensus_queue.cc:1048] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.071611 31741 raft_consensus.cc:2749] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:55.077088 31303 leader_election.cc:343] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.083956 31530 raft_consensus.cc:359] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.085619 31530 raft_consensus.cc:385] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.089005 31530 raft_consensus.cc:740] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.090226 31532 tablet_bootstrap.cc:492] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:55.090720 31532 ts_tablet_manager.cc:1403] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.135s	user 0.012s	sys 0.032s
I20260709 13:10:55.087404 31258 raft_consensus.cc:2468] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:55.093475 31532 raft_consensus.cc:359] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.094127 31532 raft_consensus.cc:385] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.094388 31532 raft_consensus.cc:740] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.094269 31303 leader_election.cc:304] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:55.095147 31532 consensus_queue.cc:260] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.095579 31793 raft_consensus.cc:2804] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.096163 31793 raft_consensus.cc:697] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:55.097082 31532 ts_tablet_manager.cc:1434] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:55.097985 31532 tablet_bootstrap.cc:492] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:55.097198 31793 consensus_queue.cc:237] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.090261 31530 consensus_queue.cc:260] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.107146 31531 tablet_bootstrap.cc:492] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:55.107656 31531 ts_tablet_manager.cc:1403] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.235s	user 0.015s	sys 0.010s
I20260709 13:10:55.110225 31531 raft_consensus.cc:359] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.119647 31532 tablet_bootstrap.cc:654] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.123498 31533 tablet_bootstrap.cc:492] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:55.123404 31723 raft_consensus.cc:493] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.123454 31802 consensus_queue.cc:1048] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:55.124248 31533 ts_tablet_manager.cc:1403] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.167s	user 0.008s	sys 0.012s
I20260709 13:10:55.124210 31723 raft_consensus.cc:515] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.126416 31723 leader_election.cc:290] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:55.111253 31530 ts_tablet_manager.cc:1434] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 13:10:55.129143 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c9185858d64cfca85bfe148ea6bef0" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:55.130815 31377 leader_election.cc:343] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.132088 31530 tablet_bootstrap.cc:492] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:55.131973 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c9185858d64cfca85bfe148ea6bef0" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:55.132727 31258 raft_consensus.cc:2468] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:55.113449 31531 raft_consensus.cc:385] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.133811 31732 raft_consensus.cc:493] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.134069 31532 tablet_bootstrap.cc:492] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:55.134606 31532 ts_tablet_manager.cc:1403] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.000s
I20260709 13:10:55.134394 31732 raft_consensus.cc:515] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.137450 31532 raft_consensus.cc:359] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.138095 31532 raft_consensus.cc:385] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.138334 31532 raft_consensus.cc:740] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.136667 31732 leader_election.cc:290] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:55.139041 31532 consensus_queue.cc:260] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.140125 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:55.141508 31532 ts_tablet_manager.cc:1434] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:55.142370 31532 tablet_bootstrap.cc:492] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:55.144685 31530 tablet_bootstrap.cc:654] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.144520 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:55.145574 31334 raft_consensus.cc:2468] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
W20260709 13:10:55.149928 31228 leader_election.cc:343] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.150714 31377 leader_election.cc:304] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:55.151800 31723 raft_consensus.cc:2804] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.152140 31723 raft_consensus.cc:493] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.152452 31723 raft_consensus.cc:3060] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.153641 31156 catalog_manager.cc:5697] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:55.155296 31156 catalog_manager.cc:5697] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:55.158350 31225 leader_election.cc:304] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:55.159976 31732 raft_consensus.cc:2804] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.160434 31732 raft_consensus.cc:493] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.160854 31732 raft_consensus.cc:3060] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.163650 31532 tablet_bootstrap.cc:654] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.181638 31663 raft_consensus.cc:493] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.182137 31663 raft_consensus.cc:515] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.184226 31663 leader_election.cc:290] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:55.185675 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5846d47686948bcb0d921291b4bc5c9" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:55.187820 31335 raft_consensus.cc:2468] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:55.189056 31374 leader_election.cc:304] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:55.189343 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5846d47686948bcb0d921291b4bc5c9" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:55.189976 31663 raft_consensus.cc:2804] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.190346 31663 raft_consensus.cc:493] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:55.190670 31377 leader_election.cc:343] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.190687 31663 raft_consensus.cc:3060] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.191454 31533 raft_consensus.cc:359] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.192178 31533 raft_consensus.cc:385] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.192481 31533 raft_consensus.cc:740] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.192711 31160 catalog_manager.cc:5697] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:55.194041 31727 consensus_queue.cc:1048] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.199012 31663 raft_consensus.cc:515] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.182242 31723 raft_consensus.cc:515] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.203866 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5846d47686948bcb0d921291b4bc5c9" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:55.204483 31334 raft_consensus.cc:3060] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.205546 31530 tablet_bootstrap.cc:492] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:55.205992 31530 ts_tablet_manager.cc:1403] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.075s	user 0.008s	sys 0.019s
I20260709 13:10:55.208339 31530 raft_consensus.cc:359] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.208806 31530 raft_consensus.cc:385] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.209060 31530 raft_consensus.cc:740] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.133850 31531 raft_consensus.cc:740] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.211823 31723 leader_election.cc:290] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:55.214250 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c9185858d64cfca85bfe148ea6bef0" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:55.194934 31732 raft_consensus.cc:515] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.214918 31260 raft_consensus.cc:3060] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.216360 31732 leader_election.cc:290] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:55.213292 31530 consensus_queue.cc:260] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.217706 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c9185858d64cfca85bfe148ea6bef0" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:55.217859 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
W20260709 13:10:55.219146 31377 leader_election.cc:343] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.219476 31728 raft_consensus.cc:493] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.193133 31533 consensus_queue.cc:260] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.220031 31728 raft_consensus.cc:515] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.222159 31533 ts_tablet_manager.cc:1434] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.000s
I20260709 13:10:55.222999 31533 tablet_bootstrap.cc:492] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
W20260709 13:10:55.223634 31228 leader_election.cc:343] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.214293 31531 consensus_queue.cc:260] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.225435 31530 ts_tablet_manager.cc:1434] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 13:10:55.226419 31530 tablet_bootstrap.cc:492] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:55.227283 31532 tablet_bootstrap.cc:492] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:55.227725 31532 ts_tablet_manager.cc:1403] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.086s	user 0.003s	sys 0.008s
I20260709 13:10:55.229715 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:55.230337 31335 raft_consensus.cc:3060] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.229856 31728 leader_election.cc:290] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:55.232673 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "862d691f6d9e4d9fb185be53b562b8ff" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:55.233349 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "862d691f6d9e4d9fb185be53b562b8ff" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:55.233937 31410 raft_consensus.cc:2468] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
I20260709 13:10:55.235060 31226 leader_election.cc:304] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:55.235862 31728 raft_consensus.cc:2804] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.236215 31728 raft_consensus.cc:493] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.236547 31728 raft_consensus.cc:3060] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.238494 31335 raft_consensus.cc:2468] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:55.238850 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5846d47686948bcb0d921291b4bc5c9" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:55.239638 31225 leader_election.cc:304] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
W20260709 13:10:55.240777 31228 leader_election.cc:343] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:55.242575 31377 leader_election.cc:343] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.240749 31732 raft_consensus.cc:2804] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.244490 31728 raft_consensus.cc:515] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.246022 31530 tablet_bootstrap.cc:654] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.246609 31334 raft_consensus.cc:2468] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:55.247134 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "862d691f6d9e4d9fb185be53b562b8ff" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:55.247743 31410 raft_consensus.cc:3060] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.252221 31663 leader_election.cc:290] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:55.253732 31374 leader_election.cc:304] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:55.256616 31260 raft_consensus.cc:2468] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:55.257211 31532 raft_consensus.cc:359] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.257947 31532 raft_consensus.cc:385] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.257824 31377 leader_election.cc:304] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:55.258262 31532 raft_consensus.cc:740] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.258980 31532 consensus_queue.cc:260] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.256490 31410 raft_consensus.cc:2468] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:55.261823 31226 leader_election.cc:304] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:55.265075 31663 raft_consensus.cc:2804] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.272679 31663 raft_consensus.cc:697] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:55.266326 31732 raft_consensus.cc:697] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:10:55.273738 31663 consensus_queue.cc:237] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.274960 31732 consensus_queue.cc:237] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.282382 31531 ts_tablet_manager.cc:1434] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.174s	user 0.005s	sys 0.002s
I20260709 13:10:55.285241 31531 tablet_bootstrap.cc:492] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:55.266929 31723 raft_consensus.cc:2804] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.287250 31723 raft_consensus.cc:697] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:55.288076 31723 consensus_queue.cc:237] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.298311 31532 ts_tablet_manager.cc:1434] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.070s	user 0.013s	sys 0.008s
I20260709 13:10:55.301226 31532 tablet_bootstrap.cc:492] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:55.304664 31727 consensus_queue.cc:1048] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:10:55.297713 31728 leader_election.cc:290] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:55.325419 31533 tablet_bootstrap.cc:654] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.327044 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "862d691f6d9e4d9fb185be53b562b8ff" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:55.315106 31732 raft_consensus.cc:2804] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.338573 31732 raft_consensus.cc:697] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:10:55.340162 31732 consensus_queue.cc:237] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
W20260709 13:10:55.341110 31301 consensus_peers.cc:597] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.347386 31728 raft_consensus.cc:493] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.347862 31728 raft_consensus.cc:515] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
W20260709 13:10:55.351902 31228 leader_election.cc:343] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.354560 31531 tablet_bootstrap.cc:654] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.355855 31728 leader_election.cc:290] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:55.357475 31530 tablet_bootstrap.cc:492] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:55.357949 31530 ts_tablet_manager.cc:1403] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.132s	user 0.016s	sys 0.035s
I20260709 13:10:55.344954 31156 catalog_manager.cc:5697] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:55.358742 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d905b9b08a224fe09df30744558819e3" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:55.360464 31530 raft_consensus.cc:359] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.361127 31530 raft_consensus.cc:385] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.362579 31530 raft_consensus.cc:740] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
W20260709 13:10:55.361037 31228 leader_election.cc:343] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.366658 31530 consensus_queue.cc:260] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.386020 31530 ts_tablet_manager.cc:1434] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:10:55.387284 31530 tablet_bootstrap.cc:492] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:55.397409 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d905b9b08a224fe09df30744558819e3" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:55.404186 31157 catalog_manager.cc:5697] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:55.406332 31724 raft_consensus.cc:493] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.406807 31724 raft_consensus.cc:515] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.409948 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:55.410686 31262 raft_consensus.cc:2393] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:55.410825 31533 tablet_bootstrap.cc:492] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:55.411347 31533 ts_tablet_manager.cc:1403] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.189s	user 0.014s	sys 0.004s
I20260709 13:10:55.415165 31533 raft_consensus.cc:359] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.415798 31533 raft_consensus.cc:385] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.416072 31533 raft_consensus.cc:740] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.416764 31533 consensus_queue.cc:260] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.418840 31533 ts_tablet_manager.cc:1434] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:55.419562 31532 tablet_bootstrap.cc:654] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.419939 31533 tablet_bootstrap.cc:492] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:55.424837 31834 raft_consensus.cc:493] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.425375 31834 raft_consensus.cc:515] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.427670 31834 leader_election.cc:290] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:55.428303 31530 tablet_bootstrap.cc:654] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:55.430089 31833 log.cc:927] Time spent T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:10:55.434391 31724 leader_election.cc:290] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
W20260709 13:10:55.436386 31225 leader_election.cc:343] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.441422 31225 leader_election.cc:304] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:55.438684 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:55.443219 31728 raft_consensus.cc:2749] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.439013 31793 consensus_queue.cc:1048] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.447224 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:55.448213 31793 consensus_queue.cc:1048] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:55.454638 31728 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.455117 31728 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.462357 31728 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:55.466562 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:55.463649 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:55.455615 31483 raft_consensus.cc:3060] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:55.467514 31226 leader_election.cc:343] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.471475 31374 leader_election.cc:304] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:55.472761 31724 raft_consensus.cc:3060] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:55.472879 31225 leader_election.cc:343] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.473899 31225 leader_election.cc:304] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:55.467494 31485 raft_consensus.cc:2468] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:55.475564 31728 raft_consensus.cc:2749] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.479894 31531 tablet_bootstrap.cc:492] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:55.480350 31531 ts_tablet_manager.cc:1403] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.197s	user 0.009s	sys 0.015s
I20260709 13:10:55.481757 31724 raft_consensus.cc:2749] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.483098 31531 raft_consensus.cc:359] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.483762 31531 raft_consensus.cc:385] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.484022 31531 raft_consensus.cc:740] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.484653 31531 consensus_queue.cc:260] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.486258 31711 consensus_queue.cc:1048] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.487987 31531 ts_tablet_manager.cc:1434] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:55.488957 31531 tablet_bootstrap.cc:492] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:55.493418 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:55.494076 31411 raft_consensus.cc:2468] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:55.491411 31228 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:55.498450 31834 raft_consensus.cc:2804] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:55.498809 31834 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.499153 31834 raft_consensus.cc:3060] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.503351 31531 tablet_bootstrap.cc:654] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.519424 31530 tablet_bootstrap.cc:492] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:55.519907 31530 ts_tablet_manager.cc:1403] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.133s	user 0.011s	sys 0.014s
I20260709 13:10:55.520365 31160 catalog_manager.cc:5697] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.521785 31159 catalog_manager.cc:5697] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:55.512856 31724 consensus_queue.cc:1048] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:10:55.527930 31532 tablet_bootstrap.cc:492] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:55.528391 31532 ts_tablet_manager.cc:1403] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.229s	user 0.018s	sys 0.012s
I20260709 13:10:55.533984 31530 raft_consensus.cc:359] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.534716 31530 raft_consensus.cc:385] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.534999 31530 raft_consensus.cc:740] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.540179 31533 tablet_bootstrap.cc:654] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.541065 31530 consensus_queue.cc:260] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.543365 31530 ts_tablet_manager.cc:1434] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:10:55.544340 31530 tablet_bootstrap.cc:492] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:55.550575 31530 tablet_bootstrap.cc:654] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.552004 31801 raft_consensus.cc:493] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.564848 31532 raft_consensus.cc:359] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.565570 31532 raft_consensus.cc:385] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.565847 31532 raft_consensus.cc:740] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.566488 31532 consensus_queue.cc:260] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.568571 31532 ts_tablet_manager.cc:1434] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.040s	user 0.003s	sys 0.002s
I20260709 13:10:55.569535 31532 tablet_bootstrap.cc:492] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:55.570056 31727 consensus_queue.cc:1048] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.552500 31801 raft_consensus.cc:515] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.575723 31532 tablet_bootstrap.cc:654] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.576548 31801 leader_election.cc:290] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:55.548869 31834 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.578950 31801 raft_consensus.cc:493] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.589685 31741 raft_consensus.cc:493] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.589182 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bc6ebfdfef4b7a81d4cbc17a86d70f" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:55.590178 31741 raft_consensus.cc:515] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.592535 31741 leader_election.cc:290] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:55.593219 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a283218fb40d45d0a877dcd23bfc51d5" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:55.597143 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a283218fb40d45d0a877dcd23bfc51d5" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:55.598238 31409 raft_consensus.cc:2393] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:55.598253 31728 raft_consensus.cc:493] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.598945 31728 raft_consensus.cc:515] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.588315 31801 raft_consensus.cc:515] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.604249 31801 leader_election.cc:290] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
W20260709 13:10:55.609118 31303 leader_election.cc:343] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.612069 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bc6ebfdfef4b7a81d4cbc17a86d70f" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:55.604321 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9baba29a028498288a27243e555fb27" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:55.622326 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9baba29a028498288a27243e555fb27" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:55.600955 31728 leader_election.cc:290] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
W20260709 13:10:55.631963 31225 leader_election.cc:343] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.635314 31793 consensus_queue.cc:1048] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:55.637939 31228 leader_election.cc:343] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.638608 31228 leader_election.cc:304] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:55.640394 31732 raft_consensus.cc:2749] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.644263 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:55.644881 31484 raft_consensus.cc:3060] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.640410 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d10f77e65c14720a8312b1dd44d3199" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:55.645959 31260 raft_consensus.cc:2468] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:55.647068 31303 leader_election.cc:304] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:55.648581 31801 raft_consensus.cc:2804] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.648923 31801 raft_consensus.cc:493] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.649221 31801 raft_consensus.cc:3060] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.630013 31834 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
W20260709 13:10:55.654095 31301 leader_election.cc:343] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.654572 31301 leader_election.cc:304] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:55.653083 31484 raft_consensus.cc:2468] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:10:55.655679 31802 raft_consensus.cc:2749] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.656311 31533 tablet_bootstrap.cc:492] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:55.656459 31228 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:55.658020 31533 ts_tablet_manager.cc:1403] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.238s	user 0.006s	sys 0.013s
I20260709 13:10:55.660661 31533 raft_consensus.cc:359] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.661365 31533 raft_consensus.cc:385] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.661841 31533 raft_consensus.cc:740] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.662662 31533 consensus_queue.cc:260] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.661499 31834 raft_consensus.cc:2804] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:55.670377 31533 ts_tablet_manager.cc:1434] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:10:55.671764 31533 tablet_bootstrap.cc:492] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:55.675311 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:55.675895 31410 raft_consensus.cc:3060] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.678176 31663 raft_consensus.cc:493] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.678651 31663 raft_consensus.cc:515] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.680532 31663 leader_election.cc:290] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:55.681864 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271f7357440943008e55795b0d6cbbda" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:55.686408 31377 leader_election.cc:343] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.686368 31410 raft_consensus.cc:2468] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:10:55.687390 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271f7357440943008e55795b0d6cbbda" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:55.672897 31834 raft_consensus.cc:697] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:10:55.688261 31262 raft_consensus.cc:2468] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:55.691968 31531 tablet_bootstrap.cc:492] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:55.692590 31531 ts_tablet_manager.cc:1403] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.204s	user 0.013s	sys 0.016s
I20260709 13:10:55.693853 31377 leader_election.cc:304] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:55.694393 31852 raft_consensus.cc:493] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.694830 31852 raft_consensus.cc:515] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.695469 31711 raft_consensus.cc:2804] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.695838 31711 raft_consensus.cc:493] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.696156 31711 raft_consensus.cc:3060] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.698616 31484 raft_consensus.cc:3060] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.698768 31448 leader_election.cc:304] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:55.701257 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5846d47686948bcb0d921291b4bc5c9" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:55.701773 31852 leader_election.cc:290] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:55.702010 31335 raft_consensus.cc:2393] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:55.698643 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d10f77e65c14720a8312b1dd44d3199" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:55.702961 31483 raft_consensus.cc:2468] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:55.703245 31801 raft_consensus.cc:515] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.702503 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5846d47686948bcb0d921291b4bc5c9" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:55.712772 31531 raft_consensus.cc:359] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.713238 31531 raft_consensus.cc:385] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.713631 31531 raft_consensus.cc:740] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.720031 31532 tablet_bootstrap.cc:492] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:55.720507 31532 ts_tablet_manager.cc:1403] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.151s	user 0.009s	sys 0.006s
I20260709 13:10:55.688716 31834 consensus_queue.cc:237] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.723753 31728 consensus_queue.cc:1048] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.726177 31531 consensus_queue.cc:260] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.728225 31448 leader_election.cc:304] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:55.729362 31852 raft_consensus.cc:3060] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.731735 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d10f77e65c14720a8312b1dd44d3199" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:55.732429 31483 raft_consensus.cc:3060] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.733599 31532 raft_consensus.cc:359] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.734325 31532 raft_consensus.cc:385] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.734658 31532 raft_consensus.cc:740] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.735386 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d10f77e65c14720a8312b1dd44d3199" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:55.735477 31532 consensus_queue.cc:260] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.736106 31261 raft_consensus.cc:3060] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.737845 31532 ts_tablet_manager.cc:1434] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:10:55.738740 31852 raft_consensus.cc:2749] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.739785 31532 tablet_bootstrap.cc:492] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:55.740417 31530 tablet_bootstrap.cc:492] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:55.743871 31724 consensus_queue.cc:1048] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.747079 31483 raft_consensus.cc:2468] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:55.749150 31742 raft_consensus.cc:2749] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.750949 31530 ts_tablet_manager.cc:1403] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.207s	user 0.020s	sys 0.015s
I20260709 13:10:55.758451 31530 raft_consensus.cc:359] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.759117 31530 raft_consensus.cc:385] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.759403 31530 raft_consensus.cc:740] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.760041 31530 consensus_queue.cc:260] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:55.762346 31530 ts_tablet_manager.cc:1434] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 13:10:55.754679 31801 leader_election.cc:290] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:55.770017 31531 ts_tablet_manager.cc:1434] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.077s	user 0.011s	sys 0.000s
I20260709 13:10:55.778612 31530 tablet_bootstrap.cc:492] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:55.771881 31716 consensus_queue.cc:1048] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.784034 31303 leader_election.cc:304] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:55.786753 31716 raft_consensus.cc:2804] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.787225 31716 raft_consensus.cc:697] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:55.788009 31716 consensus_queue.cc:237] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.792466 31261 raft_consensus.cc:2468] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
W20260709 13:10:55.774497 31301 consensus_peers.cc:597] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:55.797103 31377 consensus_peers.cc:597] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Couldn't send request to peer 001cac731610478a900ad62e830aaff6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.799418 31530 tablet_bootstrap.cc:654] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.801353 31663 consensus_queue.cc:1048] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:55.803326 31228 consensus_peers.cc:597] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 -> Peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Couldn't send request to peer 001cac731610478a900ad62e830aaff6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.806041 31156 catalog_manager.cc:5697] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 0 to 2, leader changed from <none> to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.772557 31533 tablet_bootstrap.cc:654] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.779040 31531 tablet_bootstrap.cc:492] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:55.795473 31532 tablet_bootstrap.cc:654] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.771565 31711 raft_consensus.cc:515] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.846030 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271f7357440943008e55795b0d6cbbda" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
W20260709 13:10:55.847534 31377 leader_election.cc:343] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.857450 31532 tablet_bootstrap.cc:492] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:55.857960 31532 ts_tablet_manager.cc:1403] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.118s	user 0.013s	sys 0.004s
I20260709 13:10:55.876199 31531 tablet_bootstrap.cc:654] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.877619 31711 leader_election.cc:290] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:55.879315 31533 tablet_bootstrap.cc:492] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:55.881668 31728 consensus_queue.cc:1048] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:55.883128 31533 ts_tablet_manager.cc:1403] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.212s	user 0.011s	sys 0.000s
I20260709 13:10:55.886895 31533 raft_consensus.cc:359] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.887843 31851 consensus_queue.cc:1048] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.889070 31532 raft_consensus.cc:359] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.890046 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271f7357440943008e55795b0d6cbbda" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:55.898579 31261 raft_consensus.cc:3060] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:55.895535 31228 consensus_peers.cc:597] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 -> Peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Couldn't send request to peer 001cac731610478a900ad62e830aaff6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.906920 31532 raft_consensus.cc:385] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.907233 31532 raft_consensus.cc:740] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.907902 31532 consensus_queue.cc:260] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.911190 31533 raft_consensus.cc:385] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.911484 31533 raft_consensus.cc:740] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.913167 31533 consensus_queue.cc:260] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.916850 31261 raft_consensus.cc:2468] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:55.924736 31533 ts_tablet_manager.cc:1434] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.000s
I20260709 13:10:55.921722 31377 leader_election.cc:304] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:55.940258 31851 raft_consensus.cc:2804] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.940794 31851 raft_consensus.cc:697] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:55.941633 31851 consensus_queue.cc:237] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:55.952630 31532 ts_tablet_manager.cc:1434] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.000s
I20260709 13:10:55.957333 31530 tablet_bootstrap.cc:492] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:55.958837 31530 ts_tablet_manager.cc:1403] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.196s	user 0.021s	sys 0.011s
I20260709 13:10:55.961620 31530 raft_consensus.cc:359] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.965102 31716 raft_consensus.cc:493] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.966713 31716 raft_consensus.cc:515] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.968756 31716 leader_election.cc:290] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:55.966270 31532 tablet_bootstrap.cc:492] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:55.969717 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f564510ff30e42ec890531c82405946c" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:55.970063 31530 raft_consensus.cc:385] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.970213 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f564510ff30e42ec890531c82405946c" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:55.970445 31530 raft_consensus.cc:740] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.971309 31531 tablet_bootstrap.cc:492] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:55.971807 31531 ts_tablet_manager.cc:1403] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.193s	user 0.008s	sys 0.025s
W20260709 13:10:55.972131 31303 leader_election.cc:343] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.971312 31530 consensus_queue.cc:260] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.973016 31411 raft_consensus.cc:2468] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:55.974552 31531 raft_consensus.cc:359] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.975541 31531 raft_consensus.cc:385] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.975823 31531 raft_consensus.cc:740] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.976524 31531 consensus_queue.cc:260] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:55.977061 31301 leader_election.cc:304] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:55.978890 31160 catalog_manager.cc:5697] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.980033 31860 raft_consensus.cc:2804] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.980535 31860 raft_consensus.cc:493] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.980943 31860 raft_consensus.cc:3060] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.985178 31533 tablet_bootstrap.cc:492] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:55.990376 31532 tablet_bootstrap.cc:654] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.992429 31533 tablet_bootstrap.cc:654] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.994673 31531 ts_tablet_manager.cc:1434] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:10:55.995641 31531 tablet_bootstrap.cc:492] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:55.996817 31860 raft_consensus.cc:515] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:55.999778 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f564510ff30e42ec890531c82405946c" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:56.000385 31411 raft_consensus.cc:3060] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.008134 31531 tablet_bootstrap.cc:654] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.008817 31411 raft_consensus.cc:2468] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:56.010076 31301 leader_election.cc:304] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:56.010973 31716 raft_consensus.cc:2804] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.011428 31716 raft_consensus.cc:697] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:56.012229 31716 consensus_queue.cc:237] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.022863 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f564510ff30e42ec890531c82405946c" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
W20260709 13:10:56.024667 31303 leader_election.cc:343] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.039228 31157 catalog_manager.cc:5697] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:56.055963 31860 leader_election.cc:290] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:56.058147 31160 catalog_manager.cc:5697] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:56.080490 31530 ts_tablet_manager.cc:1434] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.121s	user 0.007s	sys 0.001s
I20260709 13:10:56.084766 31530 tablet_bootstrap.cc:492] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:56.087471 31532 tablet_bootstrap.cc:492] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:56.088169 31532 ts_tablet_manager.cc:1403] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.130s	user 0.010s	sys 0.007s
I20260709 13:10:56.082774 31533 tablet_bootstrap.cc:492] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:56.088810 31533 ts_tablet_manager.cc:1403] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.001s
I20260709 13:10:56.095783 31732 raft_consensus.cc:493] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.096437 31732 raft_consensus.cc:515] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.100703 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:56.103283 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
W20260709 13:10:56.105225 31226 leader_election.cc:343] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:56.105759 31225 leader_election.cc:343] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.105165 31533 raft_consensus.cc:359] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
W20260709 13:10:56.106624 31207 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:56.106426 31225 leader_election.cc:304] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:56.107184 31530 tablet_bootstrap.cc:654] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.107441 31834 raft_consensus.cc:2749] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:56.107884 31732 leader_election.cc:290] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:56.106616 31533 raft_consensus.cc:385] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.110067 31533 raft_consensus.cc:740] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.110736 31533 consensus_queue.cc:260] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.093465 31532 raft_consensus.cc:359] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.113447 31532 raft_consensus.cc:385] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.114060 31532 raft_consensus.cc:740] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.118361 31533 ts_tablet_manager.cc:1434] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.002s
I20260709 13:10:56.115614 31532 consensus_queue.cc:260] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.119421 31533 tablet_bootstrap.cc:492] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:56.121346 31532 ts_tablet_manager.cc:1434] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 13:10:56.119602 31531 tablet_bootstrap.cc:492] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:56.123435 31531 ts_tablet_manager.cc:1403] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.128s	user 0.047s	sys 0.070s
I20260709 13:10:56.126070 31531 raft_consensus.cc:359] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.126699 31531 raft_consensus.cc:385] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.126942 31531 raft_consensus.cc:740] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.127286 31533 tablet_bootstrap.cc:654] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.127764 31531 consensus_queue.cc:260] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.129969 31531 ts_tablet_manager.cc:1434] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:10:56.131132 31860 raft_consensus.cc:493] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.131395 31531 tablet_bootstrap.cc:492] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:56.131806 31860 raft_consensus.cc:515] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.132476 31530 tablet_bootstrap.cc:492] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:56.133033 31530 ts_tablet_manager.cc:1403] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 13:10:56.134433 31860 leader_election.cc:290] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:56.137228 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9baba29a028498288a27243e555fb27" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:56.137890 31532 tablet_bootstrap.cc:492] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:56.138604 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9baba29a028498288a27243e555fb27" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
W20260709 13:10:56.139024 31303 leader_election.cc:343] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.139289 31261 raft_consensus.cc:2468] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:56.139458 31742 raft_consensus.cc:493] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.139951 31742 raft_consensus.cc:515] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.140650 31303 leader_election.cc:304] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:56.141861 31742 leader_election.cc:290] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:56.144325 31532 tablet_bootstrap.cc:654] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.144743 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:56.145812 31531 tablet_bootstrap.cc:654] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.151407 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:56.152897 31448 leader_election.cc:304] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:56.153885 31852 raft_consensus.cc:2749] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:56.154683 31860 raft_consensus.cc:2804] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.155400 31860 raft_consensus.cc:493] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.156018 31860 raft_consensus.cc:3060] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.163858 31411 raft_consensus.cc:1275] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:56.165895 31732 consensus_queue.cc:1048] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:56.171137 31531 tablet_bootstrap.cc:492] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:56.171604 31531 ts_tablet_manager.cc:1403] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.008s
I20260709 13:10:56.174577 31531 raft_consensus.cc:359] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.175233 31531 raft_consensus.cc:385] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.175475 31531 raft_consensus.cc:740] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.178018 31531 consensus_queue.cc:260] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.180228 31531 ts_tablet_manager.cc:1434] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:10:56.153127 31530 raft_consensus.cc:359] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.181505 31530 raft_consensus.cc:385] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.181878 31530 raft_consensus.cc:740] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.182000 31531 tablet_bootstrap.cc:492] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:56.182700 31530 consensus_queue.cc:260] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.185199 31860 raft_consensus.cc:515] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.188174 31851 raft_consensus.cc:493] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.189213 31531 tablet_bootstrap.cc:654] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.188771 31851 raft_consensus.cc:515] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.189766 31530 ts_tablet_manager.cc:1434] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:10:56.190681 31532 tablet_bootstrap.cc:492] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:56.191596 31851 leader_election.cc:290] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:56.192796 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b3eb1a472a546fdb51a2cc295716492" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:56.194089 31410 raft_consensus.cc:3060] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.196753 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9baba29a028498288a27243e555fb27" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:56.201267 31531 tablet_bootstrap.cc:492] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:56.201776 31531 ts_tablet_manager.cc:1403] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.009s
W20260709 13:10:56.202965 31303 leader_election.cc:343] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.203563 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b3eb1a472a546fdb51a2cc295716492" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:56.204392 31531 raft_consensus.cc:359] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.205202 31531 raft_consensus.cc:385] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.205351 31374 leader_election.cc:304] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:56.205506 31531 raft_consensus.cc:740] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.206676 31532 ts_tablet_manager.cc:1403] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.069s	user 0.011s	sys 0.004s
I20260709 13:10:56.206336 31531 consensus_queue.cc:260] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.207479 31530 tablet_bootstrap.cc:492] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:56.208691 31531 ts_tablet_manager.cc:1434] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:10:56.209543 31531 tablet_bootstrap.cc:492] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:56.211263 31532 raft_consensus.cc:359] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.212085 31860 leader_election.cc:290] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:56.218870 31483 raft_consensus.cc:1275] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:56.237133 31533 tablet_bootstrap.cc:492] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:56.237633 31533 ts_tablet_manager.cc:1403] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.118s	user 0.020s	sys 0.020s
I20260709 13:10:56.215044 31531 tablet_bootstrap.cc:654] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.226603 31530 tablet_bootstrap.cc:654] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.242067 31533 raft_consensus.cc:359] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.237357 31728 consensus_queue.cc:1048] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:56.219520 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9baba29a028498288a27243e555fb27" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:56.243719 31261 raft_consensus.cc:3060] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.242748 31533 raft_consensus.cc:385] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.245630 31533 raft_consensus.cc:740] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.246178 31483 raft_consensus.cc:3060] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.246603 31533 consensus_queue.cc:260] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.249402 31533 ts_tablet_manager.cc:1434] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:10:56.251000 31533 tablet_bootstrap.cc:492] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:56.251104 31261 raft_consensus.cc:2468] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:56.253633 31303 leader_election.cc:304] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:56.253779 31851 raft_consensus.cc:2749] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:56.255225 31716 consensus_queue.cc:1048] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:56.256594 31716 raft_consensus.cc:2804] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.257036 31716 raft_consensus.cc:697] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:56.257269 31533 tablet_bootstrap.cc:654] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.257959 31716 consensus_queue.cc:237] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.266544 31852 raft_consensus.cc:493] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.267143 31852 raft_consensus.cc:515] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.269537 31852 leader_election.cc:290] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:56.269982 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9baba29a028498288a27243e555fb27" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:56.270681 31260 raft_consensus.cc:2393] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:56.271255 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9baba29a028498288a27243e555fb27" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:56.275801 31448 leader_election.cc:304] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:56.282321 31852 raft_consensus.cc:3060] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.285128 31532 raft_consensus.cc:385] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.285480 31532 raft_consensus.cc:740] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.292166 31530 tablet_bootstrap.cc:492] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:56.292811 31530 ts_tablet_manager.cc:1403] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.086s	user 0.008s	sys 0.002s
I20260709 13:10:56.290221 31532 consensus_queue.cc:260] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.297960 31532 ts_tablet_manager.cc:1434] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.091s	user 0.003s	sys 0.003s
I20260709 13:10:56.299147 31532 tablet_bootstrap.cc:492] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:56.306049 31533 tablet_bootstrap.cc:492] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:56.306509 31533 ts_tablet_manager.cc:1403] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.012s
I20260709 13:10:56.306901 31852 raft_consensus.cc:2749] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:56.308945 31533 raft_consensus.cc:359] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.309566 31533 raft_consensus.cc:385] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.309818 31533 raft_consensus.cc:740] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.310375 31533 consensus_queue.cc:260] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.317726 31533 ts_tablet_manager.cc:1434] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:10:56.318442 31533 tablet_bootstrap.cc:492] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:56.330869 31728 raft_consensus.cc:493] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.331413 31728 raft_consensus.cc:515] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.333400 31728 leader_election.cc:290] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:56.334611 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9725e6248e7b4e67b23a3b99967003bf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
W20260709 13:10:56.335988 31226 leader_election.cc:343] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.330991 31160 catalog_manager.cc:5697] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:56.343619 31530 raft_consensus.cc:359] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.344465 31530 raft_consensus.cc:385] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.344722 31530 raft_consensus.cc:740] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.348969 31533 tablet_bootstrap.cc:654] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.350435 31530 consensus_queue.cc:260] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.352135 31531 tablet_bootstrap.cc:492] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:56.359247 31531 ts_tablet_manager.cc:1403] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.150s	user 0.016s	sys 0.017s
I20260709 13:10:56.353286 31728 raft_consensus.cc:493] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.357163 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9725e6248e7b4e67b23a3b99967003bf" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:56.362210 31728 raft_consensus.cc:515] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.364655 31728 leader_election.cc:290] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:56.365232 31860 consensus_queue.cc:1048] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:56.367712 31531 raft_consensus.cc:359] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.368341 31531 raft_consensus.cc:385] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.368588 31531 raft_consensus.cc:740] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.369180 31531 consensus_queue.cc:260] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.373809 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
W20260709 13:10:56.376048 31228 leader_election.cc:343] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:56.361132 31449 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:45117, user_credentials={real_user=slave}} blocked reactor thread for 46550.1us
I20260709 13:10:56.376116 31531 ts_tablet_manager.cc:1434] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:10:56.379841 31531 tablet_bootstrap.cc:492] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:56.376881 31228 leader_election.cc:304] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
W20260709 13:10:56.388664 31225 leader_election.cc:343] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.390134 31532 tablet_bootstrap.cc:654] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.401935 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:56.381484 31716 raft_consensus.cc:493] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.407456 31830 raft_consensus.cc:2749] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:56.407500 31530 ts_tablet_manager.cc:1434] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.114s	user 0.004s	sys 0.004s
I20260709 13:10:56.409271 31530 tablet_bootstrap.cc:492] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
W20260709 13:10:56.415884 31228 leader_election.cc:343] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.416556 31228 leader_election.cc:304] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:56.418159 31728 raft_consensus.cc:2749] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:56.420183 31716 raft_consensus.cc:515] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.429633 31530 tablet_bootstrap.cc:654] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.437767 31716 leader_election.cc:290] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:56.438380 31711 consensus_queue.cc:1048] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:56.438961 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:56.439332 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:56.439962 31261 raft_consensus.cc:2468] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
W20260709 13:10:56.441066 31301 leader_election.cc:343] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.444387 31303 leader_election.cc:304] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:56.445207 31891 raft_consensus.cc:493] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.445410 31716 raft_consensus.cc:2804] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.445926 31716 raft_consensus.cc:493] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.445765 31891 raft_consensus.cc:515] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.446306 31716 raft_consensus.cc:3060] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.447790 31891 leader_election.cc:290] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:56.448814 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "862d691f6d9e4d9fb185be53b562b8ff" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:56.452620 31716 raft_consensus.cc:515] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.458540 31532 tablet_bootstrap.cc:492] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:56.459549 31532 ts_tablet_manager.cc:1403] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.161s	user 0.006s	sys 0.012s
I20260709 13:10:56.469841 31533 tablet_bootstrap.cc:492] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:56.470535 31533 ts_tablet_manager.cc:1403] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.152s	user 0.028s	sys 0.027s
I20260709 13:10:56.479030 31533 raft_consensus.cc:359] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.479701 31533 raft_consensus.cc:385] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.479979 31533 raft_consensus.cc:740] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
W20260709 13:10:56.486053 31377 consensus_peers.cc:597] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 -> Peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Couldn't send request to peer 001cac731610478a900ad62e830aaff6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:56.483572 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:56.488564 31260 raft_consensus.cc:3060] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.472121 31482 raft_consensus.cc:3060] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.481099 31533 consensus_queue.cc:260] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.508248 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
W20260709 13:10:56.511193 31301 leader_election.cc:343] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.513752 31483 raft_consensus.cc:3060] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.505903 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "862d691f6d9e4d9fb185be53b562b8ff" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:56.518429 31716 leader_election.cc:290] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:56.522545 31531 tablet_bootstrap.cc:654] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.524227 31260 raft_consensus.cc:2468] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:56.526273 31303 leader_election.cc:304] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:56.527067 31860 raft_consensus.cc:2804] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.538492 31533 ts_tablet_manager.cc:1434] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.001s
I20260709 13:10:56.539043 31728 raft_consensus.cc:493] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.539640 31533 tablet_bootstrap.cc:492] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:56.538339 31449 leader_election.cc:304] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:56.539510 31728 raft_consensus.cc:515] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.542348 31728 leader_election.cc:290] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:56.543664 31851 raft_consensus.cc:493] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.544137 31851 raft_consensus.cc:515] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.545667 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad34ee917be4455bbe71da15c523cee" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:56.546085 31851 leader_election.cc:290] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:56.546262 31411 raft_consensus.cc:2468] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
I20260709 13:10:56.527617 31860 raft_consensus.cc:697] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:56.543679 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad34ee917be4455bbe71da15c523cee" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:56.549993 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bc6ebfdfef4b7a81d4cbc17a86d70f" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:56.550614 31334 raft_consensus.cc:2468] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:56.553973 31374 leader_election.cc:304] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:56.548287 31226 leader_election.cc:304] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:56.554692 31532 raft_consensus.cc:359] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.555948 31532 raft_consensus.cc:385] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.556241 31532 raft_consensus.cc:740] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.557684 31851 raft_consensus.cc:2804] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.558084 31851 raft_consensus.cc:493] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.558416 31851 raft_consensus.cc:3060] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.560333 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bc6ebfdfef4b7a81d4cbc17a86d70f" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:56.562522 31532 consensus_queue.cc:260] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.563911 31851 raft_consensus.cc:515] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.564688 31532 ts_tablet_manager.cc:1434] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.105s	user 0.004s	sys 0.000s
I20260709 13:10:56.565626 31532 tablet_bootstrap.cc:492] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:56.566239 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bc6ebfdfef4b7a81d4cbc17a86d70f" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:56.566840 31336 raft_consensus.cc:3060] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:56.556109 31228 leader_election.cc:343] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:56.573868 31377 leader_election.cc:343] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.557121 31832 raft_consensus.cc:2804] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.580034 31532 tablet_bootstrap.cc:654] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.585582 31530 tablet_bootstrap.cc:492] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:56.586345 31530 ts_tablet_manager.cc:1403] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.177s	user 0.045s	sys 0.016s
I20260709 13:10:56.578048 31832 raft_consensus.cc:493] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.590247 31484 raft_consensus.cc:3060] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.581743 31860 consensus_queue.cc:237] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.601773 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bc6ebfdfef4b7a81d4cbc17a86d70f" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:56.592733 31832 raft_consensus.cc:3060] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:56.611001 31377 leader_election.cc:343] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.620186 31793 consensus_queue.cc:1048] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:56.621080 31530 raft_consensus.cc:359] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.621791 31530 raft_consensus.cc:385] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.622067 31530 raft_consensus.cc:740] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.622712 31530 consensus_queue.cc:260] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.624770 31530 ts_tablet_manager.cc:1434] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.038s	user 0.002s	sys 0.001s
I20260709 13:10:56.627415 31832 raft_consensus.cc:515] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.609230 31336 raft_consensus.cc:2468] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:56.631830 31374 leader_election.cc:304] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:56.632854 31802 consensus_queue.cc:1048] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:56.633020 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad34ee917be4455bbe71da15c523cee" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:56.633637 31410 raft_consensus.cc:3060] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.634344 31711 raft_consensus.cc:2804] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.634830 31711 raft_consensus.cc:697] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:56.586411 31851 leader_election.cc:290] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:56.637507 31530 tablet_bootstrap.cc:492] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:56.636899 31832 leader_election.cc:290] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:56.648357 31410 raft_consensus.cc:2468] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:56.650125 31226 leader_election.cc:304] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:56.635695 31711 consensus_queue.cc:237] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.659409 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad34ee917be4455bbe71da15c523cee" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
W20260709 13:10:56.672104 31228 leader_election.cc:343] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.678433 31533 tablet_bootstrap.cc:654] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.681614 31157 catalog_manager.cc:5697] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:56.655375 31832 raft_consensus.cc:2804] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.685125 31793 raft_consensus.cc:493] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.693728 31793 raft_consensus.cc:515] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:56.695757 31793 leader_election.cc:290] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:56.682945 31802 raft_consensus.cc:493] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.697021 31802 raft_consensus.cc:515] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.697332 31408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f213e6b4ef544937bbd9c8e49c672693" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:56.697409 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f213e6b4ef544937bbd9c8e49c672693" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:56.701727 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d905b9b08a224fe09df30744558819e3" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:56.702525 31891 raft_consensus.cc:2749] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:56.699563 31301 leader_election.cc:343] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:56.703653 31303 leader_election.cc:343] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.703838 31802 leader_election.cc:290] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:56.704854 31303 leader_election.cc:304] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
W20260709 13:10:56.706818 31303 leader_election.cc:343] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.700126 31832 raft_consensus.cc:697] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:10:56.709635 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d905b9b08a224fe09df30744558819e3" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:56.710237 31260 raft_consensus.cc:2468] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:56.713232 31303 leader_election.cc:304] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:56.713620 31533 tablet_bootstrap.cc:492] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:56.714056 31533 ts_tablet_manager.cc:1403] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.175s	user 0.011s	sys 0.003s
I20260709 13:10:56.714210 31793 raft_consensus.cc:2804] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.714591 31793 raft_consensus.cc:493] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.714906 31793 raft_consensus.cc:3060] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.716624 31533 raft_consensus.cc:359] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.717276 31533 raft_consensus.cc:385] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.717557 31533 raft_consensus.cc:740] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.718175 31533 consensus_queue.cc:260] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.708113 31860 raft_consensus.cc:2749] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:56.709565 31832 consensus_queue.cc:237] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.737802 31533 ts_tablet_manager.cc:1434] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:10:56.738925 31533 tablet_bootstrap.cc:492] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:56.744315 31531 tablet_bootstrap.cc:492] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:56.745025 31531 ts_tablet_manager.cc:1403] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.365s	user 0.015s	sys 0.016s
I20260709 13:10:56.746471 31159 catalog_manager.cc:5697] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:56.748457 31531 raft_consensus.cc:359] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.749231 31531 raft_consensus.cc:385] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.750069 31531 raft_consensus.cc:740] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.752137 31728 raft_consensus.cc:493] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.754920 31728 raft_consensus.cc:515] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.756798 31728 leader_election.cc:290] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:56.757493 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "229af634ffc1437fb9dea4a95652ab5b" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
W20260709 13:10:56.759055 31225 leader_election.cc:343] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.760022 31860 raft_consensus.cc:493] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.760649 31860 raft_consensus.cc:515] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.763232 31860 leader_election.cc:290] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:56.766933 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
W20260709 13:10:56.771334 31301 leader_election.cc:343] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.771785 31530 tablet_bootstrap.cc:654] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.776134 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "229af634ffc1437fb9dea4a95652ab5b" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
W20260709 13:10:56.781632 31226 leader_election.cc:343] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.782317 31226 leader_election.cc:304] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:56.783947 31834 raft_consensus.cc:2749] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:56.789255 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:56.789897 31261 raft_consensus.cc:2468] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:56.751112 31531 consensus_queue.cc:260] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.793002 31303 leader_election.cc:304] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:56.802521 31531 ts_tablet_manager.cc:1434] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 13:10:56.804008 31531 tablet_bootstrap.cc:492] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:56.799001 31716 raft_consensus.cc:2804] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:56.809569 31716 raft_consensus.cc:493] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.819556 31716 raft_consensus.cc:3060] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.831768 31159 catalog_manager.cc:5697] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:56.861651 31532 tablet_bootstrap.cc:492] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:56.862213 31532 ts_tablet_manager.cc:1403] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.297s	user 0.023s	sys 0.043s
I20260709 13:10:56.853372 31793 raft_consensus.cc:515] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.879459 31531 tablet_bootstrap.cc:654] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.884886 31793 leader_election.cc:290] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:56.896325 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d905b9b08a224fe09df30744558819e3" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:56.897511 31802 consensus_queue.cc:1048] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:56.933802 31716 raft_consensus.cc:515] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.936378 31716 leader_election.cc:290] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:56.939790 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:56.939991 31532 raft_consensus.cc:359] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.940734 31532 raft_consensus.cc:385] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.941015 31532 raft_consensus.cc:740] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
W20260709 13:10:56.941563 31301 leader_election.cc:343] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.941696 31532 consensus_queue.cc:260] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:56.947714 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d905b9b08a224fe09df30744558819e3" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:56.948320 31260 raft_consensus.cc:3060] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.955564 31260 raft_consensus.cc:2468] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:56.958137 31530 tablet_bootstrap.cc:492] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:56.958612 31530 ts_tablet_manager.cc:1403] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.321s	user 0.027s	sys 0.026s
I20260709 13:10:56.962093 31530 raft_consensus.cc:359] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.962713 31530 raft_consensus.cc:385] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.962962 31530 raft_consensus.cc:740] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.963609 31530 consensus_queue.cc:260] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:56.965767 31530 ts_tablet_manager.cc:1434] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:56.967235 31532 ts_tablet_manager.cc:1434] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.105s	user 0.007s	sys 0.000s
I20260709 13:10:56.967720 31531 tablet_bootstrap.cc:492] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:56.968189 31531 ts_tablet_manager.cc:1403] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.164s	user 0.016s	sys 0.030s
I20260709 13:10:56.968334 31532 tablet_bootstrap.cc:492] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
W20260709 13:10:56.971271 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:36133, user_credentials={real_user=slave}} blocked reactor thread for 75187.9us
W20260709 13:10:56.978138 31303 leader_election.cc:343] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:56.978768 31303 leader_election.cc:304] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:56.979988 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:56.980151 31802 raft_consensus.cc:2804] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:56.980548 31261 raft_consensus.cc:3060] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:56.980680 31802 raft_consensus.cc:697] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:56.981403 31532 tablet_bootstrap.cc:654] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.981678 31802 consensus_queue.cc:237] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:56.990386 31533 tablet_bootstrap.cc:654] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.999711 31532 tablet_bootstrap.cc:492] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:57.000190 31532 ts_tablet_manager.cc:1403] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.000s
I20260709 13:10:57.002403 31834 raft_consensus.cc:493] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.002666 31532 raft_consensus.cc:359] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.002883 31834 raft_consensus.cc:515] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.003371 31532 raft_consensus.cc:385] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.003767 31532 raft_consensus.cc:740] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.005784 31834 leader_election.cc:290] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:57.004420 31532 consensus_queue.cc:260] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.007639 31532 ts_tablet_manager.cc:1434] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:57.007462 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:57.008020 31727 consensus_queue.cc:1048] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:56.992400 31531 raft_consensus.cc:359] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.009123 31531 raft_consensus.cc:385] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.009436 31531 raft_consensus.cc:740] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.009460 31851 raft_consensus.cc:493] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.010099 31851 raft_consensus.cc:515] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:10:57.011457 31228 leader_election.cc:343] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.013871 31531 consensus_queue.cc:260] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.014906 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:57.015848 31851 leader_election.cc:290] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
W20260709 13:10:57.019402 31226 leader_election.cc:343] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.019829 31226 leader_election.cc:304] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:57.020354 31485 raft_consensus.cc:3060] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.021816 31832 raft_consensus.cc:2749] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.021031 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:57.024338 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:57.025188 31262 raft_consensus.cc:2393] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:57.028090 31377 leader_election.cc:304] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:57.029374 31851 raft_consensus.cc:3060] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.031203 31531 ts_tablet_manager.cc:1434] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.002s
I20260709 13:10:57.032114 31531 tablet_bootstrap.cc:492] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:57.036540 31530 tablet_bootstrap.cc:492] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:57.042671 31531 tablet_bootstrap.cc:654] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.044883 31851 raft_consensus.cc:2749] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.046267 31261 raft_consensus.cc:2468] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:57.054468 31530 tablet_bootstrap.cc:654] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.058554 31533 tablet_bootstrap.cc:492] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:57.059815 31533 ts_tablet_manager.cc:1403] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.321s	user 0.011s	sys 0.000s
I20260709 13:10:57.062359 31533 raft_consensus.cc:359] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.065490 31533 raft_consensus.cc:385] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.065941 31533 raft_consensus.cc:740] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.067601 31533 consensus_queue.cc:260] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.069927 31533 ts_tablet_manager.cc:1434] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.001s
I20260709 13:10:57.009145 31532 tablet_bootstrap.cc:492] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:57.070986 31533 tablet_bootstrap.cc:492] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:57.072171 31303 leader_election.cc:304] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:57.073367 31913 raft_consensus.cc:2804] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.073856 31913 raft_consensus.cc:697] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:57.074615 31913 consensus_queue.cc:237] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.079753 31533 tablet_bootstrap.cc:654] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.088426 31802 raft_consensus.cc:493] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.089044 31802 raft_consensus.cc:515] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.093392 31533 tablet_bootstrap.cc:492] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:57.094210 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfb2b2292c140bbaa93438cce1746cb" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:57.095782 31802 leader_election.cc:290] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:57.095186 31852 raft_consensus.cc:493] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.096424 31852 raft_consensus.cc:515] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.096932 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfb2b2292c140bbaa93438cce1746cb" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:57.098273 31303 leader_election.cc:343] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.098695 31852 leader_election.cc:290] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:57.100256 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bc6ebfdfef4b7a81d4cbc17a86d70f" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:57.100961 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bc6ebfdfef4b7a81d4cbc17a86d70f" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:57.102011 31334 raft_consensus.cc:2393] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:57.103327 31448 leader_election.cc:304] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:57.104422 31852 raft_consensus.cc:3060] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:57.109903 31301 leader_election.cc:343] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.110450 31301 leader_election.cc:304] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:57.112274 31911 raft_consensus.cc:2749] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.112931 31852 raft_consensus.cc:2749] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.127373 31531 tablet_bootstrap.cc:492] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:57.127908 31531 ts_tablet_manager.cc:1403] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.096s	user 0.009s	sys 0.004s
I20260709 13:10:57.135782 31531 raft_consensus.cc:359] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.136464 31531 raft_consensus.cc:385] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.136936 31531 raft_consensus.cc:740] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.138409 31531 consensus_queue.cc:260] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.138695 31157 catalog_manager.cc:5697] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:57.148677 31793 consensus_queue.cc:1048] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.101197 31533 ts_tablet_manager.cc:1403] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.006s
I20260709 13:10:57.151527 31532 tablet_bootstrap.cc:654] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.156289 31530 tablet_bootstrap.cc:492] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:57.159416 31530 ts_tablet_manager.cc:1403] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.123s	user 0.013s	sys 0.025s
I20260709 13:10:57.160778 31711 raft_consensus.cc:493] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.161264 31711 raft_consensus.cc:515] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.162864 31530 raft_consensus.cc:359] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.163342 31711 leader_election.cc:290] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:57.163623 31530 raft_consensus.cc:385] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.164040 31530 raft_consensus.cc:740] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.164732 31530 consensus_queue.cc:260] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.165937 31531 ts_tablet_manager.cc:1434] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:10:57.166867 31531 tablet_bootstrap.cc:492] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:57.167650 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:57.168452 31261 raft_consensus.cc:2393] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:57.169282 31530 ts_tablet_manager.cc:1434] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:10:57.170759 31530 tablet_bootstrap.cc:492] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:57.171380 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:57.173344 31374 leader_election.cc:304] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:57.163916 31711 consensus_queue.cc:1048] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.174063 31663 raft_consensus.cc:3060] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.175805 31913 consensus_queue.cc:1048] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.181804 31802 raft_consensus.cc:493] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.182258 31802 raft_consensus.cc:515] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.153934 31533 raft_consensus.cc:359] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.186074 31533 raft_consensus.cc:385] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.193961 31533 raft_consensus.cc:740] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.207960 31802 leader_election.cc:290] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
W20260709 13:10:57.210178 31207 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:57.212989 31530 tablet_bootstrap.cc:654] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.218129 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:57.223516 31303 leader_election.cc:343] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.224865 31533 consensus_queue.cc:260] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.228520 31531 tablet_bootstrap.cc:654] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.234956 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:57.235409 31261 raft_consensus.cc:2468] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:57.236872 31303 leader_election.cc:304] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:57.237484 31533 ts_tablet_manager.cc:1434] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.000s
I20260709 13:10:57.237902 31913 raft_consensus.cc:2804] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.238314 31913 raft_consensus.cc:493] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.238656 31913 raft_consensus.cc:3060] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.242913 31533 tablet_bootstrap.cc:492] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:57.286237 31834 raft_consensus.cc:493] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.286756 31834 raft_consensus.cc:515] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.288676 31834 leader_election.cc:290] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:57.274374 31156 catalog_manager.cc:5697] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.296272 31533 tablet_bootstrap.cc:654] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.300122 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fb3ec3412341df88b759485f90f108" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:57.300491 31532 tablet_bootstrap.cc:492] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:57.300992 31532 ts_tablet_manager.cc:1403] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.292s	user 0.032s	sys 0.051s
W20260709 13:10:57.301656 31225 leader_election.cc:343] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.303504 31913 raft_consensus.cc:515] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.303583 31532 raft_consensus.cc:359] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.310120 31532 raft_consensus.cc:385] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.310396 31532 raft_consensus.cc:740] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.311033 31532 consensus_queue.cc:260] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.322880 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fb3ec3412341df88b759485f90f108" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:57.323632 31532 ts_tablet_manager.cc:1434] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:10:57.324903 31532 tablet_bootstrap.cc:492] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
W20260709 13:10:57.330090 31226 leader_election.cc:343] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.330659 31226 leader_election.cc:304] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:57.334280 31728 raft_consensus.cc:2749] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.334903 31533 tablet_bootstrap.cc:492] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:57.335407 31533 ts_tablet_manager.cc:1403] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.093s	user 0.007s	sys 0.010s
I20260709 13:10:57.337958 31533 raft_consensus.cc:359] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.338578 31533 raft_consensus.cc:385] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.338830 31533 raft_consensus.cc:740] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.339398 31533 consensus_queue.cc:260] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.345891 31530 tablet_bootstrap.cc:492] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:57.346412 31530 ts_tablet_manager.cc:1403] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.176s	user 0.016s	sys 0.001s
I20260709 13:10:57.355424 31728 raft_consensus.cc:493] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.355932 31728 raft_consensus.cc:515] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.357589 31483 raft_consensus.cc:3060] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.357599 31832 consensus_queue.cc:1048] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:57.355466 31663 raft_consensus.cc:2749] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.398521 31533 ts_tablet_manager.cc:1434] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.063s	user 0.012s	sys 0.010s
I20260709 13:10:57.405808 31533 tablet_bootstrap.cc:492] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:57.407413 31531 tablet_bootstrap.cc:492] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:57.407893 31531 ts_tablet_manager.cc:1403] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.241s	user 0.028s	sys 0.021s
I20260709 13:10:57.414543 31530 raft_consensus.cc:359] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.415223 31530 raft_consensus.cc:385] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.415489 31530 raft_consensus.cc:740] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.416054 31530 consensus_queue.cc:260] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.425814 31530 ts_tablet_manager.cc:1434] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.010s
I20260709 13:10:57.417194 31911 raft_consensus.cc:493] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.432842 31530 tablet_bootstrap.cc:492] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:57.440634 31728 leader_election.cc:290] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:57.434584 31663 consensus_queue.cc:1048] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:10:57.446334 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:57.449010 31913 leader_election.cc:290] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:57.436328 31911 raft_consensus.cc:515] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.451515 31911 leader_election.cc:290] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
W20260709 13:10:57.454838 31225 leader_election.cc:343] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.460420 31531 raft_consensus.cc:359] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.461090 31531 raft_consensus.cc:385] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.462522 31532 tablet_bootstrap.cc:654] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.463456 31260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:57.474677 31260 raft_consensus.cc:3060] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.464808 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "229af634ffc1437fb9dea4a95652ab5b" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:57.476152 31261 raft_consensus.cc:2468] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:57.468032 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:57.470582 31485 raft_consensus.cc:3060] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.472675 31533 tablet_bootstrap.cc:654] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.473443 31531 raft_consensus.cc:740] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.479952 31531 consensus_queue.cc:260] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.494350 31260 raft_consensus.cc:2468] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:57.498500 31531 ts_tablet_manager.cc:1434] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.090s	user 0.008s	sys 0.000s
I20260709 13:10:57.499452 31531 tablet_bootstrap.cc:492] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:57.500286 31663 raft_consensus.cc:493] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.500869 31663 raft_consensus.cc:515] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
W20260709 13:10:57.501754 31303 leader_election.cc:343] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.503486 31303 leader_election.cc:304] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:57.504228 31303 leader_election.cc:304] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:57.505089 31860 consensus_queue.cc:1048] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:57.508903 31301 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:45117, user_credentials={real_user=slave}} blocked reactor thread for 80890.9us
I20260709 13:10:57.508373 31663 leader_election.cc:290] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:57.508368 31913 raft_consensus.cc:2804] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.517481 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f213e6b4ef544937bbd9c8e49c672693" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:57.520016 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:57.523731 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "229af634ffc1437fb9dea4a95652ab5b" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:57.524588 31532 tablet_bootstrap.cc:492] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
W20260709 13:10:57.525583 31377 leader_election.cc:343] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.525059 31532 ts_tablet_manager.cc:1403] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.200s	user 0.013s	sys 0.003s
I20260709 13:10:57.508461 31911 raft_consensus.cc:2804] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:10:57.527293 31228 leader_election.cc:343] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.527457 31911 raft_consensus.cc:493] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.527897 31911 raft_consensus.cc:3060] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.527807 31228 leader_election.cc:304] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:57.528921 31834 raft_consensus.cc:2749] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.532095 31852 raft_consensus.cc:493] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.532557 31852 raft_consensus.cc:515] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.533579 31832 consensus_queue.cc:1048] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:57.532702 31532 raft_consensus.cc:359] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.535686 31913 raft_consensus.cc:697] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:57.535005 31532 raft_consensus.cc:385] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.536369 31532 raft_consensus.cc:740] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.537174 31913 consensus_queue.cc:237] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.536967 31532 consensus_queue.cc:260] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.541450 31531 tablet_bootstrap.cc:654] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.543380 31852 leader_election.cc:290] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:57.543992 31663 raft_consensus.cc:493] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.554312 31663 raft_consensus.cc:515] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.555263 31533 tablet_bootstrap.cc:492] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:57.555742 31533 ts_tablet_manager.cc:1403] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.156s	user 0.009s	sys 0.003s
I20260709 13:10:57.556469 31663 leader_election.cc:290] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:57.556775 31911 raft_consensus.cc:515] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.558068 31937 consensus_queue.cc:1048] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.559443 31911 leader_election.cc:290] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:57.560124 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9725e6248e7b4e67b23a3b99967003bf" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:57.561053 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad34ee917be4455bbe71da15c523cee" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:57.562531 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9725e6248e7b4e67b23a3b99967003bf" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:57.562837 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad34ee917be4455bbe71da15c523cee" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:57.563122 31261 raft_consensus.cc:2468] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
W20260709 13:10:57.563167 31377 leader_election.cc:343] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.564452 31451 leader_election.cc:304] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:57.564587 31377 leader_election.cc:304] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:57.566766 31851 raft_consensus.cc:2804] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.567123 31851 raft_consensus.cc:493] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.567444 31851 raft_consensus.cc:3060] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.568473 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "229af634ffc1437fb9dea4a95652ab5b" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:57.569058 31262 raft_consensus.cc:3060] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.545240 31530 tablet_bootstrap.cc:654] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.585384 31533 raft_consensus.cc:359] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.586418 31533 raft_consensus.cc:385] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.586858 31533 raft_consensus.cc:740] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.588510 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "229af634ffc1437fb9dea4a95652ab5b" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:57.588989 31532 ts_tablet_manager.cc:1434] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.000s
I20260709 13:10:57.591763 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f213e6b4ef544937bbd9c8e49c672693" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:57.592396 31336 raft_consensus.cc:2468] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:57.596562 31532 tablet_bootstrap.cc:492] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:10:57.602571 31533 consensus_queue.cc:260] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.605857 31851 raft_consensus.cc:515] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.607973 31851 leader_election.cc:290] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:57.610342 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9725e6248e7b4e67b23a3b99967003bf" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:57.610967 31258 raft_consensus.cc:3060] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.613811 31262 raft_consensus.cc:2468] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:57.621924 31303 leader_election.cc:304] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:57.624341 31156 catalog_manager.cc:5697] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:57.629642 31834 raft_consensus.cc:493] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.630134 31834 raft_consensus.cc:515] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.633731 31852 raft_consensus.cc:2749] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:57.630640 31913 raft_consensus.cc:2804] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.647897 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1eac6b3d8a47b59679e4aee4293326" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:57.648669 31336 raft_consensus.cc:2468] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 0.
I20260709 13:10:57.649441 31913 raft_consensus.cc:697] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:57.649998 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1eac6b3d8a47b59679e4aee4293326" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:57.650115 31225 leader_election.cc:304] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:57.650923 31832 raft_consensus.cc:2804] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.651284 31832 raft_consensus.cc:493] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.651628 31832 raft_consensus.cc:3060] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:57.651818 31228 leader_election.cc:343] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.653422 31834 leader_election.cc:290] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
W20260709 13:10:57.637602 31943 log.cc:927] Time spent T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:10:57.639214 31533 ts_tablet_manager.cc:1434] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.001s
I20260709 13:10:57.664280 31832 raft_consensus.cc:515] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.664949 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9725e6248e7b4e67b23a3b99967003bf" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
W20260709 13:10:57.667871 31377 leader_election.cc:343] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.669018 31531 tablet_bootstrap.cc:492] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:57.669521 31531 ts_tablet_manager.cc:1403] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.170s	user 0.011s	sys 0.002s
I20260709 13:10:57.672219 31531 raft_consensus.cc:359] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.672849 31531 raft_consensus.cc:385] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.673810 31482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1eac6b3d8a47b59679e4aee4293326" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
W20260709 13:10:57.675045 31228 leader_election.cc:343] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.677211 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1eac6b3d8a47b59679e4aee4293326" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:57.677812 31336 raft_consensus.cc:3060] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.673117 31531 raft_consensus.cc:740] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.685122 31531 consensus_queue.cc:260] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.651672 31913 consensus_queue.cc:237] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.685580 31258 raft_consensus.cc:2468] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:57.687983 31377 leader_election.cc:304] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:57.689015 31851 raft_consensus.cc:2804] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:10:57.655944 31301 leader_election.cc:343] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.690075 31851 raft_consensus.cc:697] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:57.692833 31336 raft_consensus.cc:2468] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 1.
I20260709 13:10:57.692705 31851 consensus_queue.cc:237] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.694209 31225 leader_election.cc:304] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:57.702570 31530 tablet_bootstrap.cc:492] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:57.702893 31834 raft_consensus.cc:2804] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.703197 31530 ts_tablet_manager.cc:1403] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.277s	user 0.017s	sys 0.010s
W20260709 13:10:57.694911 31301 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.67:45117, user_credentials={real_user=slave}} blocked reactor thread for 48806.4us
I20260709 13:10:57.684195 31832 leader_election.cc:290] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:57.703691 31834 raft_consensus.cc:697] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:10:57.704613 31834 consensus_queue.cc:237] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
W20260709 13:10:57.710731 31301 leader_election.cc:343] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.711815 31533 tablet_bootstrap.cc:492] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:57.696765 31532 tablet_bootstrap.cc:654] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.715221 31160 catalog_manager.cc:5697] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.693866 31531 ts_tablet_manager.cc:1434] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.002s
I20260709 13:10:57.717216 31937 consensus_queue.cc:1048] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:57.718153 31531 tablet_bootstrap.cc:492] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:57.719384 31374 leader_election.cc:304] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:57.721692 31898 raft_consensus.cc:2804] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.721565 31530 raft_consensus.cc:359] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.722105 31898 raft_consensus.cc:493] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.722311 31530 raft_consensus.cc:385] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.722486 31898 raft_consensus.cc:3060] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.722631 31530 raft_consensus.cc:740] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.732336 31530 consensus_queue.cc:260] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.731968 31913 consensus_queue.cc:1048] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:10:57.738353 31530 ts_tablet_manager.cc:1434] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:10:57.739300 31530 tablet_bootstrap.cc:492] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:57.751441 31531 tablet_bootstrap.cc:654] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.736804 31898 raft_consensus.cc:515] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.771627 31530 tablet_bootstrap.cc:654] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.772423 31533 tablet_bootstrap.cc:654] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.775493 31898 leader_election.cc:290] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:57.776024 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f213e6b4ef544937bbd9c8e49c672693" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:57.776577 31336 raft_consensus.cc:3060] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.776937 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f213e6b4ef544937bbd9c8e49c672693" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:57.777524 31484 raft_consensus.cc:3060] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.784844 31336 raft_consensus.cc:2468] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:57.785909 31374 leader_election.cc:304] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:57.787595 31531 tablet_bootstrap.cc:492] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:57.788232 31531 ts_tablet_manager.cc:1403] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.070s	user 0.010s	sys 0.009s
I20260709 13:10:57.791900 31532 tablet_bootstrap.cc:492] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: No bootstrap required, opened a new log
I20260709 13:10:57.788096 31663 raft_consensus.cc:2804] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.792353 31532 ts_tablet_manager.cc:1403] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.196s	user 0.011s	sys 0.001s
I20260709 13:10:57.792639 31663 raft_consensus.cc:697] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:57.793563 31663 consensus_queue.cc:237] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.801160 31532 raft_consensus.cc:359] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.803753 31532 raft_consensus.cc:385] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.804049 31532 raft_consensus.cc:740] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.792652 31484 raft_consensus.cc:2468] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:57.804888 31532 consensus_queue.cc:260] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.808311 31160 catalog_manager.cc:5697] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.815412 31531 raft_consensus.cc:359] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.816066 31531 raft_consensus.cc:385] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.816306 31531 raft_consensus.cc:740] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.816560 31834 raft_consensus.cc:493] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.817211 31834 raft_consensus.cc:515] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.818225 31156 catalog_manager.cc:5697] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.820266 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:57.820896 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:57.822660 31834 leader_election.cc:290] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
W20260709 13:10:57.822755 31226 leader_election.cc:343] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:57.823889 31228 leader_election.cc:343] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.816900 31531 consensus_queue.cc:260] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.832283 31532 ts_tablet_manager.cc:1434] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:10:57.835188 31531 ts_tablet_manager.cc:1434] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 13:10:57.836108 31531 tablet_bootstrap.cc:492] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a: Bootstrap starting.
I20260709 13:10:57.834409 31228 leader_election.cc:304] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:57.840669 31834 raft_consensus.cc:2749] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.843437 31533 tablet_bootstrap.cc:492] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:57.843917 31533 ts_tablet_manager.cc:1403] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.006s
I20260709 13:10:57.850787 31531 tablet_bootstrap.cc:654] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.856631 31530 tablet_bootstrap.cc:492] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:57.857133 31530 ts_tablet_manager.cc:1403] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.118s	user 0.014s	sys 0.017s
I20260709 13:10:57.859866 31530 raft_consensus.cc:359] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.860541 31530 raft_consensus.cc:385] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.860797 31530 raft_consensus.cc:740] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.861398 31530 consensus_queue.cc:260] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.865015 31530 ts_tablet_manager.cc:1434] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:57.865932 31530 tablet_bootstrap.cc:492] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:57.869257 31937 raft_consensus.cc:493] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.869769 31937 raft_consensus.cc:515] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.871290 31663 raft_consensus.cc:493] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.872572 31530 tablet_bootstrap.cc:654] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.872673 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fb3ec3412341df88b759485f90f108" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:57.873085 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fb3ec3412341df88b759485f90f108" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:57.873713 31258 raft_consensus.cc:2468] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
W20260709 13:10:57.874138 31301 leader_election.cc:343] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.874917 31303 leader_election.cc:304] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:57.875643 31939 raft_consensus.cc:2804] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.876273 31939 raft_consensus.cc:493] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.876621 31939 raft_consensus.cc:3060] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.875576 31533 raft_consensus.cc:359] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.877473 31533 raft_consensus.cc:385] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.877851 31533 raft_consensus.cc:740] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.878784 31533 consensus_queue.cc:260] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:57.871840 31663 raft_consensus.cc:515] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.886945 31533 ts_tablet_manager.cc:1434] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.001s
I20260709 13:10:57.887852 31533 tablet_bootstrap.cc:492] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:57.891604 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfb2b2292c140bbaa93438cce1746cb" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:57.892206 31336 raft_consensus.cc:2468] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:57.892745 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfb2b2292c140bbaa93438cce1746cb" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:10:57.894761 31377 leader_election.cc:343] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.895273 31531 tablet_bootstrap.cc:492] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a: No bootstrap required, opened a new log
I20260709 13:10:57.895725 31531 ts_tablet_manager.cc:1403] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 13:10:57.896433 31937 leader_election.cc:290] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:57.898305 31531 raft_consensus.cc:359] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.898940 31374 leader_election.cc:304] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:57.899678 31531 raft_consensus.cc:385] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.899502 31663 leader_election.cc:290] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:57.900389 31531 raft_consensus.cc:740] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.899860 31898 raft_consensus.cc:2804] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.901257 31898 raft_consensus.cc:493] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.901755 31898 raft_consensus.cc:3060] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.885085 31160 catalog_manager.cc:5697] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.900861 31939 raft_consensus.cc:515] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.901458 31531 consensus_queue.cc:260] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.911170 31898 raft_consensus.cc:515] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.912101 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fb3ec3412341df88b759485f90f108" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:57.912379 31531 ts_tablet_manager.cc:1434] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:10:57.912742 31261 raft_consensus.cc:3060] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.914422 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfb2b2292c140bbaa93438cce1746cb" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:57.915046 31336 raft_consensus.cc:3060] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.915491 31484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfb2b2292c140bbaa93438cce1746cb" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
W20260709 13:10:57.916814 31377 leader_election.cc:343] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.917348 31898 leader_election.cc:290] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:57.909462 31663 raft_consensus.cc:493] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.918051 31663 raft_consensus.cc:515] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.919749 31663 leader_election.cc:290] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:57.921037 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "229af634ffc1437fb9dea4a95652ab5b" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:57.922897 31533 tablet_bootstrap.cc:654] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.923089 31261 raft_consensus.cc:2468] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:57.924256 31303 leader_election.cc:304] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:57.925199 31530 tablet_bootstrap.cc:492] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:57.925452 31939 leader_election.cc:290] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:10:57.926183 31939 raft_consensus.cc:2804] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.926738 31939 raft_consensus.cc:697] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:57.927945 31939 consensus_queue.cc:237] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.923151 31336 raft_consensus.cc:2468] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:57.932492 31530 ts_tablet_manager.cc:1403] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.019s
I20260709 13:10:57.935467 31533 tablet_bootstrap.cc:492] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:57.936002 31533 ts_tablet_manager.cc:1403] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.001s
I20260709 13:10:57.936815 31530 raft_consensus.cc:359] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.937539 31530 raft_consensus.cc:385] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.938112 31530 raft_consensus.cc:740] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.938791 31530 consensus_queue.cc:260] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:10:57.941893 31530 ts_tablet_manager.cc:1434] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:10:57.942865 31530 tablet_bootstrap.cc:492] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:57.944257 31533 raft_consensus.cc:359] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.944955 31533 raft_consensus.cc:385] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.945214 31533 raft_consensus.cc:740] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.947952 31374 leader_election.cc:304] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:57.949790 31533 consensus_queue.cc:260] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.950261 31851 raft_consensus.cc:2804] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:57.950767 31851 raft_consensus.cc:697] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:57.951186 31530 tablet_bootstrap.cc:654] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.951527 31851 consensus_queue.cc:237] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:57.952677 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fb3ec3412341df88b759485f90f108" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:57.973660 31410 raft_consensus.cc:3060] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.958570 31533 ts_tablet_manager.cc:1434] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 13:10:57.962224 31530 tablet_bootstrap.cc:492] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:57.977931 31530 ts_tablet_manager.cc:1403] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.001s
I20260709 13:10:57.978116 31533 tablet_bootstrap.cc:492] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:57.986418 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "229af634ffc1437fb9dea4a95652ab5b" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:57.987404 31258 raft_consensus.cc:2393] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:57.991253 31530 raft_consensus.cc:359] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.992355 31530 raft_consensus.cc:385] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.992784 31530 raft_consensus.cc:740] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.993016 31377 leader_election.cc:304] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:57.993902 31530 consensus_queue.cc:260] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:57.995198 31663 raft_consensus.cc:3060] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.999776 31533 tablet_bootstrap.cc:654] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.992170 31410 raft_consensus.cc:2468] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:58.005811 31663 raft_consensus.cc:2749] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.006655 31530 ts_tablet_manager.cc:1434] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.006s
I20260709 13:10:58.008088 31530 tablet_bootstrap.cc:492] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:58.016589 31530 tablet_bootstrap.cc:654] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.019672 31533 tablet_bootstrap.cc:492] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:58.020154 31533 ts_tablet_manager.cc:1403] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.006s
I20260709 13:10:58.023036 31533 raft_consensus.cc:359] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.023720 31533 raft_consensus.cc:385] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.023972 31533 raft_consensus.cc:740] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.024593 31533 consensus_queue.cc:260] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.026593 31533 ts_tablet_manager.cc:1434] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.003s
I20260709 13:10:58.027638 31533 tablet_bootstrap.cc:492] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:58.029814 31663 raft_consensus.cc:493] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.030308 31663 raft_consensus.cc:515] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.033136 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:58.034439 31663 leader_election.cc:290] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:58.034732 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:58.035420 31258 raft_consensus.cc:2468] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
W20260709 13:10:58.035742 31377 leader_election.cc:343] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.036522 31377 leader_election.cc:304] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:58.037397 31711 raft_consensus.cc:2804] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:58.037791 31711 raft_consensus.cc:493] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.038108 31711 raft_consensus.cc:3060] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.055408 31160 catalog_manager.cc:5697] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.072721 31937 consensus_queue.cc:1048] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.080015 31533 tablet_bootstrap.cc:654] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.087844 31484 raft_consensus.cc:3060] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.095845 31530 tablet_bootstrap.cc:492] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:58.097635 31949 raft_consensus.cc:493] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.098156 31949 raft_consensus.cc:515] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.099468 31937 consensus_queue.cc:1048] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.100235 31949 leader_election.cc:290] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:58.097352 31530 ts_tablet_manager.cc:1403] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.090s	user 0.012s	sys 0.000s
I20260709 13:10:58.077481 31711 raft_consensus.cc:515] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.103590 31711 leader_election.cc:290] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:58.105656 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:58.105656 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:58.106233 31261 raft_consensus.cc:3060] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:58.107187 31377 leader_election.cc:343] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.105906 31530 raft_consensus.cc:359] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.107689 31530 raft_consensus.cc:385] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.107946 31530 raft_consensus.cc:740] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.108548 31530 consensus_queue.cc:260] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.110005 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9725e6248e7b4e67b23a3b99967003bf" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:58.110526 31530 ts_tablet_manager.cc:1434] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:10:58.111378 31530 tablet_bootstrap.cc:492] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25: Bootstrap starting.
I20260709 13:10:58.113971 31258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9725e6248e7b4e67b23a3b99967003bf" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:58.115082 31258 raft_consensus.cc:2393] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:58.115967 31156 catalog_manager.cc:5697] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.116946 31451 leader_election.cc:304] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:58.118983 31936 raft_consensus.cc:3060] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.122346 31261 raft_consensus.cc:2468] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:58.122464 31530 tablet_bootstrap.cc:654] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.131587 31533 tablet_bootstrap.cc:492] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:58.132166 31533 ts_tablet_manager.cc:1403] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.006s
I20260709 13:10:58.134593 31377 leader_election.cc:304] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:58.135877 31851 raft_consensus.cc:2804] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.136480 31530 tablet_bootstrap.cc:492] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25: No bootstrap required, opened a new log
I20260709 13:10:58.136863 31937 raft_consensus.cc:493] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.137153 31530 ts_tablet_manager.cc:1403] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.002s
I20260709 13:10:58.137410 31937 raft_consensus.cc:515] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.139834 31937 leader_election.cc:290] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:58.140388 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:58.140769 31530 raft_consensus.cc:359] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.141661 31530 raft_consensus.cc:385] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.141986 31530 raft_consensus.cc:740] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.142458 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:58.142669 31530 consensus_queue.cc:260] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.143148 31261 raft_consensus.cc:2468] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
W20260709 13:10:58.143945 31303 leader_election.cc:343] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.146816 31303 leader_election.cc:304] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:58.147984 31939 raft_consensus.cc:2804] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:58.148332 31939 raft_consensus.cc:493] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.148665 31939 raft_consensus.cc:3060] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.150262 31851 raft_consensus.cc:697] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:58.151381 31530 ts_tablet_manager.cc:1434] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:58.151414 31851 consensus_queue.cc:237] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.169693 31937 consensus_queue.cc:1048] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.170396 31533 raft_consensus.cc:359] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.172909 31533 raft_consensus.cc:385] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.173166 31533 raft_consensus.cc:740] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.173933 31533 consensus_queue.cc:260] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.180471 31533 ts_tablet_manager.cc:1434] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 13:10:58.181360 31533 tablet_bootstrap.cc:492] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:58.187945 31898 raft_consensus.cc:493] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.188453 31898 raft_consensus.cc:515] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.190423 31898 leader_election.cc:290] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:58.191943 31533 tablet_bootstrap.cc:654] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.194656 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:58.195500 31262 raft_consensus.cc:2468] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 0.
I20260709 13:10:58.199324 31377 leader_election.cc:304] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:58.200701 31663 consensus_queue.cc:1048] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:58.202585 31663 raft_consensus.cc:2804] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:58.203040 31663 raft_consensus.cc:493] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.203442 31663 raft_consensus.cc:3060] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.215250 31533 tablet_bootstrap.cc:492] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:58.215749 31533 ts_tablet_manager.cc:1403] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.008s
I20260709 13:10:58.218257 31533 raft_consensus.cc:359] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.218888 31533 raft_consensus.cc:385] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.219152 31533 raft_consensus.cc:740] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.219758 31533 consensus_queue.cc:260] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.224941 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:58.227916 31533 ts_tablet_manager.cc:1434] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:10:58.228860 31533 tablet_bootstrap.cc:492] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:58.229164 31891 raft_consensus.cc:493] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.229688 31891 raft_consensus.cc:515] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.231618 31891 leader_election.cc:290] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
W20260709 13:10:58.227957 31377 leader_election.cc:343] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.233192 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfb2b2292c140bbaa93438cce1746cb" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:58.235658 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfb2b2292c140bbaa93438cce1746cb" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:58.236312 31336 raft_consensus.cc:2393] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:58.240861 31448 leader_election.cc:304] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:10:58.244578 31891 raft_consensus.cc:3060] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.246523 31663 raft_consensus.cc:515] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.248838 31485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:58.249500 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:58.250039 31262 raft_consensus.cc:3060] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:58.253790 31377 leader_election.cc:343] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.261062 31533 tablet_bootstrap.cc:654] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.261498 31485 raft_consensus.cc:3060] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.266669 31262 raft_consensus.cc:2468] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:10:58.267489 31663 leader_election.cc:290] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:58.269522 31377 leader_election.cc:304] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6
I20260709 13:10:58.275523 31891 raft_consensus.cc:2749] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.271323 31663 raft_consensus.cc:2804] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.276836 31663 raft_consensus.cc:697] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:10:58.277843 31663 consensus_queue.cc:237] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.282377 31936 raft_consensus.cc:2749] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.282996 31930 consensus_queue.cc:1048] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.288918 31913 consensus_queue.cc:1048] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.285655 31533 tablet_bootstrap.cc:492] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:58.290334 31533 ts_tablet_manager.cc:1403] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.016s
I20260709 13:10:58.209079 31939 raft_consensus.cc:515] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.292851 31533 raft_consensus.cc:359] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.293507 31533 raft_consensus.cc:385] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.293252 31663 consensus_queue.cc:1048] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.293797 31533 raft_consensus.cc:740] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.294443 31533 consensus_queue.cc:260] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.296464 31533 ts_tablet_manager.cc:1434] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:58.297355 31533 tablet_bootstrap.cc:492] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:58.302906 31533 tablet_bootstrap.cc:654] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.313663 31936 raft_consensus.cc:493] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.314177 31936 raft_consensus.cc:515] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.315017 31930 consensus_queue.cc:1048] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.316406 31936 leader_election.cc:290] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:58.318084 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:58.318730 31262 raft_consensus.cc:2468] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:58.319139 31851 consensus_queue.cc:1048] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.319798 31451 leader_election.cc:304] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:58.322395 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:58.322615 31936 raft_consensus.cc:2804] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:58.322996 31936 raft_consensus.cc:493] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.323237 31334 raft_consensus.cc:2393] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:58.323310 31936 raft_consensus.cc:3060] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.324116 31898 consensus_queue.cc:1048] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.332739 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:58.336315 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:58.336946 31262 raft_consensus.cc:3060] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.340344 31978 raft_consensus.cc:493] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.293036 31939 leader_election.cc:290] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:58.344425 31978 raft_consensus.cc:515] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.347858 31936 raft_consensus.cc:515] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.391669 31938 raft_consensus.cc:493] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.392184 31938 raft_consensus.cc:515] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.396415 31978 leader_election.cc:290] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:58.407430 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfef8e9e4964ab3b8b7c10e84a9d293" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:58.408090 31409 raft_consensus.cc:2468] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 0.
I20260709 13:10:58.411269 31938 leader_election.cc:290] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:58.412467 31301 leader_election.cc:304] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:10:58.400974 31262 raft_consensus.cc:2468] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:58.418134 31303 leader_election.cc:304] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:10:58.425243 31533 tablet_bootstrap.cc:492] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:58.427310 31533 ts_tablet_manager.cc:1403] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.130s	user 0.020s	sys 0.014s
I20260709 13:10:58.449512 31481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfef8e9e4964ab3b8b7c10e84a9d293" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:10:58.426491 31939 raft_consensus.cc:2804] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:10:58.455324 31207 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:58.452447 31160 catalog_manager.cc:5697] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.454028 31480 raft_consensus.cc:2393] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 07b814fcedea4142ac09385b680cae3a in current term 1: Already voted for candidate 001cac731610478a900ad62e830aaff6 in this term.
I20260709 13:10:58.452553 31262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1eac6b3d8a47b59679e4aee4293326" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:58.458858 31160 catalog_manager.cc:5697] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.464119 31936 leader_election.cc:290] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:10:58.455032 31983 raft_consensus.cc:2804] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.474872 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:10:58.459147 31939 raft_consensus.cc:493] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:58.460937 31303 leader_election.cc:343] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.462906 31336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1eac6b3d8a47b59679e4aee4293326" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:58.465255 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
I20260709 13:10:58.499106 31261 raft_consensus.cc:2393] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:58.502422 31448 leader_election.cc:304] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:58.507640 31533 raft_consensus.cc:359] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.508320 31533 raft_consensus.cc:385] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.508563 31533 raft_consensus.cc:740] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.501250 31983 raft_consensus.cc:697] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:58.509166 31533 consensus_queue.cc:260] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.510206 31983 consensus_queue.cc:237] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:10:58.512050 31533 ts_tablet_manager.cc:1434] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.084s	user 0.005s	sys 0.000s
I20260709 13:10:58.512924 31533 tablet_bootstrap.cc:492] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6: Bootstrap starting.
I20260709 13:10:58.501576 31939 raft_consensus.cc:3060] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.532116 31663 consensus_queue.cc:1048] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.538619 31663 consensus_queue.cc:1048] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.539018 31949 raft_consensus.cc:493] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.539489 31949 raft_consensus.cc:515] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
W20260709 13:10:58.546954 31225 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39075, user_credentials={real_user=slave}} blocked reactor thread for 70508.3us
I20260709 13:10:58.568444 31994 raft_consensus.cc:2749] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:58.585181 31533 tablet_bootstrap.cc:654] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.590890 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:58.610129 31663 consensus_queue.cc:1048] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.618961 31949 leader_election.cc:290] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:10:58.634908 31448 leader_election.cc:304] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:58.622686 31939 raft_consensus.cc:515] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.654719 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:58.658625 31997 raft_consensus.cc:2749] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.653676 31891 raft_consensus.cc:493] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.676410 31891 raft_consensus.cc:515] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:10:58.678965 31891 leader_election.cc:290] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
W20260709 13:10:58.682076 31995 log.cc:927] Time spent T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:10:58.684583 32002 log.cc:927] Time spent T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:10:58.676767 31991 consensus_queue.cc:1048] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 13:10:58.722219 31451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:36133, user_credentials={real_user=slave}} blocked reactor thread for 57276.8us
I20260709 13:10:58.722930 31451 leader_election.cc:304] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:58.725106 31261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b6f0017ce84a4c9822ffd27e053405" is_pre_election: true
I20260709 13:10:58.725811 31261 raft_consensus.cc:2393] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 51b0533abe064293ac5047de0ac9bf25 in this term.
I20260709 13:10:58.729894 31938 raft_consensus.cc:2749] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.741736 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:58.742206 31913 consensus_queue.cc:1048] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.780019 31533 tablet_bootstrap.cc:492] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6: No bootstrap required, opened a new log
I20260709 13:10:58.780503 31533 ts_tablet_manager.cc:1403] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6: Time spent bootstrapping tablet: real 0.268s	user 0.018s	sys 0.009s
I20260709 13:10:58.783164 31533 raft_consensus.cc:359] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.783861 31533 raft_consensus.cc:385] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.784138 31533 raft_consensus.cc:740] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001cac731610478a900ad62e830aaff6, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.788017 31533 consensus_queue.cc:260] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.795145 31936 raft_consensus.cc:493] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.796744 31936 raft_consensus.cc:515] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.798718 31936 leader_election.cc:290] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
W20260709 13:10:58.800993 31374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39075, user_credentials={real_user=slave}} blocked reactor thread for 100272us
I20260709 13:10:58.801222 31939 leader_election.cc:290] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:10:58.803797 31533 ts_tablet_manager.cc:1434] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 13:10:58.812809 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfef8e9e4964ab3b8b7c10e84a9d293" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:10:58.814018 31335 raft_consensus.cc:2393] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 001cac731610478a900ad62e830aaff6 in current term 1: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:10:58.803236 31449 leader_election.cc:304] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:10:58.817866 32008 raft_consensus.cc:3060] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:58.804414 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:36133, user_credentials={real_user=slave}} blocked reactor thread for 64374.7us
I20260709 13:10:58.830979 31481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfef8e9e4964ab3b8b7c10e84a9d293" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:10:58.831544 31481 raft_consensus.cc:3060] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.834877 31409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfef8e9e4964ab3b8b7c10e84a9d293" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:10:58.835650 31409 raft_consensus.cc:2468] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 0.
I20260709 13:10:58.843290 31449 leader_election.cc:304] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a
I20260709 13:10:58.846810 32008 raft_consensus.cc:2749] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:58.863682 31377 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:36133, user_credentials={real_user=slave}} blocked reactor thread for 80161.7us
I20260709 13:10:58.869468 31483 raft_consensus.cc:3060] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.870254 31408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfef8e9e4964ab3b8b7c10e84a9d293" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:10:58.884732 31408 raft_consensus.cc:3060] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.897713 31711 consensus_queue.cc:1048] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:58.907366 31408 raft_consensus.cc:2468] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:58.910583 31157 catalog_manager.cc:5697] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:58.924575 31481 raft_consensus.cc:2468] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:10:58.926224 31936 raft_consensus.cc:2764] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:10:58.964380 31303 leader_election.cc:304] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:10:58.979044 31851 consensus_queue.cc:1048] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.984126 31983 raft_consensus.cc:2804] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.985172 31983 raft_consensus.cc:697] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:10:58.986310 31983 consensus_queue.cc:237] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:10:58.988574 31724 consensus_queue.cc:1048] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:59.032194 31939 consensus_queue.cc:1048] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:59.068874 31983 consensus_queue.cc:1048] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:59.153147 31160 catalog_manager.cc:5697] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 1 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:59.522984 31978 consensus_queue.cc:1048] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:59.562587 31978 consensus_queue.cc:1048] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:59.880121 21629 tablet_server.cc:179] TabletServer@127.21.31.65:0 shutting down...
W20260709 13:10:59.958698 31451 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 13:11:00.010342 31377 consensus_peers.cc:597] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.075004 31451 consensus_peers.cc:597] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.079070 31451 consensus_peers.cc:597] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.095250 31451 consensus_peers.cc:597] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.104713 31451 consensus_peers.cc:597] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.113517 31451 consensus_peers.cc:597] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.125643 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:36133, user_credentials={real_user=slave}} blocked reactor thread for 59922.6us
W20260709 13:11:00.141678 31451 consensus_peers.cc:597] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.196565 31377 consensus_peers.cc:597] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.199018 31377 consensus_peers.cc:597] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.203354 31377 consensus_peers.cc:597] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.244385 31303 consensus_peers.cc:597] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.245246 31303 consensus_peers.cc:597] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.246305 31303 consensus_peers.cc:597] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.246970 31303 consensus_peers.cc:597] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.256217 31226 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:45117, user_credentials={real_user=slave}} blocked reactor thread for 43489.5us
W20260709 13:11:00.264501 31303 consensus_peers.cc:597] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.265203 31303 consensus_peers.cc:597] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.270936 31451 consensus_peers.cc:597] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.275195 31377 consensus_peers.cc:597] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.275408 31303 consensus_peers.cc:597] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.275857 31377 consensus_peers.cc:597] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.275974 31303 consensus_peers.cc:597] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.276458 31377 consensus_peers.cc:597] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.276633 31303 consensus_peers.cc:597] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.277181 31303 consensus_peers.cc:597] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.278719 31303 consensus_peers.cc:597] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.285955 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:11:00.286710 31377 consensus_peers.cc:597] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.287999 21629 tablet_replica.cc:333] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
W20260709 13:11:00.288584 31377 consensus_peers.cc:597] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.289573 21629 raft_consensus.cc:2243] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.290612 21629 raft_consensus.cc:2272] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:00.291646 31377 consensus_peers.cc:597] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.294493 21629 tablet_replica.cc:333] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.295611 21629 raft_consensus.cc:2243] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.296852 21629 raft_consensus.cc:2272] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:00.297266 31303 consensus_peers.cc:597] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.297849 31303 consensus_peers.cc:597] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:00.298292 31303 consensus_peers.cc:597] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.301903 21629 tablet_replica.cc:333] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.304122 21629 raft_consensus.cc:2243] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:00.305653 21629 raft_consensus.cc:2272] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.309199 21629 tablet_replica.cc:333] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.310181 21629 raft_consensus.cc:2243] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.310890 21629 raft_consensus.cc:2272] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.314003 21629 tablet_replica.cc:333] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.315030 21629 raft_consensus.cc:2243] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.315798 21629 raft_consensus.cc:2272] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.318877 21629 tablet_replica.cc:333] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.319774 21629 raft_consensus.cc:2243] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.320461 21629 raft_consensus.cc:2272] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.323621 21629 tablet_replica.cc:333] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.324716 21629 raft_consensus.cc:2243] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.325501 21629 raft_consensus.cc:2272] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:00.328833 31303 consensus_peers.cc:597] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.328969 21629 tablet_replica.cc:333] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.330121 21629 raft_consensus.cc:2243] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.330746 21629 raft_consensus.cc:2272] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.333916 21629 tablet_replica.cc:333] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.334738 21629 raft_consensus.cc:2243] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:00.335917 21629 raft_consensus.cc:2272] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.339283 21629 tablet_replica.cc:333] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.340152 21629 raft_consensus.cc:2243] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.340838 21629 raft_consensus.cc:2272] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.344199 21629 tablet_replica.cc:333] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.345134 21629 raft_consensus.cc:2243] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.345947 21629 raft_consensus.cc:2272] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:00.355033 31451 consensus_peers.cc:597] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.356037 21629 tablet_replica.cc:333] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.356930 21629 raft_consensus.cc:2243] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:00.358059 21629 raft_consensus.cc:2272] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:00.361645 31303 consensus_peers.cc:597] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.362044 21629 tablet_replica.cc:333] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.362920 21629 raft_consensus.cc:2243] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.363662 21629 raft_consensus.cc:2272] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.366633 21629 tablet_replica.cc:333] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.367616 21629 raft_consensus.cc:2243] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.368480 21629 raft_consensus.cc:2272] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.372624 21629 tablet_replica.cc:333] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.381162 21629 raft_consensus.cc:2243] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.383222 21629 raft_consensus.cc:2272] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.387216 21629 tablet_replica.cc:333] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.388064 21629 raft_consensus.cc:2243] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.388873 21629 raft_consensus.cc:2272] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.391948 21629 tablet_replica.cc:333] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.392751 21629 raft_consensus.cc:2243] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.393782 21629 raft_consensus.cc:2272] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.397547 21629 tablet_replica.cc:333] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.398926 21629 raft_consensus.cc:2243] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.399735 21629 raft_consensus.cc:2272] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.403323 21629 tablet_replica.cc:333] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.404484 21629 raft_consensus.cc:2243] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:00.404664 31303 consensus_peers.cc:597] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.405180 21629 raft_consensus.cc:2272] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.408450 21629 tablet_replica.cc:333] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.409091 21629 raft_consensus.cc:2243] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.409816 21629 raft_consensus.cc:2272] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.414878 21629 tablet_replica.cc:333] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.415567 21629 raft_consensus.cc:2243] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:00.416618 21629 raft_consensus.cc:2272] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.419644 21629 tablet_replica.cc:333] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.420463 21629 raft_consensus.cc:2243] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.421151 21629 raft_consensus.cc:2272] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.424304 21629 tablet_replica.cc:333] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.425433 21629 raft_consensus.cc:2243] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.426064 21629 raft_consensus.cc:2272] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.430105 21629 tablet_replica.cc:333] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
W20260709 13:11:00.431854 31303 consensus_peers.cc:597] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.437595 21629 raft_consensus.cc:2243] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.438287 21629 raft_consensus.cc:2272] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.441124 21629 tablet_replica.cc:333] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.441910 21629 raft_consensus.cc:2243] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.442509 21629 raft_consensus.cc:2272] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.445370 21629 tablet_replica.cc:333] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.446200 21629 raft_consensus.cc:2243] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.446894 21629 raft_consensus.cc:2272] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.450086 21629 tablet_replica.cc:333] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.450752 21629 raft_consensus.cc:2243] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.451465 21629 raft_consensus.cc:2272] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.455502 21629 tablet_replica.cc:333] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.456310 21629 raft_consensus.cc:2243] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:00.457629 21629 raft_consensus.cc:2272] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.460899 21629 tablet_replica.cc:333] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.461803 21629 raft_consensus.cc:2243] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:00.462962 21629 raft_consensus.cc:2272] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.466300 21629 tablet_replica.cc:333] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.466989 21629 raft_consensus.cc:2243] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.467514 21629 raft_consensus.cc:2272] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.470247 21629 tablet_replica.cc:333] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.470968 21629 raft_consensus.cc:2243] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.471603 21629 raft_consensus.cc:2272] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.474248 21629 tablet_replica.cc:333] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.475003 21629 raft_consensus.cc:2243] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:00.476264 21629 raft_consensus.cc:2272] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.479379 21629 tablet_replica.cc:333] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.480240 21629 raft_consensus.cc:2243] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.480962 21629 raft_consensus.cc:2272] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.484244 21629 tablet_replica.cc:333] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.485108 21629 raft_consensus.cc:2243] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.485922 21629 raft_consensus.cc:2272] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.489496 21629 tablet_replica.cc:333] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.490370 21629 raft_consensus.cc:2243] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.490996 21629 raft_consensus.cc:2272] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.496294 21629 tablet_replica.cc:333] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.497078 21629 raft_consensus.cc:2243] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.497776 21629 raft_consensus.cc:2272] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.500802 21629 tablet_replica.cc:333] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.501693 21629 raft_consensus.cc:2243] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.502391 21629 raft_consensus.cc:2272] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.505683 21629 tablet_replica.cc:333] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.506399 21629 raft_consensus.cc:2243] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.507082 21629 raft_consensus.cc:2272] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.510092 21629 tablet_replica.cc:333] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.510921 21629 raft_consensus.cc:2243] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:00.512122 21629 raft_consensus.cc:2272] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.514808 21629 tablet_replica.cc:333] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.515542 21629 raft_consensus.cc:2243] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.516220 21629 raft_consensus.cc:2272] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.519281 21629 tablet_replica.cc:333] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.519999 21629 raft_consensus.cc:2243] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.520699 21629 raft_consensus.cc:2272] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.528903 21629 tablet_replica.cc:333] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.550354 21629 raft_consensus.cc:2243] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.551013 21629 raft_consensus.cc:2272] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.569482 21629 tablet_replica.cc:333] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.570219 21629 raft_consensus.cc:2243] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:00.605963 21629 raft_consensus.cc:2272] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.617556 21629 tablet_replica.cc:333] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.618206 21629 raft_consensus.cc:2243] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.618850 21629 raft_consensus.cc:2272] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.631410 21629 tablet_replica.cc:333] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: stopping tablet replica
I20260709 13:11:00.632053 21629 raft_consensus.cc:2243] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:00.632759 21629 raft_consensus.cc:2272] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:00.787950 21629 tablet_server.cc:196] TabletServer@127.21.31.65:0 shutdown complete.
I20260709 13:11:00.901758 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:00.919222 32046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:00.944068 32047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:00.952522 32049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:00.954638 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:00.956967 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:00.957264 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:00.957523 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602660957506 us; error 0 us; skew 500 ppm
I20260709 13:11:00.958271 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:00.962273 21629 webserver.cc:533] Webserver started at http://127.21.31.65:33339/ using document root <none> and password file <none>
I20260709 13:11:00.962843 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:00.963055 21629 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:11:00.972630 21629 fs_manager.cc:714] Time spent opening directory manager: real 0.008s	user 0.006s	sys 0.004s
I20260709 13:11:00.990100 32054 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:00.991027 21629 fs_manager.cc:730] Time spent opening block manager: real 0.009s	user 0.001s	sys 0.007s
I20260709 13:11:00.991345 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "75b6f0017ce84a4c9822ffd27e053405"
format_stamp: "Formatted at 2026-07-09 13:10:48 on dist-test-slave-9s6w"
I20260709 13:11:00.991915 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:01.020814 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:01.022552 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:01.032100 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:01.039364 32061 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:11:01.352547 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:11:01.352965 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.315s	user 0.009s	sys 0.001s
I20260709 13:11:01.353308 21629 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:11:01.358101 32061 tablet_bootstrap.cc:492] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:01.375155 32061 tablet_bootstrap.cc:492] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:01.376147 32061 tablet_bootstrap.cc:492] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:01.376789 32061 ts_tablet_manager.cc:1403] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.019s	user 0.018s	sys 0.000s
I20260709 13:11:01.379794 32061 raft_consensus.cc:359] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:01.380736 32061 raft_consensus.cc:740] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.386137 32061 consensus_queue.cc:260] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:01.393752 32061 ts_tablet_manager.cc:1434] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:11:01.394512 32061 tablet_bootstrap.cc:492] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:01.418021 32061 tablet_bootstrap.cc:492] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:01.418993 32061 tablet_bootstrap.cc:492] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:01.419662 32061 ts_tablet_manager.cc:1403] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.004s
I20260709 13:11:01.422241 32061 raft_consensus.cc:359] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.422924 32061 raft_consensus.cc:740] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.423617 32061 consensus_queue.cc:260] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.425643 32061 ts_tablet_manager.cc:1434] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:01.426386 32061 tablet_bootstrap.cc:492] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:01.441432 32061 tablet_bootstrap.cc:492] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:01.442301 32061 tablet_bootstrap.cc:492] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:01.442883 32061 ts_tablet_manager.cc:1403] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:11:01.445398 32061 raft_consensus.cc:359] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.446060 32061 raft_consensus.cc:740] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.446899 32061 consensus_queue.cc:260] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.449028 32061 ts_tablet_manager.cc:1434] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:01.449801 32061 tablet_bootstrap.cc:492] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:01.486788 32034 raft_consensus.cc:493] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.487494 32034 raft_consensus.cc:515] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:01.489791 32034 leader_election.cc:290] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.492062 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1eac6b3d8a47b59679e4aee4293326" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:11:01.496726 32032 raft_consensus.cc:493] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.501590 32032 raft_consensus.cc:515] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.504070 32032 leader_election.cc:290] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.507725 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad34ee917be4455bbe71da15c523cee" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:11:01.511740 31377 leader_election.cc:336] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.513783 31377 leader_election.cc:304] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:01.517192 32032 raft_consensus.cc:2749] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:01.527577 32041 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.528100 32041 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.530296 32041 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.542471 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:11:01.548601 31303 leader_election.cc:336] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.550690 31303 leader_election.cc:304] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:01.552564 32034 raft_consensus.cc:2749] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:01.556350 31377 leader_election.cc:336] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.564016 31377 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:01.566741 32032 raft_consensus.cc:2749] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:01.598795 32032 raft_consensus.cc:493] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.603358 32003 raft_consensus.cc:493] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.604022 32003 raft_consensus.cc:515] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:01.606528 31983 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.607007 31983 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:01.608919 31983 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:01.610183 32003 leader_election.cc:290] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:01.613364 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:01.615027 31408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:01.599407 32032 raft_consensus.cc:515] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.622684 32032 leader_election.cc:290] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.637557 21629 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:11:01.637923 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.285s	user 0.233s	sys 0.003s
I20260709 13:11:01.670663 31481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "862d691f6d9e4d9fb185be53b562b8ff" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:11:01.675221 31303 leader_election.cc:336] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.675674 31303 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:01.688107 32061 tablet_bootstrap.cc:492] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:01.689129 32061 tablet_bootstrap.cc:492] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:01.689225 31983 raft_consensus.cc:2749] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:01.689942 32061 ts_tablet_manager.cc:1403] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.240s	user 0.071s	sys 0.134s
W20260709 13:11:01.690281 31303 leader_election.cc:336] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.690747 31303 leader_election.cc:304] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:01.693037 32061 raft_consensus.cc:359] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:01.693760 32061 raft_consensus.cc:740] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.694522 32061 consensus_queue.cc:260] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
W20260709 13:11:01.696127 31377 leader_election.cc:336] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.696607 31377 leader_election.cc:304] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:01.697978 32061 ts_tablet_manager.cc:1434] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:11:01.698951 32061 tablet_bootstrap.cc:492] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:01.701759 31983 raft_consensus.cc:2749] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:01.705140 32068 raft_consensus.cc:2749] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:01.705921 32068 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.706346 32068 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:01.708387 32068 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:01.709146 32040 raft_consensus.cc:493] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.709682 32040 raft_consensus.cc:515] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:01.712049 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:01.712668 31337 raft_consensus.cc:2468] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 2.
I20260709 13:11:01.713423 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1eac6b3d8a47b59679e4aee4293326" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:01.714013 31334 raft_consensus.cc:2468] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:11:01.715302 31448 leader_election.cc:304] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:11:01.719903 31374 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:01.721305 32032 raft_consensus.cc:2804] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:01.721722 32032 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.722182 32032 raft_consensus.cc:3060] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.724856 31983 raft_consensus.cc:493] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.721304 32040 leader_election.cc:290] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:01.725489 31983 raft_consensus.cc:515] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:01.725919 32040 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.726347 32040 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.727542 31983 leader_election.cc:290] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:01.728315 32040 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.728839 32040 raft_consensus.cc:2804] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:01.729158 32040 raft_consensus.cc:493] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.729269 31408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b810039c1e415192775c9f611cd825" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:01.729581 32040 raft_consensus.cc:3060] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:01.739351 31377 leader_election.cc:336] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.741771 31408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:01.742564 31408 raft_consensus.cc:2468] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 2.
I20260709 13:11:01.745007 32072 raft_consensus.cc:493] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.745533 32072 raft_consensus.cc:515] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.746487 31449 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:01.747562 32072 leader_election.cc:290] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.748107 32072 raft_consensus.cc:2804] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:01.748428 32072 raft_consensus.cc:493] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.748785 32072 raft_consensus.cc:3060] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.748651 31408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad34ee917be4455bbe71da15c523cee" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:01.749636 31408 raft_consensus.cc:2468] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
I20260709 13:11:01.751231 32032 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:11:01.758402 31377 leader_election.cc:336] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.751142 31449 leader_election.cc:304] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:01.761965 32073 raft_consensus.cc:2804] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:01.762329 32073 raft_consensus.cc:493] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.762681 32073 raft_consensus.cc:3060] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:01.761225 32040 raft_consensus.cc:515] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:01.762835 31983 raft_consensus.cc:493] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.767529 31983 raft_consensus.cc:515] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:01.768625 32040 leader_election.cc:290] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:01.769563 31983 leader_election.cc:290] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.769697 32032 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 3 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:01.773025 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:01.773967 31334 raft_consensus.cc:3060] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.775218 32073 raft_consensus.cc:515] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.777485 32072 raft_consensus.cc:515] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.779840 32072 leader_election.cc:290] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.780156 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b9a979d5b8b4d02b133ffc4078d9cbf" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:01.780763 31410 raft_consensus.cc:3060] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:01.781978 31334 raft_consensus.cc:2468] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 3.
I20260709 13:11:01.782060 32073 leader_election.cc:290] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.782810 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad34ee917be4455bbe71da15c523cee" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:01.783376 31411 raft_consensus.cc:3060] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:01.783262 31374 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:01.786144 32070 raft_consensus.cc:2804] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:01.786628 32070 raft_consensus.cc:697] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 3 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:11:01.788511 31410 raft_consensus.cc:2468] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 3.
I20260709 13:11:01.788344 32070 consensus_queue.cc:237] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:01.792999 31411 raft_consensus.cc:2468] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 2.
I20260709 13:11:01.793501 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1eac6b3d8a47b59679e4aee4293326" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
W20260709 13:11:01.793819 31451 leader_election.cc:336] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.794121 31449 leader_election.cc:304] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:01.794188 31333 raft_consensus.cc:3060] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:01.795060 32040 raft_consensus.cc:493] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.795564 32040 raft_consensus.cc:515] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.799326 32040 leader_election.cc:290] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:01.801157 32040 raft_consensus.cc:2804] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:01.798553 31449 leader_election.cc:304] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:01.801810 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "862d691f6d9e4d9fb185be53b562b8ff" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:01.805493 31410 raft_consensus.cc:2468] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 1.
W20260709 13:11:01.798765 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.65:36133, user_credentials={real_user=slave}} blocked reactor thread for 44669.4us
W20260709 13:11:01.811995 31451 leader_election.cc:336] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
W20260709 13:11:01.814101 31451 leader_election.cc:336] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.815639 31333 raft_consensus.cc:2468] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 2.
I20260709 13:11:01.815667 32074 raft_consensus.cc:2804] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:01.816543 31449 leader_election.cc:304] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:01.817528 32061 tablet_bootstrap.cc:492] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:01.820619 31157 catalog_manager.cc:5697] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 2 to 3, leader changed from 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65) to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 3 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:01.832229 32072 raft_consensus.cc:2804] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:01.832826 32061 tablet_bootstrap.cc:492] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:01.832914 32072 raft_consensus.cc:493] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.833634 32061 ts_tablet_manager.cc:1403] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.135s	user 0.013s	sys 0.004s
I20260709 13:11:01.833750 32072 raft_consensus.cc:3060] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:01.832376 32074 raft_consensus.cc:697] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 2 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
W20260709 13:11:01.836328 31303 leader_election.cc:336] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.835139 31448 leader_election.cc:304] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:11:01.838680 31303 leader_election.cc:304] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
W20260709 13:11:01.840905 31451 leader_election.cc:336] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
W20260709 13:11:01.842223 31451 leader_election.cc:336] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
W20260709 13:11:01.842861 31451 leader_election.cc:336] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
W20260709 13:11:01.843502 31451 leader_election.cc:336] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.846004 32061 raft_consensus.cc:359] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:01.846715 32061 raft_consensus.cc:740] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.847421 32061 consensus_queue.cc:260] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:01.849666 32061 ts_tablet_manager.cc:1434] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:11:01.850466 32061 tablet_bootstrap.cc:492] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:01.845501 32082 raft_consensus.cc:2804] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:01.859210 32074 consensus_queue.cc:237] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.847148 31983 raft_consensus.cc:2749] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:01.854841 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:11:01.841562 32040 raft_consensus.cc:697] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 3 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:11:01.867057 31480 raft_consensus.cc:2468] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:11:01.861770 32082 raft_consensus.cc:697] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 2 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:11:01.881862 32082 consensus_queue.cc:237] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:01.899959 32068 raft_consensus.cc:493] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.900681 32068 raft_consensus.cc:515] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:01.911669 32072 raft_consensus.cc:515] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
W20260709 13:11:01.914386 31377 leader_election.cc:336] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.922012 32068 leader_election.cc:290] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:01.961957 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b810039c1e415192775c9f611cd825" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:01.962812 31334 raft_consensus.cc:2468] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:11:01.975615 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "862d691f6d9e4d9fb185be53b562b8ff" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:01.976248 31410 raft_consensus.cc:3060] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:01.959702 32003 raft_consensus.cc:493] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.987577 31410 raft_consensus.cc:2468] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 2.
I20260709 13:11:01.989174 31449 leader_election.cc:304] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
W20260709 13:11:01.989457 31303 leader_election.cc:336] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:01.945860 32040 consensus_queue.cc:237] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:01.995362 31374 leader_election.cc:304] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:01.997013 32072 leader_election.cc:290] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:02.002241 32070 raft_consensus.cc:2804] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:02.015177 32068 raft_consensus.cc:493] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:01.992897 32003 raft_consensus.cc:515] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.010216 32083 raft_consensus.cc:2804] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:02.041198 32083 raft_consensus.cc:697] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 2 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:11:02.042196 32083 consensus_queue.cc:237] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:02.015686 32070 raft_consensus.cc:493] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:02.045727 32070 raft_consensus.cc:3060] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.050873 32003 leader_election.cc:290] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:02.019034 32068 raft_consensus.cc:515] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.053368 32068 leader_election.cc:290] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:02.038832 32061 tablet_bootstrap.cc:492] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:11:02.066989 31377 leader_election.cc:336] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:02.068167 32061 tablet_bootstrap.cc:492] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:02.068876 32061 ts_tablet_manager.cc:1403] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.219s	user 0.010s	sys 0.016s
I20260709 13:11:02.071522 32061 raft_consensus.cc:359] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:02.075789 32061 raft_consensus.cc:740] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
W20260709 13:11:02.079977 31451 leader_election.cc:336] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:02.081645 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:02.082536 31334 raft_consensus.cc:2468] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 1.
I20260709 13:11:02.080602 32061 consensus_queue.cc:260] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:02.099226 32061 ts_tablet_manager.cc:1434] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.030s	user 0.002s	sys 0.005s
I20260709 13:11:02.123404 31374 leader_election.cc:304] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:02.125399 32061 tablet_bootstrap.cc:492] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:02.127003 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:02.127759 31410 raft_consensus.cc:2468] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 1.
I20260709 13:11:02.130199 32070 raft_consensus.cc:515] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.133052 32070 leader_election.cc:290] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:02.134357 32070 raft_consensus.cc:493] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:02.136238 32068 raft_consensus.cc:2804] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:02.136691 32068 raft_consensus.cc:493] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:02.135608 32070 raft_consensus.cc:515] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.140486 31303 leader_election.cc:304] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:02.137085 32068 raft_consensus.cc:3060] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.147661 31156 catalog_manager.cc:5697] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 2 to 3, leader changed from 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65) to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 3 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:02.168519 31374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:39075, user_credentials={real_user=slave}} blocked reactor thread for 46264.1us
I20260709 13:11:02.173835 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b810039c1e415192775c9f611cd825" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:02.174413 31334 raft_consensus.cc:3060] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.169847 32088 raft_consensus.cc:2804] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:02.184696 32088 raft_consensus.cc:493] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:02.185238 32088 raft_consensus.cc:3060] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.187153 31334 raft_consensus.cc:2468] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 2.
I20260709 13:11:02.199180 31301 leader_election.cc:304] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:02.200664 32101 raft_consensus.cc:2804] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:02.201013 32101 raft_consensus.cc:493] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:02.201381 32101 raft_consensus.cc:3060] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.193638 32068 raft_consensus.cc:515] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.212515 32068 leader_election.cc:290] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:02.218523 31374 leader_election.cc:304] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:02.220252 32111 raft_consensus.cc:2804] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:02.220702 32111 raft_consensus.cc:697] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 2 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:11:02.223660 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:02.224493 31333 raft_consensus.cc:3060] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.238124 32088 raft_consensus.cc:515] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:02.241398 32111 consensus_queue.cc:237] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.258435 32070 leader_election.cc:290] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:02.258767 31333 raft_consensus.cc:2468] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0533abe064293ac5047de0ac9bf25 in term 2.
I20260709 13:11:02.261152 31374 leader_election.cc:304] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:02.261994 32070 raft_consensus.cc:2804] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:02.262449 32070 raft_consensus.cc:697] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 2 LEADER]: Becoming Leader. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:11:02.263590 32101 raft_consensus.cc:515] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.263247 32070 consensus_queue.cc:237] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:11:02.269290 31451 consensus_peers.cc:597] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.274791 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "51b0533abe064293ac5047de0ac9bf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:02.275631 31333 raft_consensus.cc:2393] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51b0533abe064293ac5047de0ac9bf25 in current term 2: Already voted for candidate 07b814fcedea4142ac09385b680cae3a in this term.
I20260709 13:11:02.278609 32088 leader_election.cc:290] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:02.280076 32061 tablet_bootstrap.cc:492] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:02.280157 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b34e35ea0429ea3c77b7d89fd748a" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:02.280762 31411 raft_consensus.cc:3060] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.281038 32061 tablet_bootstrap.cc:492] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:02.289439 31411 raft_consensus.cc:2468] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 2.
I20260709 13:11:02.291453 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "07b814fcedea4142ac09385b680cae3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:02.292068 31483 raft_consensus.cc:3060] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:02.295394 32061 ts_tablet_manager.cc:1403] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.170s	user 0.019s	sys 0.006s
I20260709 13:11:02.299116 31483 raft_consensus.cc:2468] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b814fcedea4142ac09385b680cae3a in term 2.
W20260709 13:11:02.299587 31451 consensus_peers.cc:597] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.300209 31301 leader_election.cc:304] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:02.302956 32102 raft_consensus.cc:2804] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:02.303402 32102 raft_consensus.cc:697] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 2 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:02.303709 32061 raft_consensus.cc:359] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.304383 32061 raft_consensus.cc:740] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.304212 32102 consensus_queue.cc:237] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:11:02.305510 31303 leader_election.cc:336] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:02.300762 32101 leader_election.cc:290] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 election: Requested vote from peers 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:02.305044 32061 consensus_queue.cc:260] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:11:02.320328 31303 consensus_peers.cc:597] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:02.320950 31303 leader_election.cc:336] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:02.323097 32061 ts_tablet_manager.cc:1434] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:11:02.327282 32061 tablet_bootstrap.cc:492] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:02.332261 31333 raft_consensus.cc:1275] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 3 FOLLOWER]: Refusing update from remote peer 51b0533abe064293ac5047de0ac9bf25: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:02.332340 31451 consensus_peers.cc:597] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.334860 32070 consensus_queue.cc:1048] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:02.370319 31160 catalog_manager.cc:5697] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 1 to 2, leader changed from 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65) to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 2 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:02.372177 31377 leader_election.cc:336] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
W20260709 13:11:02.373014 31377 leader_election.cc:336] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
W20260709 13:11:02.373852 31377 leader_election.cc:336] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:02.374228 31377 leader_election.cc:304] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25; no voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:02.375578 32111 raft_consensus.cc:2749] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:02.376750 31156 catalog_manager.cc:5697] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 1 to 2, leader changed from 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65) to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:02.379206 31156 catalog_manager.cc:5697] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 reported cstate change: term changed from 1 to 2, leader changed from 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65) to 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "51b0533abe064293ac5047de0ac9bf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:02.380715 31337 raft_consensus.cc:1275] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 001cac731610478a900ad62e830aaff6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:02.372545 31157 catalog_manager.cc:5697] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 1 to 2, leader changed from 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65) to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 2 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.378106 31303 leader_election.cc:304] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a; no voters: 
I20260709 13:11:02.382189 32083 consensus_queue.cc:1048] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:02.382648 32088 raft_consensus.cc:2804] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:02.383167 32088 raft_consensus.cc:697] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 2 LEADER]: Becoming Leader. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
W20260709 13:11:02.384248 31377 consensus_peers.cc:597] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.384063 32088 consensus_queue.cc:237] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
W20260709 13:11:02.384759 31377 consensus_peers.cc:597] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.387017 31157 catalog_manager.cc:5697] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 1 to 2, leader changed from 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65) to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 2 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.392310 31157 catalog_manager.cc:5697] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 1 to 2, leader changed from 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65) to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 2 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:02.406196 31451 consensus_peers.cc:597] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:02.414450 31451 consensus_peers.cc:597] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:02.404516 31303 leader_election.cc:336] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
W20260709 13:11:02.427448 31377 consensus_peers.cc:597] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.431107 31408 raft_consensus.cc:1275] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 001cac731610478a900ad62e830aaff6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:02.433535 32085 consensus_queue.cc:1048] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:02.440924 31411 raft_consensus.cc:1275] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 001cac731610478a900ad62e830aaff6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:02.443003 32085 consensus_queue.cc:1048] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:02.473752 32061 tablet_bootstrap.cc:492] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:02.474689 32061 tablet_bootstrap.cc:492] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:02.475265 32061 ts_tablet_manager.cc:1403] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.148s	user 0.012s	sys 0.004s
I20260709 13:11:02.477509 32061 raft_consensus.cc:359] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.478139 32061 raft_consensus.cc:740] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.478837 32061 consensus_queue.cc:260] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:11:02.479499 31451 consensus_peers.cc:597] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:02.480788 32061 ts_tablet_manager.cc:1434] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:11:02.481564 32061 tablet_bootstrap.cc:492] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:02.495826 32061 tablet_bootstrap.cc:492] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:02.496680 32061 tablet_bootstrap.cc:492] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:02.497265 32061 ts_tablet_manager.cc:1403] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:11:02.500097 32061 raft_consensus.cc:359] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.500905 32061 raft_consensus.cc:740] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.503618 32061 consensus_queue.cc:260] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.507104 32061 ts_tablet_manager.cc:1434] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:11:02.508186 32061 tablet_bootstrap.cc:492] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
W20260709 13:11:02.545009 31451 consensus_peers.cc:597] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:02.554039 31303 consensus_peers.cc:597] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:02.554577 31303 consensus_peers.cc:597] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:02.554980 31303 consensus_peers.cc:597] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:02.559981 31377 consensus_peers.cc:597] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.571141 31411 raft_consensus.cc:1275] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Refusing update from remote peer 001cac731610478a900ad62e830aaff6: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:02.573309 31303 consensus_peers.cc:597] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.573875 31157 catalog_manager.cc:5697] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a reported cstate change: term changed from 1 to 2, leader changed from 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65) to 07b814fcedea4142ac09385b680cae3a (127.21.31.66). New cstate: current_term: 2 leader_uuid: "07b814fcedea4142ac09385b680cae3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:02.575779 32083 consensus_queue.cc:1048] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:02.603714 31377 consensus_peers.cc:597] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.618536 32061 tablet_bootstrap.cc:492] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:02.619608 32061 tablet_bootstrap.cc:492] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
W20260709 13:11:02.619808 31377 consensus_peers.cc:597] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.620205 32061 ts_tablet_manager.cc:1403] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.112s	user 0.024s	sys 0.000s
W20260709 13:11:02.590564 31303 consensus_peers.cc:597] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:02.621567 31303 consensus_peers.cc:597] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.634508 32061 raft_consensus.cc:359] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:11:02.635188 32061 raft_consensus.cc:740] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.635939 32061 consensus_queue.cc:260] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
W20260709 13:11:02.636484 31451 consensus_peers.cc:597] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.637982 32061 ts_tablet_manager.cc:1434] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
W20260709 13:11:02.641815 31451 consensus_peers.cc:597] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.645823 32061 tablet_bootstrap.cc:492] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
W20260709 13:11:02.654645 31303 consensus_peers.cc:597] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:02.660163 31377 consensus_peers.cc:597] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:02.667589 31377 consensus_peers.cc:597] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.674502 32061 tablet_bootstrap.cc:492] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:02.675388 32061 tablet_bootstrap.cc:492] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:02.675998 32061 ts_tablet_manager.cc:1403] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.008s
I20260709 13:11:02.682627 32061 raft_consensus.cc:359] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:02.683246 32061 raft_consensus.cc:740] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.684712 32061 consensus_queue.cc:260] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
W20260709 13:11:02.702860 31377 consensus_peers.cc:597] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.710016 32061 ts_tablet_manager.cc:1434] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:11:02.711151 32061 tablet_bootstrap.cc:492] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
W20260709 13:11:02.720031 31303 consensus_peers.cc:597] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.739039 32061 tablet_bootstrap.cc:492] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:02.740231 32061 tablet_bootstrap.cc:492] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:02.741072 32061 ts_tablet_manager.cc:1403] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.004s
I20260709 13:11:02.748519 32061 raft_consensus.cc:359] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:02.749384 32061 raft_consensus.cc:740] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.750350 32061 consensus_queue.cc:260] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:02.753294 32061 ts_tablet_manager.cc:1434] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:11:02.762562 32061 tablet_bootstrap.cc:492] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:02.763293 31337 raft_consensus.cc:1275] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 51b0533abe064293ac5047de0ac9bf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:02.757326 31451 consensus_peers.cc:597] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.767316 32107 consensus_queue.cc:1048] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:02.783164 31377 consensus_peers.cc:597] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:02.795277 31303 consensus_peers.cc:597] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.797940 31333 raft_consensus.cc:1275] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 51b0533abe064293ac5047de0ac9bf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:02.809864 32107 consensus_queue.cc:1048] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:02.827410 31377 consensus_peers.cc:597] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.860496 31408 raft_consensus.cc:1275] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 07b814fcedea4142ac09385b680cae3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:02.875836 32102 consensus_queue.cc:1048] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:02.882154 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:44895, user_credentials={real_user=slave}} blocked reactor thread for 64108.4us
W20260709 13:11:02.884817 31377 consensus_peers.cc:597] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:02.901415 31303 consensus_peers.cc:597] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:02.964725 31480 raft_consensus.cc:1275] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 07b814fcedea4142ac09385b680cae3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:02.997131 31377 consensus_peers.cc:597] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:03.098150 32159 log.cc:927] Time spent T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:11:03.113544 32061 tablet_bootstrap.cc:492] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.114548 32061 tablet_bootstrap.cc:492] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.115183 32061 ts_tablet_manager.cc:1403] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.353s	user 0.077s	sys 0.101s
I20260709 13:11:03.121102 32166 consensus_queue.cc:1048] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:03.146505 32063 raft_consensus.cc:493] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.147090 32063 raft_consensus.cc:515] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:03.147148 32061 raft_consensus.cc:359] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.150957 32061 raft_consensus.cc:740] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.152179 32061 consensus_queue.cc:260] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
W20260709 13:11:03.154122 31377 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:44895, user_credentials={real_user=slave}} blocked reactor thread for 47319.9us
I20260709 13:11:03.174198 32063 leader_election.cc:290] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
W20260709 13:11:03.168563 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:44895, user_credentials={real_user=slave}} blocked reactor thread for 56049.2us
W20260709 13:11:03.186645 31451 consensus_peers.cc:597] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:03.187158 31451 consensus_peers.cc:597] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:03.200011 32063 raft_consensus.cc:493] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.200853 32063 raft_consensus.cc:515] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:03.203585 32063 leader_election.cc:290] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
W20260709 13:11:03.207675 31303 consensus_peers.cc:597] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:03.215934 32061 ts_tablet_manager.cc:1434] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.100s	user 0.014s	sys 0.011s
I20260709 13:11:03.220557 32061 tablet_bootstrap.cc:492] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.269253 32174 raft_consensus.cc:493] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.292166 32174 raft_consensus.cc:515] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.335944 32174 leader_election.cc:290] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
W20260709 13:11:03.344187 31451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:36133, user_credentials={real_user=slave}} blocked reactor thread for 46849.1us
I20260709 13:11:03.358801 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:11:03.358898 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db13a06e22e411ba4c29d105c7e93f7" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
W20260709 13:11:03.369452 31377 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:44895, user_credentials={real_user=slave}} blocked reactor thread for 149836us
W20260709 13:11:03.370455 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:44895, user_credentials={real_user=slave}} blocked reactor thread for 73246.1us
W20260709 13:11:03.374516 31303 consensus_peers.cc:597] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:03.381482 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb1e844232d4ed0b8752e51e31204b8" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:03.377481 31408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29510c165a1f48b1949c6ee31e136064" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:03.377648 32174 raft_consensus.cc:493] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.383605 32174 raft_consensus.cc:515] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
W20260709 13:11:03.383170 31303 consensus_peers.cc:597] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:03.387506 31303 consensus_peers.cc:597] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.388070 32056 leader_election.cc:304] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
W20260709 13:11:03.388500 31303 consensus_peers.cc:597] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:03.389685 32063 raft_consensus.cc:2749] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.391340 32063 raft_consensus.cc:493] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.391784 32063 raft_consensus.cc:515] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.390276 31408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6498a8fe8ab41d19cfe4afd8ccfb904" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
W20260709 13:11:03.392637 31374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.66:39075, user_credentials={real_user=slave}} blocked reactor thread for 169295us
I20260709 13:11:03.406989 32063 leader_election.cc:290] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:03.410504 32174 leader_election.cc:290] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
W20260709 13:11:03.452287 31303 consensus_peers.cc:597] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:03.458211 31303 consensus_peers.cc:597] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:03.485348 31334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db13a06e22e411ba4c29d105c7e93f7" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:03.485666 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29510c165a1f48b1949c6ee31e136064" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:03.486778 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:11:03.486873 31481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6498a8fe8ab41d19cfe4afd8ccfb904" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:11:03.487098 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b5b2bc298df4947b958fefdc9c2846e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:03.488699 32058 leader_election.cc:304] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:11:03.489383 32055 leader_election.cc:304] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:11:03.490208 32192 raft_consensus.cc:2749] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.490789 32055 leader_election.cc:304] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:11:03.492022 32055 leader_election.cc:304] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:11:03.492918 32174 raft_consensus.cc:2749] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.493477 32063 raft_consensus.cc:2749] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.494144 32063 raft_consensus.cc:2749] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:03.497253 31303 consensus_peers.cc:597] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:03.541513 32061 tablet_bootstrap.cc:492] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.542598 32061 tablet_bootstrap.cc:492] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.543258 32061 ts_tablet_manager.cc:1403] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.325s	user 0.016s	sys 0.007s
I20260709 13:11:03.545943 32061 raft_consensus.cc:359] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.546619 32061 raft_consensus.cc:740] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.547320 32061 consensus_queue.cc:260] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.549974 32061 ts_tablet_manager.cc:1434] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:11:03.550711 32061 tablet_bootstrap.cc:492] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.562732 32194 raft_consensus.cc:493] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.564636 32194 raft_consensus.cc:515] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:03.567955 32194 leader_election.cc:290] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:03.569664 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:03.573035 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a57045456429c857b8fb94b54c356" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:03.575414 32055 leader_election.cc:304] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:11:03.578816 32061 tablet_bootstrap.cc:492] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.579697 32194 raft_consensus.cc:2749] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.579742 32061 tablet_bootstrap.cc:492] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.584182 32061 ts_tablet_manager.cc:1403] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.001s
I20260709 13:11:03.586752 32061 raft_consensus.cc:359] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:03.587376 32061 raft_consensus.cc:740] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.592494 32061 consensus_queue.cc:260] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:03.595785 32061 ts_tablet_manager.cc:1434] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:11:03.596536 32061 tablet_bootstrap.cc:492] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.607182 32194 raft_consensus.cc:493] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.607892 32194 raft_consensus.cc:515] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.611284 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077ed73927a64748baae88b0b8dc4a13" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "001cac731610478a900ad62e830aaff6" is_pre_election: true
I20260709 13:11:03.613483 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077ed73927a64748baae88b0b8dc4a13" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:03.616621 32055 leader_election.cc:304] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 001cac731610478a900ad62e830aaff6, 07b814fcedea4142ac09385b680cae3a
I20260709 13:11:03.617655 32174 raft_consensus.cc:2749] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:03.620153 32194 leader_election.cc:290] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:03.626544 32061 tablet_bootstrap.cc:492] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.631742 32061 tablet_bootstrap.cc:492] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.632356 32061 ts_tablet_manager.cc:1403] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.002s
I20260709 13:11:03.634791 32061 raft_consensus.cc:359] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:03.635432 32061 raft_consensus.cc:740] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.640746 32061 consensus_queue.cc:260] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:03.642697 32061 ts_tablet_manager.cc:1434] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.002s
I20260709 13:11:03.643466 32061 tablet_bootstrap.cc:492] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.664398 32061 tablet_bootstrap.cc:492] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.665297 32061 tablet_bootstrap.cc:492] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.666028 32061 ts_tablet_manager.cc:1403] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.003s
I20260709 13:11:03.668449 32061 raft_consensus.cc:359] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.669126 32061 raft_consensus.cc:740] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.669899 32061 consensus_queue.cc:260] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.672106 32061 ts_tablet_manager.cc:1434] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:03.672871 32061 tablet_bootstrap.cc:492] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.694123 32061 tablet_bootstrap.cc:492] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.694967 32061 tablet_bootstrap.cc:492] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.695559 32061 ts_tablet_manager.cc:1403] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.007s
I20260709 13:11:03.697933 32061 raft_consensus.cc:359] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:03.698556 32061 raft_consensus.cc:740] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.699256 32061 consensus_queue.cc:260] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:03.701332 32061 ts_tablet_manager.cc:1434] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:11:03.702066 32061 tablet_bootstrap.cc:492] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.722712 32061 tablet_bootstrap.cc:492] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.723676 32061 tablet_bootstrap.cc:492] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.724309 32061 ts_tablet_manager.cc:1403] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 13:11:03.726824 32061 raft_consensus.cc:359] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.727505 32061 raft_consensus.cc:740] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.728394 32061 consensus_queue.cc:260] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:03.730629 32061 ts_tablet_manager.cc:1434] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:11:03.731413 32061 tablet_bootstrap.cc:492] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.733445 32194 raft_consensus.cc:493] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.733971 32194 raft_consensus.cc:515] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:03.735991 32194 leader_election.cc:290] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:03.737843 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:03.739073 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12592f1e5624454a04480b80ff628e2" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:03.740513 32056 leader_election.cc:304] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:11:03.741644 32194 raft_consensus.cc:2749] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:03.764408 32061 tablet_bootstrap.cc:492] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.765285 32061 tablet_bootstrap.cc:492] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.765909 32061 ts_tablet_manager.cc:1403] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.004s
I20260709 13:11:03.768390 32061 raft_consensus.cc:359] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:03.769044 32061 raft_consensus.cc:740] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.770018 32061 consensus_queue.cc:260] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:03.772449 32061 ts_tablet_manager.cc:1434] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:11:03.773269 32061 tablet_bootstrap.cc:492] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.774509 32214 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:36133 every 8 connection(s)
I20260709 13:11:03.774282 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:36133
I20260709 13:11:03.797494 32061 tablet_bootstrap.cc:492] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.798588 32061 tablet_bootstrap.cc:492] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.799325 32061 ts_tablet_manager.cc:1403] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.000s
I20260709 13:11:03.802369 32061 raft_consensus.cc:359] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:03.803471 32061 raft_consensus.cc:740] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.804243 32061 consensus_queue.cc:260] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:03.806633 32061 ts_tablet_manager.cc:1434] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:11:03.807519 32061 tablet_bootstrap.cc:492] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.864015 32061 tablet_bootstrap.cc:492] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.865265 32061 tablet_bootstrap.cc:492] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.866158 32061 ts_tablet_manager.cc:1403] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.059s	user 0.041s	sys 0.012s
I20260709 13:11:03.889976 32061 raft_consensus.cc:359] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:03.890501 32061 raft_consensus.cc:740] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.891112 32061 consensus_queue.cc:260] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:03.896353 32149 raft_consensus.cc:3060] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:03.901449 32061 ts_tablet_manager.cc:1434] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:11:03.902319 32061 tablet_bootstrap.cc:492] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:03.950193 32061 tablet_bootstrap.cc:492] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:03.953015 32061 tablet_bootstrap.cc:492] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:03.954906 32061 ts_tablet_manager.cc:1403] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.002s
I20260709 13:11:03.960142 32061 raft_consensus.cc:359] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:03.962921 32061 raft_consensus.cc:740] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.963987 32061 consensus_queue.cc:260] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:03.966236 32061 ts_tablet_manager.cc:1434] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.001s
I20260709 13:11:03.966848 32061 tablet_bootstrap.cc:492] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.014569 32219 heartbeater.cc:344] Connected to a master server at 127.21.31.126:37055
I20260709 13:11:04.015017 32219 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:04.015899 32219 heartbeater.cc:507] Master 127.21.31.126:37055 requested a full tablet report, sending...
I20260709 13:11:04.030849 32061 tablet_bootstrap.cc:492] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.031898 32061 tablet_bootstrap.cc:492] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.032609 32061 ts_tablet_manager.cc:1403] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.007s
I20260709 13:11:04.035164 32061 raft_consensus.cc:359] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.035900 32061 raft_consensus.cc:740] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.036742 32061 consensus_queue.cc:260] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.039145 32061 ts_tablet_manager.cc:1434] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:04.039980 32061 tablet_bootstrap.cc:492] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.061619 32061 tablet_bootstrap.cc:492] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.062175 31156 ts_manager.cc:194] Re-registered known tserver with Master: 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:04.062932 32061 tablet_bootstrap.cc:492] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.063717 32061 ts_tablet_manager.cc:1403] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.004s
I20260709 13:11:04.066430 32061 raft_consensus.cc:359] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.068619 32146 raft_consensus.cc:3060] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.067734 32061 raft_consensus.cc:740] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.069811 32061 consensus_queue.cc:260] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.072125 32061 ts_tablet_manager.cc:1434] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:11:04.072922 32061 tablet_bootstrap.cc:492] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.118594 32147 raft_consensus.cc:3060] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.128204 32146 raft_consensus.cc:3060] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.142460 32061 tablet_bootstrap.cc:492] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.143366 32061 tablet_bootstrap.cc:492] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.143961 32061 ts_tablet_manager.cc:1403] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.010s
I20260709 13:11:04.149785 32061 raft_consensus.cc:359] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.150424 32061 raft_consensus.cc:740] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.156220 32061 consensus_queue.cc:260] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.163129 32061 ts_tablet_manager.cc:1434] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 13:11:04.163908 32061 tablet_bootstrap.cc:492] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.227182 32061 tablet_bootstrap.cc:492] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.227914 32061 tablet_bootstrap.cc:492] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.228561 32061 ts_tablet_manager.cc:1403] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.015s
I20260709 13:11:04.230736 32061 raft_consensus.cc:359] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.231226 32061 raft_consensus.cc:740] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.231805 32061 consensus_queue.cc:260] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.232054 31156 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55516
I20260709 13:11:04.233866 32061 ts_tablet_manager.cc:1434] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:11:04.234589 32061 tablet_bootstrap.cc:492] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.238287 32219 heartbeater.cc:499] Master 127.21.31.126:37055 was elected leader, sending a full tablet report...
I20260709 13:11:04.286597 32061 tablet_bootstrap.cc:492] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.287549 32061 tablet_bootstrap.cc:492] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.288188 32061 ts_tablet_manager.cc:1403] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.054s	user 0.005s	sys 0.011s
I20260709 13:11:04.290745 32061 raft_consensus.cc:359] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.291429 32061 raft_consensus.cc:740] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.292225 32061 consensus_queue.cc:260] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.293270 32194 raft_consensus.cc:493] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.293831 32194 raft_consensus.cc:515] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.294662 32061 ts_tablet_manager.cc:1434] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:11:04.295509 32061 tablet_bootstrap.cc:492] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.295887 32194 leader_election.cc:290] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:04.297458 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81f5f6441044395977dbda8239d7289" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "07b814fcedea4142ac09385b680cae3a" is_pre_election: true
I20260709 13:11:04.301669 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81f5f6441044395977dbda8239d7289" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "51b0533abe064293ac5047de0ac9bf25" is_pre_election: true
I20260709 13:11:04.306828 32056 leader_election.cc:304] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75b6f0017ce84a4c9822ffd27e053405; no voters: 07b814fcedea4142ac09385b680cae3a, 51b0533abe064293ac5047de0ac9bf25
I20260709 13:11:04.307746 32174 raft_consensus.cc:2749] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:04.332041 32061 tablet_bootstrap.cc:492] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.333024 32061 tablet_bootstrap.cc:492] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.333721 32061 ts_tablet_manager.cc:1403] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.024s
I20260709 13:11:04.336360 32061 raft_consensus.cc:359] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.337033 32061 raft_consensus.cc:740] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.337838 32061 consensus_queue.cc:260] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.340080 32061 ts_tablet_manager.cc:1434] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:04.340842 32061 tablet_bootstrap.cc:492] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.363543 32061 tablet_bootstrap.cc:492] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.364450 32061 tablet_bootstrap.cc:492] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.365093 32061 ts_tablet_manager.cc:1403] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 13:11:04.370476 32061 raft_consensus.cc:359] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.371268 32061 raft_consensus.cc:740] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.371995 32061 consensus_queue.cc:260] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.375756 32061 ts_tablet_manager.cc:1434] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:11:04.377553 32061 tablet_bootstrap.cc:492] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.418360 32061 tablet_bootstrap.cc:492] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.419297 32061 tablet_bootstrap.cc:492] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.419904 32061 ts_tablet_manager.cc:1403] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.004s
I20260709 13:11:04.422093 32061 raft_consensus.cc:359] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.422710 32061 raft_consensus.cc:740] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.423300 32061 consensus_queue.cc:260] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.425303 32061 ts_tablet_manager.cc:1434] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:04.426376 32061 tablet_bootstrap.cc:492] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.447345 32061 tablet_bootstrap.cc:492] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.450054 32061 tablet_bootstrap.cc:492] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.450659 32061 ts_tablet_manager.cc:1403] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.015s
I20260709 13:11:04.452853 32061 raft_consensus.cc:359] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.453372 32061 raft_consensus.cc:740] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.453941 32061 consensus_queue.cc:260] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.457599 32061 ts_tablet_manager.cc:1434] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:11:04.459072 32061 tablet_bootstrap.cc:492] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.484126 32061 tablet_bootstrap.cc:492] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.484993 32061 tablet_bootstrap.cc:492] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.485591 32061 ts_tablet_manager.cc:1403] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.003s
I20260709 13:11:04.488225 32061 raft_consensus.cc:359] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.488718 32061 raft_consensus.cc:740] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.489357 32061 consensus_queue.cc:260] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.491492 32061 ts_tablet_manager.cc:1434] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:04.492197 32061 tablet_bootstrap.cc:492] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.538228 32061 tablet_bootstrap.cc:492] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.539067 32061 tablet_bootstrap.cc:492] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.539675 32061 ts_tablet_manager.cc:1403] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.000s
I20260709 13:11:04.542040 32061 raft_consensus.cc:359] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.542539 32061 raft_consensus.cc:740] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.543167 32061 consensus_queue.cc:260] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.547279 32061 ts_tablet_manager.cc:1434] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:11:04.547964 32061 tablet_bootstrap.cc:492] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.573522 32061 tablet_bootstrap.cc:492] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.574530 32061 tablet_bootstrap.cc:492] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.575191 32061 ts_tablet_manager.cc:1403] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.002s
I20260709 13:11:04.577729 32061 raft_consensus.cc:359] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.578425 32061 raft_consensus.cc:740] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.579136 32061 consensus_queue.cc:260] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.581266 32061 ts_tablet_manager.cc:1434] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:04.584911 32061 tablet_bootstrap.cc:492] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.606048 32061 tablet_bootstrap.cc:492] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.607002 32061 tablet_bootstrap.cc:492] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.607743 32061 ts_tablet_manager.cc:1403] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.000s
I20260709 13:11:04.612924 32061 raft_consensus.cc:359] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.613639 32061 raft_consensus.cc:740] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.615549 32061 consensus_queue.cc:260] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.618854 32061 ts_tablet_manager.cc:1434] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:11:04.619596 32061 tablet_bootstrap.cc:492] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.644212 32061 tablet_bootstrap.cc:492] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.645615 32061 tablet_bootstrap.cc:492] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.646939 32061 ts_tablet_manager.cc:1403] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.006s
I20260709 13:11:04.649335 32061 raft_consensus.cc:359] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.649968 32061 raft_consensus.cc:740] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.650568 32061 consensus_queue.cc:260] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.653079 32061 ts_tablet_manager.cc:1434] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:11:04.653808 32061 tablet_bootstrap.cc:492] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.668910 32061 tablet_bootstrap.cc:492] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.670380 32061 tablet_bootstrap.cc:492] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.671190 32061 ts_tablet_manager.cc:1403] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:11:04.674608 32061 raft_consensus.cc:359] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.675612 32061 raft_consensus.cc:740] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.676466 32061 consensus_queue.cc:260] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.679529 32061 ts_tablet_manager.cc:1434] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:11:04.680632 32061 tablet_bootstrap.cc:492] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.704362 32061 tablet_bootstrap.cc:492] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.705307 32061 tablet_bootstrap.cc:492] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.705924 32061 ts_tablet_manager.cc:1403] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.002s
I20260709 13:11:04.708716 32061 raft_consensus.cc:359] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.709311 32061 raft_consensus.cc:740] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.710078 32061 consensus_queue.cc:260] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.712172 32061 ts_tablet_manager.cc:1434] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:04.714607 32061 tablet_bootstrap.cc:492] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.769368 32061 tablet_bootstrap.cc:492] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.770334 32061 tablet_bootstrap.cc:492] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.770961 32061 ts_tablet_manager.cc:1403] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.057s	user 0.024s	sys 0.010s
I20260709 13:11:04.773574 32061 raft_consensus.cc:359] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.774266 32061 raft_consensus.cc:740] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.774923 32061 consensus_queue.cc:260] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.776292 32146 raft_consensus.cc:3060] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.777087 32061 ts_tablet_manager.cc:1434] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:04.778559 32061 tablet_bootstrap.cc:492] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.809490 32061 tablet_bootstrap.cc:492] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.810443 32061 tablet_bootstrap.cc:492] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.811077 32061 ts_tablet_manager.cc:1403] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.013s
I20260709 13:11:04.813544 32061 raft_consensus.cc:359] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.814211 32061 raft_consensus.cc:740] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.814867 32061 consensus_queue.cc:260] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:04.817091 32061 ts_tablet_manager.cc:1434] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:04.817862 32061 tablet_bootstrap.cc:492] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.839166 32061 tablet_bootstrap.cc:492] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.840092 32061 tablet_bootstrap.cc:492] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.840709 32061 ts_tablet_manager.cc:1403] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 13:11:04.843645 32061 raft_consensus.cc:359] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.844348 32061 raft_consensus.cc:740] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.845005 32061 consensus_queue.cc:260] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.847096 32061 ts_tablet_manager.cc:1434] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.003s
I20260709 13:11:04.847857 32061 tablet_bootstrap.cc:492] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.850971 32146 raft_consensus.cc:3060] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:04.869232 32061 tablet_bootstrap.cc:492] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.870198 32061 tablet_bootstrap.cc:492] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.870857 32061 ts_tablet_manager.cc:1403] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 13:11:04.873288 32061 raft_consensus.cc:359] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.873966 32061 raft_consensus.cc:740] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.874795 32061 consensus_queue.cc:260] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.876320 32147 raft_consensus.cc:3060] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:04.876983 32061 ts_tablet_manager.cc:1434] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:11:04.877897 32061 tablet_bootstrap.cc:492] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.916311 32061 tablet_bootstrap.cc:492] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.917212 32061 tablet_bootstrap.cc:492] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.917833 32061 ts_tablet_manager.cc:1403] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.040s	user 0.023s	sys 0.016s
I20260709 13:11:04.920523 32061 raft_consensus.cc:359] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.921211 32061 raft_consensus.cc:740] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.921870 32061 consensus_queue.cc:260] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:04.924867 32061 ts_tablet_manager.cc:1434] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:04.925652 32061 tablet_bootstrap.cc:492] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap starting.
I20260709 13:11:04.942938 32061 tablet_bootstrap.cc:492] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:04.943814 32061 tablet_bootstrap.cc:492] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: Bootstrap complete.
I20260709 13:11:04.944424 32061 ts_tablet_manager.cc:1403] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.003s
I20260709 13:11:04.946974 32061 raft_consensus.cc:359] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.947587 32061 raft_consensus.cc:740] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.948303 32061 consensus_queue.cc:260] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:04.950465 32061 ts_tablet_manager.cc:1434] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:04.986217 32147 raft_consensus.cc:3060] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:05.188627 32147 raft_consensus.cc:3060] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:10.128472 31303 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.68:44895, user_credentials={real_user=slave}} blocked reactor thread for 45646.6us
I20260709 13:11:13.868363 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:13.920843 31410 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71"
dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:34306
I20260709 13:11:13.922003 31410 raft_consensus.cc:606] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 2 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:13.940336 31337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:13.940831 31337 raft_consensus.cc:606] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:13.944506 31337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:13.945001 31337 raft_consensus.cc:606] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:13.947749 31337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:13.948247 31337 raft_consensus.cc:606] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:13.950889 31337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:13.951365 31337 raft_consensus.cc:606] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:13.953896 31337 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f564510ff30e42ec890531c82405946c"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "001cac731610478a900ad62e830aaff6"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:13.954423 31337 raft_consensus.cc:606] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 001cac731610478a900ad62e830aaff6
I20260709 13:11:13.955520 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:11:13.957583 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:13.970300 32283 raft_consensus.cc:993] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:13.971019 32289 raft_consensus.cc:1081] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:13.972945 32146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:13.973524 32146 raft_consensus.cc:493] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:13.973856 32146 raft_consensus.cc:3060] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.980611 32146 raft_consensus.cc:515] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:13.982430 32146 leader_election.cc:290] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:13.983479 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:13.983479 31481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9645a5374ed42a79a83e6af2d395bcc" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:13.984270 31337 raft_consensus.cc:3055] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:13.984438 31481 raft_consensus.cc:3060] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.984639 31337 raft_consensus.cc:740] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:13.985626 31337 consensus_queue.cc:260] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:13.987105 31337 raft_consensus.cc:3060] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:13.991475 31481 raft_consensus.cc:2468] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:13.992888 32058 leader_election.cc:304] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:13.994513 31337 raft_consensus.cc:2468] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:13.994606 32293 raft_consensus.cc:2804] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:13.996021 32293 raft_consensus.cc:697] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:13.997076 32293 consensus_queue.cc:237] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:14.007630 31160 catalog_manager.cc:5697] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.080313 32283 raft_consensus.cc:993] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:14.080806 32289 raft_consensus.cc:1081] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:14.082298 32146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:14.082927 32146 raft_consensus.cc:493] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:14.083284 32146 raft_consensus.cc:3060] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.090413 32146 raft_consensus.cc:515] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:14.093044 31483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:14.093705 31483 raft_consensus.cc:3060] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.098102 32146 leader_election.cc:290] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:14.099509 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23babc199fb49fe9e6423834c97d1f7" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:14.100209 31337 raft_consensus.cc:3055] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:14.100541 31337 raft_consensus.cc:740] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:14.101204 31483 raft_consensus.cc:2468] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:14.101356 31337 consensus_queue.cc:260] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:14.102310 32058 leader_election.cc:304] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:14.102747 31337 raft_consensus.cc:3060] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.103006 32293 raft_consensus.cc:2804] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:14.103458 32293 raft_consensus.cc:697] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:14.104315 32293 consensus_queue.cc:237] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:14.109258 31337 raft_consensus.cc:2468] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:14.118211 31160 catalog_manager.cc:5697] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.335836 32289 raft_consensus.cc:993] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 001cac731610478a900ad62e830aaff6 to start an election
I20260709 13:11:14.336292 32283 raft_consensus.cc:1081] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 001cac731610478a900ad62e830aaff6 to start an election
I20260709 13:11:14.337603 31481 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f564510ff30e42ec890531c82405946c"
dest_uuid: "001cac731610478a900ad62e830aaff6"
 from {username='slave'} at 127.0.0.1:35560
I20260709 13:11:14.338372 31481 raft_consensus.cc:493] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:14.338670 31481 raft_consensus.cc:3060] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.342918 31481 raft_consensus.cc:515] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:11:14.344944 31481 leader_election.cc:290] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:14.346060 31337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f564510ff30e42ec890531c82405946c" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:14.346815 31337 raft_consensus.cc:3055] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:14.347441 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f564510ff30e42ec890531c82405946c" candidate_uuid: "001cac731610478a900ad62e830aaff6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:14.348114 31410 raft_consensus.cc:3060] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.347234 31337 raft_consensus.cc:740] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:14.349298 31337 consensus_queue.cc:260] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:11:14.350718 31337 raft_consensus.cc:3060] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.357468 31410 raft_consensus.cc:2468] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 2.
I20260709 13:11:14.358731 31449 leader_election.cc:304] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 51b0533abe064293ac5047de0ac9bf25; no voters: 
I20260709 13:11:14.359683 32281 raft_consensus.cc:2804] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:14.360143 32281 raft_consensus.cc:697] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 2 LEADER]: Becoming Leader. State: Replica: 001cac731610478a900ad62e830aaff6, State: Running, Role: LEADER
I20260709 13:11:14.361595 32283 raft_consensus.cc:993] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:14.361083 32281 consensus_queue.cc:237] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } }
I20260709 13:11:14.362211 32288 raft_consensus.cc:1081] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:14.362896 31337 raft_consensus.cc:2468] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001cac731610478a900ad62e830aaff6 in term 2.
I20260709 13:11:14.364424 32147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:14.365078 32147 raft_consensus.cc:493] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:14.365453 32147 raft_consensus.cc:3060] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.379213 32147 raft_consensus.cc:515] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:14.379549 31481 raft_consensus.cc:1275] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.381179 32293 consensus_queue.cc:1048] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.381763 31160 catalog_manager.cc:5697] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 001cac731610478a900ad62e830aaff6 (127.21.31.68). New cstate: current_term: 2 leader_uuid: "001cac731610478a900ad62e830aaff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:14.395071 32283 raft_consensus.cc:993] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:14.395551 32289 raft_consensus.cc:1081] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:14.398909 32149 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:14.399453 32149 raft_consensus.cc:493] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:14.399801 32149 raft_consensus.cc:3060] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.399924 31335 raft_consensus.cc:1275] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.401510 32293 consensus_queue.cc:1048] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.402931 32147 leader_election.cc:290] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:14.403784 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:14.404379 31335 raft_consensus.cc:3055] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:14.404700 31335 raft_consensus.cc:740] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:14.405527 31335 consensus_queue.cc:260] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:14.408071 31335 raft_consensus.cc:3060] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.410877 32149 raft_consensus.cc:515] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:14.406440 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c415b45c0cf0456bae70f23527c3488e" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:14.411916 31410 raft_consensus.cc:3060] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.422892 31481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:14.424018 31481 raft_consensus.cc:3060] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.443619 31335 raft_consensus.cc:2468] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:14.444033 31410 raft_consensus.cc:2468] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:14.446254 32055 leader_election.cc:304] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:14.447283 32293 raft_consensus.cc:2804] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:14.447798 32293 raft_consensus.cc:697] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:14.448024 31411 raft_consensus.cc:1240] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Rejecting Update request from peer 07b814fcedea4142ac09385b680cae3a for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:14.448776 32293 consensus_queue.cc:237] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:14.457587 32149 leader_election.cc:290] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:14.474972 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d913d7bf7d154005a431dc28e2bd91f6" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:14.476043 31333 raft_consensus.cc:3055] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:14.476353 31333 raft_consensus.cc:740] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:14.477293 31333 consensus_queue.cc:260] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:14.478626 31333 raft_consensus.cc:3060] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:14.486043 31333 raft_consensus.cc:2468] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:14.487968 32055 leader_election.cc:304] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:14.488911 32305 raft_consensus.cc:2804] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:14.489485 32305 raft_consensus.cc:697] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:14.490621 32305 consensus_queue.cc:237] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:14.487986 31481 raft_consensus.cc:2468] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:14.509711 32286 raft_consensus.cc:993] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:14.510232 32286 raft_consensus.cc:1081] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 2 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:14.514072 32149 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37394
I20260709 13:11:14.514576 32149 raft_consensus.cc:493] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:14.514854 32149 raft_consensus.cc:3060] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:14.516649 31160 catalog_manager.cc:5697] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.518030 31160 catalog_manager.cc:5697] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.520833 32149 raft_consensus.cc:515] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:14.523420 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:14.523628 32149 leader_election.cc:290] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 3 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:14.524046 31335 raft_consensus.cc:3060] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:14.524917 31411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb349a8fff1b4d6f8dda0834dcec5b71" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:14.525501 31411 raft_consensus.cc:3055] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:14.525759 31411 raft_consensus.cc:740] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:11:14.526494 31411 consensus_queue.cc:260] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:14.527684 31411 raft_consensus.cc:3060] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:14.530335 31335 raft_consensus.cc:2468] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 3.
I20260709 13:11:14.531404 32055 leader_election.cc:304] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:14.532162 32305 raft_consensus.cc:2804] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:14.532651 32305 raft_consensus.cc:697] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 3 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:14.533274 32305 consensus_queue.cc:237] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:14.534112 31411 raft_consensus.cc:2468] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 3.
I20260709 13:11:14.542945 31160 catalog_manager.cc:5697] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 2 to 3, leader changed from 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 3 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.647305 31481 raft_consensus.cc:1275] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.649271 32305 consensus_queue.cc:1048] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.649919 31335 raft_consensus.cc:1275] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.651368 32305 consensus_queue.cc:1048] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.802695 31411 raft_consensus.cc:1275] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 001cac731610478a900ad62e830aaff6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.803718 32281 consensus_queue.cc:1048] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.812601 31335 raft_consensus.cc:1275] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 001cac731610478a900ad62e830aaff6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.814489 32302 consensus_queue.cc:1048] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:14.874055 31335 raft_consensus.cc:1275] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.875545 32294 consensus_queue.cc:1048] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:14.888784 31411 raft_consensus.cc:1275] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.890934 32305 consensus_queue.cc:1048] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.894101 31480 raft_consensus.cc:1275] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.897616 32293 consensus_queue.cc:1048] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.920621 31335 raft_consensus.cc:1275] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.924019 32293 consensus_queue.cc:1048] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.943781 31405 raft_consensus.cc:1275] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:14.945025 32293 consensus_queue.cc:1048] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:14.958875 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:14.986013 31335 raft_consensus.cc:1275] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 3 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:14.990100 32294 consensus_queue.cc:1048] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:15.029759 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:15.070451 31333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:15.071105 31333 raft_consensus.cc:606] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:15.073572 31333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:15.073968 31333 raft_consensus.cc:606] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:15.076589 31333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:15.077095 31333 raft_consensus.cc:606] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:15.079636 31405 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b"
dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:34306
I20260709 13:11:15.080169 31405 raft_consensus.cc:606] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:15.082921 31333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:15.083526 31333 raft_consensus.cc:606] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:15.085974 31333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f81f5f6441044395977dbda8239d7289"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:15.086460 31333 raft_consensus.cc:606] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:15.087551 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:11:15.088649 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:15.236126 32286 raft_consensus.cc:993] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.239240 32308 raft_consensus.cc:1081] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.243230 32151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37394
I20260709 13:11:15.243965 32151 raft_consensus.cc:493] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:15.244414 32151 raft_consensus.cc:3060] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.251734 32151 raft_consensus.cc:515] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:15.253088 32151 leader_election.cc:290] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:11:15.254045 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:15.254534 31480 raft_consensus.cc:3060] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.254534 31405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c90bfbd1ec9d458faadce4b9a625024b" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:15.255311 31405 raft_consensus.cc:3055] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:15.255595 31405 raft_consensus.cc:740] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:11:15.256304 31405 consensus_queue.cc:260] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:15.257437 31405 raft_consensus.cc:3060] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.259891 31480 raft_consensus.cc:2468] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.260998 32058 leader_election.cc:304] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:15.261757 32294 raft_consensus.cc:2804] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:15.262256 32294 raft_consensus.cc:697] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:15.263170 32294 consensus_queue.cc:237] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:15.263942 32327 raft_consensus.cc:993] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.263927 31405 raft_consensus.cc:2468] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.264612 32327 raft_consensus.cc:1081] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.266206 32151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f81f5f6441044395977dbda8239d7289"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:15.266619 32151 raft_consensus.cc:493] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:15.266916 32151 raft_consensus.cc:3060] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.273650 32151 raft_consensus.cc:515] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:15.275629 32151 leader_election.cc:290] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:15.276672 31405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81f5f6441044395977dbda8239d7289" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:15.277338 31405 raft_consensus.cc:3060] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.277392 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f81f5f6441044395977dbda8239d7289" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:15.278551 31333 raft_consensus.cc:3055] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:15.278877 31333 raft_consensus.cc:740] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:15.279364 31156 catalog_manager.cc:5697] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:15.279742 31333 consensus_queue.cc:260] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:15.283529 31333 raft_consensus.cc:3060] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.285183 31405 raft_consensus.cc:2468] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.286849 32056 leader_election.cc:304] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:15.287899 32294 raft_consensus.cc:2804] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:15.288432 32294 raft_consensus.cc:697] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:15.289402 32294 consensus_queue.cc:237] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:15.291100 31333 raft_consensus.cc:2468] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.300263 32288 raft_consensus.cc:993] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.300735 32288 raft_consensus.cc:1081] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.302194 32151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:15.302676 32151 raft_consensus.cc:493] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:15.303016 32151 raft_consensus.cc:3060] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.313226 32151 raft_consensus.cc:515] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:15.315094 32151 leader_election.cc:290] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:15.314626 31156 catalog_manager.cc:5697] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:15.315659 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:15.316201 31480 raft_consensus.cc:3060] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.316846 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663d2c98ef924d139ff1835938fef0df" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:15.317729 31333 raft_consensus.cc:3055] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:15.318104 31333 raft_consensus.cc:740] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:15.319157 31333 consensus_queue.cc:260] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:15.321213 31480 raft_consensus.cc:2468] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.320519 31333 raft_consensus.cc:3060] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.322324 32058 leader_election.cc:304] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:15.323282 32294 raft_consensus.cc:2804] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:15.323941 32294 raft_consensus.cc:697] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:15.324883 32294 consensus_queue.cc:237] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:15.329859 31333 raft_consensus.cc:2468] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.337194 31156 catalog_manager.cc:5697] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:15.418866 32327 raft_consensus.cc:993] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.419348 32283 raft_consensus.cc:1081] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.420640 32151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:15.421068 32151 raft_consensus.cc:493] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:15.421380 32151 raft_consensus.cc:3060] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.427822 32151 raft_consensus.cc:515] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:15.430409 32151 leader_election.cc:290] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:15.430418 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:15.431397 31405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e69c3326d8874749a268ae8dfdca3b3c" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:15.432080 31405 raft_consensus.cc:3060] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.433821 31333 raft_consensus.cc:3055] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:15.434188 31333 raft_consensus.cc:740] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:15.434998 31333 consensus_queue.cc:260] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:15.436035 31333 raft_consensus.cc:3060] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.440631 31405 raft_consensus.cc:2468] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.441901 32056 leader_election.cc:304] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:15.442792 32306 raft_consensus.cc:2804] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:15.443245 32306 raft_consensus.cc:697] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:15.443833 31333 raft_consensus.cc:2468] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.444064 32306 consensus_queue.cc:237] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:15.456471 31156 catalog_manager.cc:5697] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:15.495368 32327 raft_consensus.cc:993] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.495844 32283 raft_consensus.cc:1081] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.497428 32151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:15.497994 32151 raft_consensus.cc:493] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:15.498302 32151 raft_consensus.cc:3060] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.505295 32151 raft_consensus.cc:515] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:15.508239 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:15.508790 31480 raft_consensus.cc:3060] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.509007 32151 leader_election.cc:290] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:15.509585 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ecee4c21f34ad0a27f7f36add491e9" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:15.510210 31335 raft_consensus.cc:3055] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:15.510494 31335 raft_consensus.cc:740] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:15.511178 31335 consensus_queue.cc:260] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:15.512221 31335 raft_consensus.cc:3060] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.518325 31480 raft_consensus.cc:2468] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.519644 32058 leader_election.cc:304] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:15.520499 32306 raft_consensus.cc:2804] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:15.518325 31335 raft_consensus.cc:2468] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.520952 32306 raft_consensus.cc:697] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:15.522353 32306 consensus_queue.cc:237] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:15.525435 32327 raft_consensus.cc:993] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.525907 32327 raft_consensus.cc:1081] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:15.528131 32146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:15.528723 32146 raft_consensus.cc:493] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:15.529124 32146 raft_consensus.cc:3060] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.534446 32146 raft_consensus.cc:515] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:15.535136 31156 catalog_manager.cc:5697] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:15.535871 32146 leader_election.cc:290] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895), 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075)
I20260709 13:11:15.536551 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:15.536832 31333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24424b197c343e1a57a4f490c67aaf3" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:15.537075 31480 raft_consensus.cc:3060] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.537362 31333 raft_consensus.cc:3055] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:15.537660 31333 raft_consensus.cc:740] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:15.538349 31333 consensus_queue.cc:260] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:15.539575 31333 raft_consensus.cc:3060] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:15.541849 31480 raft_consensus.cc:2468] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.542940 32058 leader_election.cc:304] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:15.543752 32306 raft_consensus.cc:2804] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:15.546253 32306 raft_consensus.cc:697] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:15.547088 32306 consensus_queue.cc:237] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } }
I20260709 13:11:15.547832 31333 raft_consensus.cc:2468] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:15.555562 31156 catalog_manager.cc:5697] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:15.742655 31411 raft_consensus.cc:1275] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:15.743933 32306 consensus_queue.cc:1048] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:15.758847 31333 raft_consensus.cc:1275] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:15.760555 31481 raft_consensus.cc:1275] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:15.760370 32306 consensus_queue.cc:1048] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:15.762344 32294 consensus_queue.cc:1048] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:15.770918 31410 raft_consensus.cc:1275] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:15.777992 32305 consensus_queue.cc:1048] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:15.801678 31335 raft_consensus.cc:1275] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:15.803715 32306 consensus_queue.cc:1048] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:15.831357 31451 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:36133, user_credentials={real_user=slave}} blocked reactor thread for 45091.4us
I20260709 13:11:15.857775 31481 raft_consensus.cc:1275] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:15.859200 32306 consensus_queue.cc:1048] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:15.861145 31410 raft_consensus.cc:1275] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:15.864979 32306 consensus_queue.cc:1048] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:15.882918 31333 raft_consensus.cc:1275] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:15.884177 32306 consensus_queue.cc:1048] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.031021 31481 raft_consensus.cc:1275] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.032450 32294 consensus_queue.cc:1048] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.042451 31335 raft_consensus.cc:1275] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.047142 32305 consensus_queue.cc:1048] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:16.049649 31335 raft_consensus.cc:1275] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.052665 32305 consensus_queue.cc:1048] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.070544 31480 raft_consensus.cc:1275] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.073581 32351 consensus_queue.cc:1048] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.089583 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:16.130564 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:16.161289 31405 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271f7357440943008e55795b0d6cbbda"
dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:34306
I20260709 13:11:16.161926 31405 raft_consensus.cc:606] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:16.164520 31333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85"
dest_uuid: "07b814fcedea4142ac09385b680cae3a"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:33310
I20260709 13:11:16.164940 31333 raft_consensus.cc:606] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:16.168013 31410 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949"
dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:34306
I20260709 13:11:16.168690 31410 raft_consensus.cc:606] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:16.171247 31410 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310"
dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
mode: GRACEFUL
new_leader_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:34306
I20260709 13:11:16.171729 31410 raft_consensus.cc:606] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Received request to transfer leadership to TS 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:16.173033 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:11:16.175885 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:16.247330 32286 raft_consensus.cc:993] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:16.247828 32286 raft_consensus.cc:1081] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:16.249469 32149 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "271f7357440943008e55795b0d6cbbda"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37394
I20260709 13:11:16.250181 32149 raft_consensus.cc:493] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:16.250502 32149 raft_consensus.cc:3060] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.258129 32149 raft_consensus.cc:515] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:16.260396 32149 leader_election.cc:290] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:11:16.261504 31480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271f7357440943008e55795b0d6cbbda" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:16.261906 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271f7357440943008e55795b0d6cbbda" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:16.262156 31480 raft_consensus.cc:3060] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.262483 31410 raft_consensus.cc:3055] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:16.262784 31410 raft_consensus.cc:740] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:11:16.263463 31410 consensus_queue.cc:260] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:16.264629 31410 raft_consensus.cc:3060] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.268581 31480 raft_consensus.cc:2468] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:16.269747 32058 leader_election.cc:304] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:16.270442 32351 raft_consensus.cc:2804] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:16.270735 31410 raft_consensus.cc:2468] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:16.271332 32351 raft_consensus.cc:697] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:16.272418 32351 consensus_queue.cc:237] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:16.281522 31157 catalog_manager.cc:5697] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:16.424260 32283 raft_consensus.cc:993] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:16.424702 32283 raft_consensus.cc:1081] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:16.425992 32151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37404
I20260709 13:11:16.426440 32151 raft_consensus.cc:493] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:16.426713 32151 raft_consensus.cc:3060] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.431303 32151 raft_consensus.cc:515] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:16.432559 32151 leader_election.cc:290] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:16.433269 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:16.433506 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c760b7263fd41fea6c0a6dfd9dc5d85" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:16.433919 31335 raft_consensus.cc:3055] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:16.434043 31410 raft_consensus.cc:3060] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.434273 31335 raft_consensus.cc:740] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07b814fcedea4142ac09385b680cae3a, State: Running, Role: LEADER
I20260709 13:11:16.435111 31335 consensus_queue.cc:260] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:16.436314 31335 raft_consensus.cc:3060] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.445472 31335 raft_consensus.cc:2468] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:16.446599 32055 leader_election.cc:304] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:16.446707 31410 raft_consensus.cc:2468] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:16.447392 32306 raft_consensus.cc:2804] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:16.447854 32306 raft_consensus.cc:697] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:16.448709 32306 consensus_queue.cc:237] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:16.458047 32350 raft_consensus.cc:993] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:16.458503 32286 raft_consensus.cc:1081] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:16.460146 32149 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37394
I20260709 13:11:16.460652 32149 raft_consensus.cc:493] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:16.460975 32149 raft_consensus.cc:3060] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.463347 31157 catalog_manager.cc:5697] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 07b814fcedea4142ac09385b680cae3a (127.21.31.66) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:16.467741 32149 raft_consensus.cc:515] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:16.469887 32149 leader_election.cc:290] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 001cac731610478a900ad62e830aaff6 (127.21.31.68:44895)
I20260709 13:11:16.470583 31481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "001cac731610478a900ad62e830aaff6"
I20260709 13:11:16.471184 31481 raft_consensus.cc:3060] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.475660 31481 raft_consensus.cc:2468] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:16.470580 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfee9be6a99740c4ab0759431ad94310" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:16.477054 31410 raft_consensus.cc:3055] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:16.476946 32058 leader_election.cc:304] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001cac731610478a900ad62e830aaff6, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:16.477803 31410 raft_consensus.cc:740] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:11:16.478528 32306 raft_consensus.cc:2804] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:16.479048 32306 raft_consensus.cc:697] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:16.479061 31410 consensus_queue.cc:260] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:16.479857 32306 consensus_queue.cc:237] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:16.480868 31410 raft_consensus.cc:3060] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.487108 31410 raft_consensus.cc:2468] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:16.492684 31157 catalog_manager.cc:5697] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:16.734678 32350 raft_consensus.cc:993] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25: : Instructing follower 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:16.735177 32286 raft_consensus.cc:1081] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Signalling peer 75b6f0017ce84a4c9822ffd27e053405 to start an election
I20260709 13:11:16.736692 32151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949"
dest_uuid: "75b6f0017ce84a4c9822ffd27e053405"
 from {username='slave'} at 127.0.0.1:37394
I20260709 13:11:16.737243 32151 raft_consensus.cc:493] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:16.737654 32151 raft_consensus.cc:3060] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.745921 32151 raft_consensus.cc:515] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:16.748306 31335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07b814fcedea4142ac09385b680cae3a"
I20260709 13:11:16.748415 32151 leader_election.cc:290] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Requested vote from peers 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075), 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117)
I20260709 13:11:16.748890 31335 raft_consensus.cc:3060] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.748991 31410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfcdfdb50a8b463e851c89aa160fd949" candidate_uuid: "75b6f0017ce84a4c9822ffd27e053405" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "51b0533abe064293ac5047de0ac9bf25"
I20260709 13:11:16.749625 31410 raft_consensus.cc:3055] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:16.749866 31410 raft_consensus.cc:740] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51b0533abe064293ac5047de0ac9bf25, State: Running, Role: LEADER
I20260709 13:11:16.750687 31410 consensus_queue.cc:260] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:16.751600 31410 raft_consensus.cc:3060] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.753646 31335 raft_consensus.cc:2468] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:16.754722 32055 leader_election.cc:304] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b814fcedea4142ac09385b680cae3a, 75b6f0017ce84a4c9822ffd27e053405; no voters: 
I20260709 13:11:16.755435 32306 raft_consensus.cc:2804] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:16.755977 32306 raft_consensus.cc:697] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Becoming Leader. State: Replica: 75b6f0017ce84a4c9822ffd27e053405, State: Running, Role: LEADER
I20260709 13:11:16.756727 31410 raft_consensus.cc:2468] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b6f0017ce84a4c9822ffd27e053405 in term 2.
I20260709 13:11:16.756815 32306 consensus_queue.cc:237] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } }
I20260709 13:11:16.768222 31157 catalog_manager.cc:5697] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 reported cstate change: term changed from 1 to 2, leader changed from 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67) to 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75b6f0017ce84a4c9822ffd27e053405" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:16.809279 31410 raft_consensus.cc:1275] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.810724 32306 consensus_queue.cc:1048] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.829288 31481 raft_consensus.cc:1275] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.831563 32306 consensus_queue.cc:1048] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.844156 31410 raft_consensus.cc:1275] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.845840 32351 consensus_queue.cc:1048] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.857489 31335 raft_consensus.cc:1275] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.859073 32305 consensus_queue.cc:1048] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b814fcedea4142ac09385b680cae3a" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.979681 31410 raft_consensus.cc:1275] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.980768 32305 consensus_queue.cc:1048] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.990159 31481 raft_consensus.cc:1275] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Refusing update from remote peer 75b6f0017ce84a4c9822ffd27e053405: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:16.991665 32306 consensus_queue.cc:1048] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:17.176709 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:17.214911 21629 tablet_server.cc:179] TabletServer@127.21.31.65:36133 shutting down...
W20260709 13:11:17.233505 31303 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 318 similar messages]
W20260709 13:11:17.239621 31303 consensus_peers.cc:597] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.248158 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:17.249262 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.250233 21629 raft_consensus.cc:2243] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.250955 21629 raft_consensus.cc:2272] T 24fe066d118d48c5939576417491e9ec P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.254573 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.255525 21629 raft_consensus.cc:2243] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.256104 21629 raft_consensus.cc:2272] T 862d691f6d9e4d9fb185be53b562b8ff P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.258229 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.258860 21629 raft_consensus.cc:2243] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.259608 21629 raft_consensus.cc:2272] T d23babc199fb49fe9e6423834c97d1f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.261758 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.262401 21629 raft_consensus.cc:2243] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.262833 21629 raft_consensus.cc:2272] T f12b34e35ea0429ea3c77b7d89fd748a P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.264814 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.265429 21629 raft_consensus.cc:2243] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.265841 21629 raft_consensus.cc:2272] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 75b6f0017ce84a4c9822ffd27e053405 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.268412 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.269109 21629 raft_consensus.cc:2243] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.269579 21629 raft_consensus.cc:2272] T 9b3eb1a472a546fdb51a2cc295716492 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.270702 31303 consensus_peers.cc:597] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.271883 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.272739 21629 raft_consensus.cc:2243] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.273270 21629 raft_consensus.cc:2272] T fa7546f06a454f3899cb0bf7d0c0d084 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.276247 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.276990 21629 raft_consensus.cc:2243] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.277957 21629 raft_consensus.cc:2272] T bb349a8fff1b4d6f8dda0834dcec5b71 P 75b6f0017ce84a4c9822ffd27e053405 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.279795 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.280253 21629 raft_consensus.cc:2243] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.280555 21629 raft_consensus.cc:2272] T 77fb3ec3412341df88b759485f90f108 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.282382 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.282814 21629 raft_consensus.cc:2243] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.283349 21629 pending_rounds.cc:70] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Trying to abort 1 pending ops.
I20260709 13:11:17.283525 21629 pending_rounds.cc:77] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305636564009504768 op_type: NO_OP noop_request { }
I20260709 13:11:17.283757 21629 raft_consensus.cc:2889] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:17.284075 21629 raft_consensus.cc:2272] T bfcdfdb50a8b463e851c89aa160fd949 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.286540 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.287253 21629 raft_consensus.cc:2243] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.287613 21629 raft_consensus.cc:2272] T 1d10f77e65c14720a8312b1dd44d3199 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.290072 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.290910 21629 raft_consensus.cc:2243] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.291409 21629 raft_consensus.cc:2272] T 92573de449ff4002a92b5ffcecfb8521 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.294196 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.294997 21629 raft_consensus.cc:2243] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.295475 21629 raft_consensus.cc:2272] T dda3de9e2b1643049b35455ababf5d82 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.298192 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.298951 21629 raft_consensus.cc:2243] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.299392 21629 raft_consensus.cc:2272] T 181fa1bbe1e14e098cad0b23561576d2 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.301954 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.302759 21629 raft_consensus.cc:2243] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.303229 21629 raft_consensus.cc:2272] T a6c9185858d64cfca85bfe148ea6bef0 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.305902 21629 tablet_replica.cc:333] stopping tablet replica
W20260709 13:11:17.306520 31451 consensus_peers.cc:597] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.306700 21629 raft_consensus.cc:2243] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.307265 21629 raft_consensus.cc:2272] T 67b810039c1e415192775c9f611cd825 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.310091 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.310775 21629 raft_consensus.cc:2243] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.311668 21629 raft_consensus.cc:2272] T cfee9be6a99740c4ab0759431ad94310 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.314683 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.315375 21629 raft_consensus.cc:2243] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.316247 21629 raft_consensus.cc:2272] T e69c3326d8874749a268ae8dfdca3b3c P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.318728 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.319458 21629 raft_consensus.cc:2243] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.320314 21629 raft_consensus.cc:2272] T c24424b197c343e1a57a4f490c67aaf3 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.322829 21629 tablet_replica.cc:333] stopping tablet replica
W20260709 13:11:17.322925 31377 consensus_peers.cc:597] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.323400 21629 raft_consensus.cc:2243] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.324136 21629 raft_consensus.cc:2272] T 663d2c98ef924d139ff1835938fef0df P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.326532 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.327229 21629 raft_consensus.cc:2243] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.327845 21629 raft_consensus.cc:2272] T e12592f1e5624454a04480b80ff628e2 P 75b6f0017ce84a4c9822ffd27e053405 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.330549 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.331091 21629 raft_consensus.cc:2243] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.331550 21629 raft_consensus.cc:2272] T 0ad34ee917be4455bbe71da15c523cee P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.334120 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.334779 21629 raft_consensus.cc:2243] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.335217 21629 raft_consensus.cc:2272] T b41a57045456429c857b8fb94b54c356 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.335670 31451 consensus_peers.cc:597] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.337718 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.338589 21629 raft_consensus.cc:2243] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.339043 21629 raft_consensus.cc:2272] T d905b9b08a224fe09df30744558819e3 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.340332 31303 consensus_peers.cc:597] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.341634 21629 tablet_replica.cc:333] stopping tablet replica
W20260709 13:11:17.341900 31451 consensus_peers.cc:597] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.342262 21629 raft_consensus.cc:2243] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.342984 21629 raft_consensus.cc:2272] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.345194 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.345724 21629 raft_consensus.cc:2243] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.346335 21629 raft_consensus.cc:2272] T a2ecee4c21f34ad0a27f7f36add491e9 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.349112 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.349829 21629 raft_consensus.cc:2243] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:11:17.350183 31451 consensus_peers.cc:597] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.350682 21629 raft_consensus.cc:2272] T d913d7bf7d154005a431dc28e2bd91f6 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.353304 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.354429 21629 raft_consensus.cc:2243] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:11:17.355255 31451 consensus_peers.cc:597] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.355311 21629 raft_consensus.cc:2272] T d9645a5374ed42a79a83e6af2d395bcc P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.357723 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.358404 21629 raft_consensus.cc:2243] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.359375 21629 raft_consensus.cc:2272] T 271f7357440943008e55795b0d6cbbda P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.361918 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.362465 21629 raft_consensus.cc:2243] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.362915 21629 raft_consensus.cc:2272] T 229af634ffc1437fb9dea4a95652ab5b P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.364926 31303 consensus_peers.cc:597] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.365638 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.366420 21629 raft_consensus.cc:2243] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:17.366576 31451 consensus_peers.cc:597] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.366897 21629 raft_consensus.cc:2272] T 8e1eac6b3d8a47b59679e4aee4293326 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.370836 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.371410 21629 raft_consensus.cc:2243] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.371865 21629 raft_consensus.cc:2272] T 9725e6248e7b4e67b23a3b99967003bf P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.375093 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.375759 21629 raft_consensus.cc:2243] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.376269 21629 raft_consensus.cc:2272] T 4b06bd7c5b2a4586b9caf425bc23720a P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.376403 31303 consensus_peers.cc:597] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.378505 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.379128 21629 raft_consensus.cc:2243] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.380053 21629 raft_consensus.cc:2272] T c90bfbd1ec9d458faadce4b9a625024b P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.382097 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.382608 21629 raft_consensus.cc:2243] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.383293 21629 raft_consensus.cc:2272] T f81f5f6441044395977dbda8239d7289 P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.385797 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.386498 21629 raft_consensus.cc:2243] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.386943 21629 raft_consensus.cc:2272] T 077ed73927a64748baae88b0b8dc4a13 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.389120 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.389770 21629 raft_consensus.cc:2243] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.390213 21629 raft_consensus.cc:2272] T 29510c165a1f48b1949c6ee31e136064 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.393090 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.393761 21629 raft_consensus.cc:2243] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.394309 21629 raft_consensus.cc:2272] T 5db13a06e22e411ba4c29d105c7e93f7 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.396585 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.397197 21629 raft_consensus.cc:2243] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.397745 21629 raft_consensus.cc:2272] T 914efa83b45e44dbbabeca35e75d0f3e P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.400362 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.401070 21629 raft_consensus.cc:2243] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:17.401068 31303 consensus_peers.cc:597] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.401600 21629 raft_consensus.cc:2272] T f70d36cace8741d7b866e776f6cc03c1 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.403942 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.404484 21629 raft_consensus.cc:2243] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.405002 21629 raft_consensus.cc:2272] T 1b5b2bc298df4947b958fefdc9c2846e P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.407115 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.407548 21629 raft_consensus.cc:2243] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.408243 21629 raft_consensus.cc:2272] T c415b45c0cf0456bae70f23527c3488e P 75b6f0017ce84a4c9822ffd27e053405 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.410022 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.410481 21629 raft_consensus.cc:2243] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.410854 21629 raft_consensus.cc:2272] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.412652 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.413103 21629 raft_consensus.cc:2243] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.413544 21629 raft_consensus.cc:2272] T f9baba29a028498288a27243e555fb27 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.415671 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:17.416108 21629 raft_consensus.cc:2243] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.416501 21629 raft_consensus.cc:2272] T 2fb1e844232d4ed0b8752e51e31204b8 P 75b6f0017ce84a4c9822ffd27e053405 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.433769 31451 consensus_peers.cc:597] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.461463 31451 consensus_peers.cc:597] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.465947 31377 consensus_peers.cc:597] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.485731 31377 consensus_peers.cc:597] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.492645 21629 tablet_server.cc:196] TabletServer@127.21.31.65:36133 shutdown complete.
W20260709 13:11:17.530812 31451 consensus_peers.cc:597] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.567400 31451 consensus_peers.cc:597] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.570863 21629 tablet_server.cc:179] TabletServer@127.21.31.66:0 shutting down...
W20260709 13:11:17.579429 31448 consensus_peers.cc:597] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.582744 31377 consensus_peers.cc:597] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.591825 31448 consensus_peers.cc:597] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.593258 31377 consensus_peers.cc:597] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.608475 31374 consensus_peers.cc:597] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.613397 31377 consensus_peers.cc:597] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.618029 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:11:17.618708 31451 consensus_peers.cc:597] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.619917 21629 tablet_replica.cc:333] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.620994 21629 raft_consensus.cc:2243] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.622138 21629 raft_consensus.cc:2272] T 77fb3ec3412341df88b759485f90f108 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.624148 31451 consensus_peers.cc:597] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.625591 21629 tablet_replica.cc:333] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.626564 21629 raft_consensus.cc:2243] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.627138 21629 raft_consensus.cc:2272] T 8e1eac6b3d8a47b59679e4aee4293326 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.628747 31303 consensus_peers.cc:597] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.629339 31303 consensus_peers.cc:597] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.629823 31377 consensus_peers.cc:597] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.629985 21629 tablet_replica.cc:333] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.630899 21629 raft_consensus.cc:2243] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.631668 21629 raft_consensus.cc:2272] T 229af634ffc1437fb9dea4a95652ab5b P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.632174 31303 consensus_peers.cc:597] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.634394 21629 tablet_replica.cc:333] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
W20260709 13:11:17.634733 31448 consensus_peers.cc:597] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.635154 21629 raft_consensus.cc:2243] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.635661 21629 raft_consensus.cc:2272] T bbfb2b2292c140bbaa93438cce1746cb P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.638208 21629 tablet_replica.cc:333] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.638963 21629 raft_consensus.cc:2243] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.639357 21629 raft_consensus.cc:2272] T d23babc199fb49fe9e6423834c97d1f7 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.641494 21629 tablet_replica.cc:333] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.642060 21629 raft_consensus.cc:2243] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.642551 21629 raft_consensus.cc:2272] T d9645a5374ed42a79a83e6af2d395bcc P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.644965 21629 tablet_replica.cc:333] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.645598 21629 raft_consensus.cc:2243] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.646054 21629 raft_consensus.cc:2272] T f213e6b4ef544937bbd9c8e49c672693 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.646755 31374 consensus_peers.cc:597] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.647367 31374 consensus_peers.cc:597] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.648659 21629 tablet_replica.cc:333] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.649336 21629 raft_consensus.cc:2243] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.650070 21629 raft_consensus.cc:2272] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.651880 21629 tablet_replica.cc:333] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.652413 21629 raft_consensus.cc:2243] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.652814 21629 raft_consensus.cc:2272] T c415b45c0cf0456bae70f23527c3488e P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.654749 21629 tablet_replica.cc:333] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.655265 21629 raft_consensus.cc:2243] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.656010 21629 raft_consensus.cc:2272] T f9baba29a028498288a27243e555fb27 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.658540 21629 tablet_replica.cc:333] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.659224 21629 raft_consensus.cc:2243] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.659677 21629 raft_consensus.cc:2272] T f564510ff30e42ec890531c82405946c P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.661517 21629 tablet_replica.cc:333] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.661974 21629 raft_consensus.cc:2243] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.662649 21629 raft_consensus.cc:2272] T 1d10f77e65c14720a8312b1dd44d3199 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.664587 21629 tablet_replica.cc:333] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.665191 21629 raft_consensus.cc:2243] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.665704 21629 raft_consensus.cc:2272] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.667136 31374 consensus_peers.cc:597] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.668136 21629 tablet_replica.cc:333] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.668852 21629 raft_consensus.cc:2243] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.669620 21629 raft_consensus.cc:2272] T 1b5b2bc298df4947b958fefdc9c2846e P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.671542 21629 tablet_replica.cc:333] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.672070 21629 raft_consensus.cc:2243] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.672442 21629 raft_consensus.cc:2272] T e69c3326d8874749a268ae8dfdca3b3c P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.674198 21629 tablet_replica.cc:333] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.674701 21629 raft_consensus.cc:2243] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.675421 21629 raft_consensus.cc:2272] T acf056205d504594827b6fda57d3cb62 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.677222 21629 tablet_replica.cc:333] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.677764 21629 raft_consensus.cc:2243] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.678161 21629 raft_consensus.cc:2272] T e12592f1e5624454a04480b80ff628e2 P 07b814fcedea4142ac09385b680cae3a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.679955 21629 tablet_replica.cc:333] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.680459 21629 raft_consensus.cc:2243] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.680881 21629 raft_consensus.cc:2272] T d4be50f8b2fe4ae4a303702831a3f006 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.682790 21629 tablet_replica.cc:333] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.683367 21629 raft_consensus.cc:2243] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.683827 21629 raft_consensus.cc:2272] T f4aaf32f729d46578d080ffe9a172eda P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.685636 21629 tablet_replica.cc:333] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.686164 21629 raft_consensus.cc:2243] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.686997 21629 raft_consensus.cc:2272] T a283218fb40d45d0a877dcd23bfc51d5 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.688815 21629 tablet_replica.cc:333] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.689342 21629 raft_consensus.cc:2243] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.689719 21629 raft_consensus.cc:2272] T c5846d47686948bcb0d921291b4bc5c9 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.691608 21629 tablet_replica.cc:333] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.692128 21629 raft_consensus.cc:2243] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.692549 21629 raft_consensus.cc:2272] T 92573de449ff4002a92b5ffcecfb8521 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.694444 31374 consensus_peers.cc:597] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.694559 21629 tablet_replica.cc:333] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.695204 21629 raft_consensus.cc:2243] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.695590 21629 raft_consensus.cc:2272] T c24424b197c343e1a57a4f490c67aaf3 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.697310 21629 tablet_replica.cc:333] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.697822 21629 raft_consensus.cc:2243] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.698199 21629 raft_consensus.cc:2272] T 663d2c98ef924d139ff1835938fef0df P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.699927 21629 tablet_replica.cc:333] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.700419 21629 raft_consensus.cc:2243] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.700811 21629 raft_consensus.cc:2272] T bfcdfdb50a8b463e851c89aa160fd949 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.702553 21629 tablet_replica.cc:333] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.703341 21629 raft_consensus.cc:2243] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.703724 21629 raft_consensus.cc:2272] T f81f5f6441044395977dbda8239d7289 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.705498 21629 tablet_replica.cc:333] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.706001 21629 raft_consensus.cc:2243] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.706980 21629 raft_consensus.cc:2272] T 4b06bd7c5b2a4586b9caf425bc23720a P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.709555 21629 tablet_replica.cc:333] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.710070 21629 raft_consensus.cc:2243] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.710485 21629 raft_consensus.cc:2272] T 5db13a06e22e411ba4c29d105c7e93f7 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.712323 31377 consensus_peers.cc:597] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 -> Peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Couldn't send request to peer 75b6f0017ce84a4c9822ffd27e053405. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.712572 21629 tablet_replica.cc:333] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.713197 21629 raft_consensus.cc:2243] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.714116 21629 raft_consensus.cc:2272] T 29510c165a1f48b1949c6ee31e136064 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.720780 21629 tablet_replica.cc:333] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.721558 21629 raft_consensus.cc:2243] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.722046 21629 raft_consensus.cc:2272] T 077ed73927a64748baae88b0b8dc4a13 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.724399 21629 tablet_replica.cc:333] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.724902 21629 raft_consensus.cc:2243] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.725273 21629 raft_consensus.cc:2272] T 142741f7f41a4baa824a20e79ddf5cb6 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.727169 21629 tablet_replica.cc:333] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.727676 21629 raft_consensus.cc:2243] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.728416 21629 raft_consensus.cc:2272] T 9b3eb1a472a546fdb51a2cc295716492 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.730916 21629 tablet_replica.cc:333] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.731549 21629 raft_consensus.cc:2243] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.732024 21629 raft_consensus.cc:2272] T fa7546f06a454f3899cb0bf7d0c0d084 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.732374 31374 consensus_peers.cc:597] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.734346 21629 tablet_replica.cc:333] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.735108 21629 raft_consensus.cc:2243] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.735447 21629 raft_consensus.cc:2272] T 67b810039c1e415192775c9f611cd825 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.736706 31374 consensus_peers.cc:597] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.737598 21629 tablet_replica.cc:333] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.738083 21629 raft_consensus.cc:2243] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.738708 21629 raft_consensus.cc:2272] T 35faa1d23acb4a6da70dc14e16ed6a69 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.740494 21629 tablet_replica.cc:333] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.741156 21629 raft_consensus.cc:2243] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.742046 21629 raft_consensus.cc:2272] T d905b9b08a224fe09df30744558819e3 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.744586 21629 tablet_replica.cc:333] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.745116 21629 raft_consensus.cc:2243] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.745528 21629 raft_consensus.cc:2272] T b41a57045456429c857b8fb94b54c356 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.747298 21629 tablet_replica.cc:333] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.747814 21629 raft_consensus.cc:2243] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.748688 21629 raft_consensus.cc:2272] T 942702984f5b4342bdd5399274624f90 P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.750443 21629 tablet_replica.cc:333] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.750895 21629 raft_consensus.cc:2243] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.751310 21629 raft_consensus.cc:2272] T bb349a8fff1b4d6f8dda0834dcec5b71 P 07b814fcedea4142ac09385b680cae3a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.752983 21629 tablet_replica.cc:333] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.753492 21629 raft_consensus.cc:2243] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:17.754160 21629 raft_consensus.cc:2272] T f12b34e35ea0429ea3c77b7d89fd748a P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.755853 21629 tablet_replica.cc:333] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.756304 21629 raft_consensus.cc:2243] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.756669 21629 raft_consensus.cc:2272] T d913d7bf7d154005a431dc28e2bd91f6 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.758424 21629 tablet_replica.cc:333] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.758868 21629 raft_consensus.cc:2243] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.759270 21629 raft_consensus.cc:2272] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.761850 21629 tablet_replica.cc:333] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.762311 21629 raft_consensus.cc:2243] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.762702 21629 raft_consensus.cc:2272] T a2ecee4c21f34ad0a27f7f36add491e9 P 07b814fcedea4142ac09385b680cae3a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.764423 21629 tablet_replica.cc:333] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a: stopping tablet replica
I20260709 13:11:17.764874 21629 raft_consensus.cc:2243] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.765259 21629 raft_consensus.cc:2272] T 73bf9042b4104cb693d9daf0f280166c P 07b814fcedea4142ac09385b680cae3a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:17.803678 31374 consensus_peers.cc:597] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.826619 31374 consensus_peers.cc:597] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.831959 31448 consensus_peers.cc:597] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.836187 31374 consensus_peers.cc:597] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.853153 31448 consensus_peers.cc:597] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.889424 21629 tablet_server.cc:196] TabletServer@127.21.31.66:0 shutdown complete.
W20260709 13:11:17.908855 31374 consensus_peers.cc:597] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:17.951468 31448 consensus_peers.cc:597] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 -> Peer 07b814fcedea4142ac09385b680cae3a (127.21.31.66:39075): Couldn't send request to peer 07b814fcedea4142ac09385b680cae3a. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:39075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:17.972168 21629 tablet_server.cc:179] TabletServer@127.21.31.67:0 shutting down...
W20260709 13:11:18.004623 31449 consensus_peers.cc:597] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.012419 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:18.013430 21629 tablet_replica.cc:333] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.014547 21629 raft_consensus.cc:2243] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.015492 21629 raft_consensus.cc:2272] T 2fb1e844232d4ed0b8752e51e31204b8 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.017720 21629 tablet_replica.cc:333] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.018481 21629 raft_consensus.cc:2243] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.018934 21629 raft_consensus.cc:2272] T 77fb3ec3412341df88b759485f90f108 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.021358 21629 tablet_replica.cc:333] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.022110 21629 raft_consensus.cc:2243] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.022533 21629 raft_consensus.cc:2272] T 229af634ffc1437fb9dea4a95652ab5b P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.024395 21629 tablet_replica.cc:333] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.025014 21629 raft_consensus.cc:2243] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.025938 21629 raft_consensus.cc:2272] T 9725e6248e7b4e67b23a3b99967003bf P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.027725 21629 tablet_replica.cc:333] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.028329 21629 raft_consensus.cc:2243] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.028800 21629 raft_consensus.cc:2272] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.032133 21629 tablet_replica.cc:333] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
W20260709 13:11:18.032363 31449 consensus_peers.cc:597] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.032923 21629 raft_consensus.cc:2243] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.033529 21629 raft_consensus.cc:2272] T c415b45c0cf0456bae70f23527c3488e P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.035473 21629 tablet_replica.cc:333] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.036067 21629 raft_consensus.cc:2243] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.036804 21629 raft_consensus.cc:2272] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.038694 21629 tablet_replica.cc:333] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.039254 21629 raft_consensus.cc:2243] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.039733 21629 raft_consensus.cc:2272] T f564510ff30e42ec890531c82405946c P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.041993 21629 tablet_replica.cc:333] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.042604 21629 raft_consensus.cc:2243] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.043023 21629 raft_consensus.cc:2272] T 9b3eb1a472a546fdb51a2cc295716492 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.044746 21629 tablet_replica.cc:333] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.045367 21629 raft_consensus.cc:2243] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.046074 21629 raft_consensus.cc:2272] T f213e6b4ef544937bbd9c8e49c672693 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.047830 21629 tablet_replica.cc:333] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.048398 21629 raft_consensus.cc:2243] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.048902 21629 raft_consensus.cc:2272] T 271f7357440943008e55795b0d6cbbda P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.050750 21629 tablet_replica.cc:333] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.051306 21629 raft_consensus.cc:2243] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.051688 21629 raft_consensus.cc:2272] T cfee9be6a99740c4ab0759431ad94310 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.054286 21629 tablet_replica.cc:333] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.054903 21629 raft_consensus.cc:2243] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.055388 21629 raft_consensus.cc:2272] T e69c3326d8874749a268ae8dfdca3b3c P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:18.056927 31449 consensus_peers.cc:597] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.057806 21629 tablet_replica.cc:333] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.058611 21629 raft_consensus.cc:2243] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.059547 21629 raft_consensus.cc:2272] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.061739 21629 tablet_replica.cc:333] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.062511 21629 raft_consensus.cc:2243] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.063023 21629 raft_consensus.cc:2272] T 29510c165a1f48b1949c6ee31e136064 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.065971 21629 tablet_replica.cc:333] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.066877 21629 raft_consensus.cc:2243] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.068094 21629 raft_consensus.cc:2272] T c5846d47686948bcb0d921291b4bc5c9 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.070890 21629 tablet_replica.cc:333] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.071700 21629 raft_consensus.cc:2243] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.072793 21629 raft_consensus.cc:2272] T 92573de449ff4002a92b5ffcecfb8521 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.075405 21629 tablet_replica.cc:333] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.076169 21629 raft_consensus.cc:2243] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.076701 21629 raft_consensus.cc:2272] T a283218fb40d45d0a877dcd23bfc51d5 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.079057 21629 tablet_replica.cc:333] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.079772 21629 raft_consensus.cc:2243] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.080225 21629 raft_consensus.cc:2272] T c90bfbd1ec9d458faadce4b9a625024b P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.082567 21629 tablet_replica.cc:333] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.083150 21629 raft_consensus.cc:2243] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.083564 21629 raft_consensus.cc:2272] T f81f5f6441044395977dbda8239d7289 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.085805 21629 tablet_replica.cc:333] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.086481 21629 raft_consensus.cc:2243] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.086949 21629 raft_consensus.cc:2272] T 0ad34ee917be4455bbe71da15c523cee P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.089280 21629 tablet_replica.cc:333] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.089954 21629 raft_consensus.cc:2243] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.090852 21629 raft_consensus.cc:2272] T e12592f1e5624454a04480b80ff628e2 P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.093367 21629 tablet_replica.cc:333] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.094072 21629 raft_consensus.cc:2243] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.094584 21629 raft_consensus.cc:2272] T acf056205d504594827b6fda57d3cb62 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.097090 21629 tablet_replica.cc:333] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.097801 21629 raft_consensus.cc:2243] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.098734 21629 raft_consensus.cc:2272] T d4be50f8b2fe4ae4a303702831a3f006 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:18.101049 31449 consensus_peers.cc:597] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.101167 21629 tablet_replica.cc:333] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.102129 21629 raft_consensus.cc:2243] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.103070 21629 raft_consensus.cc:2272] T f4aaf32f729d46578d080ffe9a172eda P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.105751 21629 tablet_replica.cc:333] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.106460 21629 raft_consensus.cc:2243] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.106921 21629 raft_consensus.cc:2272] T bfcdfdb50a8b463e851c89aa160fd949 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.110219 21629 tablet_replica.cc:333] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.110886 21629 raft_consensus.cc:2243] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.111375 21629 raft_consensus.cc:2272] T 142741f7f41a4baa824a20e79ddf5cb6 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.113987 21629 tablet_replica.cc:333] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.114676 21629 raft_consensus.cc:2243] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.115167 21629 raft_consensus.cc:2272] T 862d691f6d9e4d9fb185be53b562b8ff P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.117252 21629 tablet_replica.cc:333] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.117801 21629 raft_consensus.cc:2243] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.118221 21629 raft_consensus.cc:2272] T 35faa1d23acb4a6da70dc14e16ed6a69 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.120201 21629 tablet_replica.cc:333] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.120684 21629 raft_consensus.cc:2243] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.121389 21629 raft_consensus.cc:2272] T b41a57045456429c857b8fb94b54c356 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.123203 21629 tablet_replica.cc:333] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.123685 21629 raft_consensus.cc:2243] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.124083 21629 raft_consensus.cc:2272] T f70d36cace8741d7b866e776f6cc03c1 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.125988 21629 tablet_replica.cc:333] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.126472 21629 raft_consensus.cc:2243] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.126842 21629 raft_consensus.cc:2272] T f12b34e35ea0429ea3c77b7d89fd748a P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.128592 21629 tablet_replica.cc:333] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.129081 21629 raft_consensus.cc:2243] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.129503 21629 raft_consensus.cc:2272] T bb349a8fff1b4d6f8dda0834dcec5b71 P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.131245 21629 tablet_replica.cc:333] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.131731 21629 raft_consensus.cc:2243] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.132115 21629 raft_consensus.cc:2272] T 942702984f5b4342bdd5399274624f90 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.134029 21629 tablet_replica.cc:333] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.134505 21629 raft_consensus.cc:2243] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.134893 21629 raft_consensus.cc:2272] T 914efa83b45e44dbbabeca35e75d0f3e P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.136576 21629 tablet_replica.cc:333] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.137065 21629 raft_consensus.cc:2243] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.137522 21629 raft_consensus.cc:2272] T 24fe066d118d48c5939576417491e9ec P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.139269 21629 tablet_replica.cc:333] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.139729 21629 raft_consensus.cc:2243] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.140087 21629 raft_consensus.cc:2272] T 181fa1bbe1e14e098cad0b23561576d2 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.142365 21629 tablet_replica.cc:333] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.142856 21629 raft_consensus.cc:2243] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.143697 21629 raft_consensus.cc:2272] T a6c9185858d64cfca85bfe148ea6bef0 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.146552 21629 tablet_replica.cc:333] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.147168 21629 raft_consensus.cc:2243] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.148268 21629 raft_consensus.cc:2272] T 67b810039c1e415192775c9f611cd825 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.150481 21629 tablet_replica.cc:333] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.151029 21629 raft_consensus.cc:2243] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.151463 21629 raft_consensus.cc:2272] T dda3de9e2b1643049b35455ababf5d82 P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.153587 21629 tablet_replica.cc:333] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.154134 21629 raft_consensus.cc:2243] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.155012 21629 raft_consensus.cc:2272] T bbfb2b2292c140bbaa93438cce1746cb P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.157086 21629 tablet_replica.cc:333] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.157649 21629 raft_consensus.cc:2243] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.158052 21629 raft_consensus.cc:2272] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 51b0533abe064293ac5047de0ac9bf25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.159929 21629 tablet_replica.cc:333] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.160470 21629 raft_consensus.cc:2243] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.160862 21629 raft_consensus.cc:2272] T 7c760b7263fd41fea6c0a6dfd9dc5d85 P 51b0533abe064293ac5047de0ac9bf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.162654 21629 tablet_replica.cc:333] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25: stopping tablet replica
I20260709 13:11:18.163285 21629 raft_consensus.cc:2243] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.164113 21629 raft_consensus.cc:2272] T 73bf9042b4104cb693d9daf0f280166c P 51b0533abe064293ac5047de0ac9bf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.235035 21629 tablet_server.cc:196] TabletServer@127.21.31.67:0 shutdown complete.
W20260709 13:11:18.273967 31449 consensus_peers.cc:597] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.67:45117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:18.289803 31449 consensus_peers.cc:597] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.67:45117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.300275 21629 tablet_server.cc:179] TabletServer@127.21.31.68:0 shutting down...
I20260709 13:11:18.336513 32371 raft_consensus.cc:493] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:18.337117 32371 raft_consensus.cc:515] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:18.339457 32371 leader_election.cc:290] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
W20260709 13:11:18.345140 31451 leader_election.cc:336] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
W20260709 13:11:18.346598 31449 leader_election.cc:336] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Network error: Client connection negotiation failed: client connection to 127.21.31.67:45117: connect: Connection refused (error 111)
I20260709 13:11:18.346920 31449 leader_election.cc:304] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:18.347458 32371 raft_consensus.cc:2749] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:18.348876 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:18.349694 21629 tablet_replica.cc:333] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.350317 21629 raft_consensus.cc:2243] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.350775 21629 raft_consensus.cc:2272] T 1dfef8e9e4964ab3b8b7c10e84a9d293 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.352845 21629 tablet_replica.cc:333] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.353453 21629 raft_consensus.cc:2243] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.353896 21629 raft_consensus.cc:2272] T cfee9be6a99740c4ab0759431ad94310 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.356168 21629 tablet_replica.cc:333] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.356861 21629 raft_consensus.cc:2243] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.357820 21629 raft_consensus.cc:2272] T 8e1eac6b3d8a47b59679e4aee4293326 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.360440 21629 tablet_replica.cc:333] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.361129 21629 raft_consensus.cc:2243] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.361613 21629 raft_consensus.cc:2272] T 9725e6248e7b4e67b23a3b99967003bf P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:18.363267 31449 consensus_peers.cc:597] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.67:45117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.364200 21629 tablet_replica.cc:333] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.364876 21629 raft_consensus.cc:2243] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.365545 21629 raft_consensus.cc:2272] T d23babc199fb49fe9e6423834c97d1f7 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.367733 21629 tablet_replica.cc:333] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.368352 21629 raft_consensus.cc:2243] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.368845 21629 raft_consensus.cc:2272] T d905b9b08a224fe09df30744558819e3 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.371018 21629 tablet_replica.cc:333] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.371577 21629 raft_consensus.cc:2243] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.371994 21629 raft_consensus.cc:2272] T e0bc6ebfdfef4b7a81d4cbc17a86d70f P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.374095 21629 tablet_replica.cc:333] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.374680 21629 raft_consensus.cc:2243] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.375095 21629 raft_consensus.cc:2272] T d9645a5374ed42a79a83e6af2d395bcc P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.377075 21629 tablet_replica.cc:333] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.377668 21629 raft_consensus.cc:2243] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.378144 21629 raft_consensus.cc:2272] T f213e6b4ef544937bbd9c8e49c672693 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.380384 21629 tablet_replica.cc:333] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.380956 21629 raft_consensus.cc:2243] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:18.381029 31449 consensus_peers.cc:597] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.67:45117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.381450 21629 raft_consensus.cc:2272] T 271f7357440943008e55795b0d6cbbda P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.383649 21629 tablet_replica.cc:333] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.384312 21629 raft_consensus.cc:2243] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.385020 21629 raft_consensus.cc:2272] T f564510ff30e42ec890531c82405946c P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.386792 21629 tablet_replica.cc:333] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.387316 21629 raft_consensus.cc:2243] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.387687 21629 raft_consensus.cc:2272] T 2fb1e844232d4ed0b8752e51e31204b8 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.389609 21629 tablet_replica.cc:333] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.390122 21629 raft_consensus.cc:2243] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.390480 21629 raft_consensus.cc:2272] T f9baba29a028498288a27243e555fb27 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.392251 21629 tablet_replica.cc:333] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.392763 21629 raft_consensus.cc:2243] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.393164 21629 raft_consensus.cc:2272] T 1b5b2bc298df4947b958fefdc9c2846e P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.394902 21629 tablet_replica.cc:333] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.395416 21629 raft_consensus.cc:2243] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.395793 21629 raft_consensus.cc:2272] T a6c9185858d64cfca85bfe148ea6bef0 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.397557 21629 tablet_replica.cc:333] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.398242 21629 raft_consensus.cc:2243] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.398703 21629 raft_consensus.cc:2272] T 1d10f77e65c14720a8312b1dd44d3199 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.401122 21629 tablet_replica.cc:333] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.401885 21629 raft_consensus.cc:2243] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.402421 21629 raft_consensus.cc:2272] T c5846d47686948bcb0d921291b4bc5c9 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.404469 21629 tablet_replica.cc:333] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.405016 21629 raft_consensus.cc:2243] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.405447 21629 raft_consensus.cc:2272] T a283218fb40d45d0a877dcd23bfc51d5 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:18.405555 31449 consensus_peers.cc:597] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 -> Peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Couldn't send request to peer 51b0533abe064293ac5047de0ac9bf25. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.67:45117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.407256 21629 tablet_replica.cc:333] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.407828 21629 raft_consensus.cc:2243] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.408839 21629 raft_consensus.cc:2272] T 0ad34ee917be4455bbe71da15c523cee P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.410749 21629 tablet_replica.cc:333] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.411247 21629 raft_consensus.cc:2243] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.411631 21629 raft_consensus.cc:2272] T acf056205d504594827b6fda57d3cb62 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.413414 21629 tablet_replica.cc:333] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.413913 21629 raft_consensus.cc:2243] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.414101 32371 raft_consensus.cc:493] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 75b6f0017ce84a4c9822ffd27e053405)
I20260709 13:11:18.414309 21629 raft_consensus.cc:2272] T d4be50f8b2fe4ae4a303702831a3f006 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.414559 32371 raft_consensus.cc:515] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b0533abe064293ac5047de0ac9bf25" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 45117 } } peers { permanent_uuid: "001cac731610478a900ad62e830aaff6" member_type: VOTER last_known_addr { host: "127.21.31.68" port: 44895 } } peers { permanent_uuid: "75b6f0017ce84a4c9822ffd27e053405" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 36133 } }
I20260709 13:11:18.416319 21629 tablet_replica.cc:333] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.417196 21629 raft_consensus.cc:2243] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.416337 32371 leader_election.cc:290] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117), 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133)
I20260709 13:11:18.418422 21629 raft_consensus.cc:2272] T d6498a8fe8ab41d19cfe4afd8ccfb904 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:18.420508 31449 leader_election.cc:336] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 51b0533abe064293ac5047de0ac9bf25 (127.21.31.67:45117): Network error: Client connection negotiation failed: client connection to 127.21.31.67:45117: connect: Connection refused (error 111)
I20260709 13:11:18.420881 21629 tablet_replica.cc:333] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6: stopping tablet replica
W20260709 13:11:18.421221 31451 leader_election.cc:336] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 75b6f0017ce84a4c9822ffd27e053405 (127.21.31.65:36133): Network error: Client connection negotiation failed: client connection to 127.21.31.65:36133: connect: Connection refused (error 111)
I20260709 13:11:18.421670 21629 raft_consensus.cc:2243] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.421641 31451 leader_election.cc:304] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 001cac731610478a900ad62e830aaff6; no voters: 51b0533abe064293ac5047de0ac9bf25, 75b6f0017ce84a4c9822ffd27e053405
I20260709 13:11:18.422271 21629 raft_consensus.cc:2272] T 4b06bd7c5b2a4586b9caf425bc23720a P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.422374 32371 raft_consensus.cc:2749] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:18.424314 21629 tablet_replica.cc:333] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.424886 21629 raft_consensus.cc:2243] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.425272 21629 raft_consensus.cc:2272] T c90bfbd1ec9d458faadce4b9a625024b P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.427465 21629 tablet_replica.cc:333] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.428011 21629 raft_consensus.cc:2243] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.428525 21629 raft_consensus.cc:2272] T f4aaf32f729d46578d080ffe9a172eda P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.431058 21629 tablet_replica.cc:333] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.431799 21629 raft_consensus.cc:2243] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.432236 21629 raft_consensus.cc:2272] T c24424b197c343e1a57a4f490c67aaf3 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.435341 21629 tablet_replica.cc:333] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.436201 21629 raft_consensus.cc:2243] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.436753 21629 raft_consensus.cc:2272] T 663d2c98ef924d139ff1835938fef0df P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.439126 21629 tablet_replica.cc:333] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.439723 21629 raft_consensus.cc:2243] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.440534 21629 raft_consensus.cc:2272] T 24fe066d118d48c5939576417491e9ec P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.442611 21629 tablet_replica.cc:333] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.443218 21629 raft_consensus.cc:2243] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.444021 21629 raft_consensus.cc:2272] T 862d691f6d9e4d9fb185be53b562b8ff P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.446563 21629 tablet_replica.cc:333] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.447106 21629 raft_consensus.cc:2243] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.447547 21629 raft_consensus.cc:2272] T 35faa1d23acb4a6da70dc14e16ed6a69 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.449947 21629 tablet_replica.cc:333] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.450661 21629 raft_consensus.cc:2243] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.451797 21629 raft_consensus.cc:2272] T f70d36cace8741d7b866e776f6cc03c1 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.454088 21629 tablet_replica.cc:333] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.454622 21629 raft_consensus.cc:2243] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.455394 21629 raft_consensus.cc:2272] T 5db13a06e22e411ba4c29d105c7e93f7 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.457537 21629 tablet_replica.cc:333] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.458058 21629 raft_consensus.cc:2243] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.458846 21629 raft_consensus.cc:2272] T 077ed73927a64748baae88b0b8dc4a13 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.460798 21629 tablet_replica.cc:333] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.461457 21629 raft_consensus.cc:2243] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.462226 21629 raft_consensus.cc:2272] T 142741f7f41a4baa824a20e79ddf5cb6 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.464365 21629 tablet_replica.cc:333] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.464942 21629 raft_consensus.cc:2243] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.465793 21629 raft_consensus.cc:2272] T fa7546f06a454f3899cb0bf7d0c0d084 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.467895 21629 tablet_replica.cc:333] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.468467 21629 raft_consensus.cc:2243] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.468884 21629 raft_consensus.cc:2272] T d913d7bf7d154005a431dc28e2bd91f6 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.470721 21629 tablet_replica.cc:333] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.471297 21629 raft_consensus.cc:2243] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.471712 21629 raft_consensus.cc:2272] T 73bf9042b4104cb693d9daf0f280166c P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.473688 21629 tablet_replica.cc:333] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.474223 21629 raft_consensus.cc:2243] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.474653 21629 raft_consensus.cc:2272] T a2ecee4c21f34ad0a27f7f36add491e9 P 001cac731610478a900ad62e830aaff6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.476526 21629 tablet_replica.cc:333] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.477046 21629 raft_consensus.cc:2243] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.477474 21629 raft_consensus.cc:2272] T 942702984f5b4342bdd5399274624f90 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.479449 21629 tablet_replica.cc:333] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.480136 21629 raft_consensus.cc:2243] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.480976 21629 raft_consensus.cc:2272] T 914efa83b45e44dbbabeca35e75d0f3e P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.483302 21629 tablet_replica.cc:333] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.483901 21629 raft_consensus.cc:2243] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.484791 21629 raft_consensus.cc:2272] T 181fa1bbe1e14e098cad0b23561576d2 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.486611 21629 tablet_replica.cc:333] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.487138 21629 raft_consensus.cc:2243] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.487861 21629 raft_consensus.cc:2272] T dda3de9e2b1643049b35455ababf5d82 P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.489768 21629 tablet_replica.cc:333] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.490218 21629 raft_consensus.cc:2243] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.490638 21629 raft_consensus.cc:2272] T bbfb2b2292c140bbaa93438cce1746cb P 001cac731610478a900ad62e830aaff6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.492527 21629 tablet_replica.cc:333] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6: stopping tablet replica
I20260709 13:11:18.492980 21629 raft_consensus.cc:2243] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.493729 21629 raft_consensus.cc:2272] T 1b9a979d5b8b4d02b133ffc4078d9cbf P 001cac731610478a900ad62e830aaff6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.558287 21629 tablet_server.cc:196] TabletServer@127.21.31.68:0 shutdown complete.
I20260709 13:11:18.621626 21629 master.cc:562] Master@127.21.31.126:37055 shutting down...
I20260709 13:11:18.647254 21629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:18.647734 21629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.648072 21629 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9ca35edff3a347dc8711fd15b6385750: stopping tablet replica
I20260709 13:11:18.668571 21629 master.cc:584] Master@127.21.31.126:37055 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (30509 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:11:18.755693 21629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.31.126:40843
I20260709 13:11:18.756793 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:18.762657 32387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:18.762665 32388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:18.763847 32390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:18.768034 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:18.769073 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:18.769244 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:18.769670 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602678769653 us; error 0 us; skew 500 ppm
I20260709 13:11:18.770121 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:18.772550 21629 webserver.cc:533] Webserver started at http://127.21.31.126:35191/ using document root <none> and password file <none>
I20260709 13:11:18.772965 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:18.773123 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:18.773388 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:18.774492 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/master-0-root/instance:
uuid: "76af26a549894d93aa89efc93864862b"
format_stamp: "Formatted at 2026-07-09 13:11:18 on dist-test-slave-9s6w"
I20260709 13:11:18.778618 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:11:18.781844 32395 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:18.782721 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:11:18.782963 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/master-0-root
uuid: "76af26a549894d93aa89efc93864862b"
format_stamp: "Formatted at 2026-07-09 13:11:18 on dist-test-slave-9s6w"
I20260709 13:11:18.783187 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:18.802284 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:18.803368 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:18.841353 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.126:40843
I20260709 13:11:18.841509 32446 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.126:40843 every 8 connection(s)
I20260709 13:11:18.844981 32447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:18.854668 32447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b: Bootstrap starting.
I20260709 13:11:18.859145 32447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:18.863061 32447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b: No bootstrap required, opened a new log
I20260709 13:11:18.865011 32447 raft_consensus.cc:359] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76af26a549894d93aa89efc93864862b" member_type: VOTER }
I20260709 13:11:18.865407 32447 raft_consensus.cc:385] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:18.865626 32447 raft_consensus.cc:740] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76af26a549894d93aa89efc93864862b, State: Initialized, Role: FOLLOWER
I20260709 13:11:18.866243 32447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76af26a549894d93aa89efc93864862b" member_type: VOTER }
I20260709 13:11:18.866904 32447 raft_consensus.cc:399] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:11:18.867206 32447 raft_consensus.cc:493] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:11:18.867517 32447 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:18.873958 32447 raft_consensus.cc:515] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76af26a549894d93aa89efc93864862b" member_type: VOTER }
I20260709 13:11:18.874471 32447 leader_election.cc:304] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76af26a549894d93aa89efc93864862b; no voters: 
I20260709 13:11:18.875526 32447 leader_election.cc:290] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:11:18.875823 32450 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:18.877151 32450 raft_consensus.cc:697] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 1 LEADER]: Becoming Leader. State: Replica: 76af26a549894d93aa89efc93864862b, State: Running, Role: LEADER
I20260709 13:11:18.877828 32450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76af26a549894d93aa89efc93864862b" member_type: VOTER }
I20260709 13:11:18.878393 32447 sys_catalog.cc:565] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:11:18.881819 32451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76af26a549894d93aa89efc93864862b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76af26a549894d93aa89efc93864862b" member_type: VOTER } }
I20260709 13:11:18.882582 32451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:18.882588 32452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76af26a549894d93aa89efc93864862b. Latest consensus state: current_term: 1 leader_uuid: "76af26a549894d93aa89efc93864862b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76af26a549894d93aa89efc93864862b" member_type: VOTER } }
I20260709 13:11:18.884063 32452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:18.884634 32457 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:11:18.892570 32457 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:11:18.894676 21629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:11:18.901680 32457 catalog_manager.cc:1383] Generated new cluster ID: 0bc837c7c4c74821b5e1f1b87deb41e8
I20260709 13:11:18.901957 32457 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:11:18.920125 32457 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:11:18.921339 32457 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:11:18.935451 32457 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b: Generated new TSK 0
I20260709 13:11:18.935966 32457 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:11:18.960860 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:18.968333 32468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:18.968477 32469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:18.972841 21629 server_base.cc:1061] running on GCE node
W20260709 13:11:18.972713 32471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:18.976744 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:18.976939 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:18.977146 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602678977113 us; error 0 us; skew 500 ppm
I20260709 13:11:18.977699 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:18.979945 21629 webserver.cc:533] Webserver started at http://127.21.31.65:43653/ using document root <none> and password file <none>
I20260709 13:11:18.980403 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:18.980563 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:18.980856 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:18.981961 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-0-root/instance:
uuid: "7edad4013cb44461aad4b40ad080cd98"
format_stamp: "Formatted at 2026-07-09 13:11:18 on dist-test-slave-9s6w"
I20260709 13:11:18.986519 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:11:18.990535 32476 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:18.991438 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:11:18.991719 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "7edad4013cb44461aad4b40ad080cd98"
format_stamp: "Formatted at 2026-07-09 13:11:18 on dist-test-slave-9s6w"
I20260709 13:11:18.991945 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:19.016325 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:19.017457 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:19.018771 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:19.020855 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:19.021031 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:19.021303 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:19.021531 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.001s
I20260709 13:11:19.063714 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:38411
I20260709 13:11:19.063805 32538 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:38411 every 8 connection(s)
I20260709 13:11:19.068320 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:19.076368 32543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:19.077493 32544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:19.081074 32546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:19.081550 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:19.082435 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:19.082653 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:19.082855 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602679082834 us; error 0 us; skew 500 ppm
I20260709 13:11:19.083428 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:19.086158 21629 webserver.cc:533] Webserver started at http://127.21.31.66:34067/ using document root <none> and password file <none>
I20260709 13:11:19.086282 32539 heartbeater.cc:344] Connected to a master server at 127.21.31.126:40843
I20260709 13:11:19.086674 32539 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:19.086741 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:19.087056 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:19.087319 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:19.087399 32539 heartbeater.cc:507] Master 127.21.31.126:40843 requested a full tablet report, sending...
I20260709 13:11:19.088613 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-1-root/instance:
uuid: "c372ed6a9e6546f591cd1c6ab584f015"
format_stamp: "Formatted at 2026-07-09 13:11:19 on dist-test-slave-9s6w"
I20260709 13:11:19.089479 32412 ts_manager.cc:194] Registered new tserver with Master: 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411)
I20260709 13:11:19.091109 32412 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33082
I20260709 13:11:19.093727 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:11:19.096944 32551 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:19.097921 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:11:19.098168 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-1-root
uuid: "c372ed6a9e6546f591cd1c6ab584f015"
format_stamp: "Formatted at 2026-07-09 13:11:19 on dist-test-slave-9s6w"
I20260709 13:11:19.098389 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:19.110826 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:19.111835 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:19.113215 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:19.115244 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:19.115407 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:19.115689 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:19.115825 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:19.157603 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.66:43781
I20260709 13:11:19.157718 32613 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.66:43781 every 8 connection(s)
I20260709 13:11:19.162249 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:19.169721 32617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:19.170585 32618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:19.172217 32620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:19.173612 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:19.174602 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:19.174836 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:19.175043 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602679175023 us; error 0 us; skew 500 ppm
I20260709 13:11:19.175124 32614 heartbeater.cc:344] Connected to a master server at 127.21.31.126:40843
I20260709 13:11:19.175523 32614 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:19.175622 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:19.176339 32614 heartbeater.cc:507] Master 127.21.31.126:40843 requested a full tablet report, sending...
I20260709 13:11:19.178650 21629 webserver.cc:533] Webserver started at http://127.21.31.67:39395/ using document root <none> and password file <none>
I20260709 13:11:19.178722 32412 ts_manager.cc:194] Registered new tserver with Master: c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:19.179477 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:19.179836 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:19.180215 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:19.180402 32412 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33086
I20260709 13:11:19.181512 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-2-root/instance:
uuid: "b50e45fe51014825a6cb95a1dac5603d"
format_stamp: "Formatted at 2026-07-09 13:11:19 on dist-test-slave-9s6w"
I20260709 13:11:19.186117 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:11:19.189572 32625 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:19.190408 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:11:19.190708 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-2-root
uuid: "b50e45fe51014825a6cb95a1dac5603d"
format_stamp: "Formatted at 2026-07-09 13:11:19 on dist-test-slave-9s6w"
I20260709 13:11:19.190958 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:19.205440 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:19.206743 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:19.208318 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:19.210717 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:19.210953 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:19.211205 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:19.211373 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:19.251873 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.67:46139
I20260709 13:11:19.252139 32687 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.67:46139 every 8 connection(s)
I20260709 13:11:19.266657 32688 heartbeater.cc:344] Connected to a master server at 127.21.31.126:40843
I20260709 13:11:19.267091 32688 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:19.267750 32688 heartbeater.cc:507] Master 127.21.31.126:40843 requested a full tablet report, sending...
I20260709 13:11:19.269721 32412 ts_manager.cc:194] Registered new tserver with Master: b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:19.270699 21629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015036796s
I20260709 13:11:19.271016 32412 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33102
I20260709 13:11:19.298295 32412 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33106:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:19.301235 32412 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:19.377660 32504 tablet_service.cc:1511] Processing CreateTablet for tablet 8bb6e466e80141f88334cafd8db9d24f (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:11:19.379065 32504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bb6e466e80141f88334cafd8db9d24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.378944 32502 tablet_service.cc:1511] Processing CreateTablet for tablet 3884a2add97c4cb197b49ef63fc4fabe (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:11:19.380409 32502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3884a2add97c4cb197b49ef63fc4fabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.382536 32503 tablet_service.cc:1511] Processing CreateTablet for tablet 24939c381a3346b1b36b86c1ce7b8ceb (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:11:19.383863 32503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24939c381a3346b1b36b86c1ce7b8ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.384173 32651 tablet_service.cc:1511] Processing CreateTablet for tablet 3884a2add97c4cb197b49ef63fc4fabe (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:11:19.385038 32650 tablet_service.cc:1511] Processing CreateTablet for tablet de79c8f3ab0142cfbc6d474965e86646 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:11:19.385571 32651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3884a2add97c4cb197b49ef63fc4fabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.386353 32650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de79c8f3ab0142cfbc6d474965e86646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.387373 32501 tablet_service.cc:1511] Processing CreateTablet for tablet de79c8f3ab0142cfbc6d474965e86646 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:11:19.388684 32501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de79c8f3ab0142cfbc6d474965e86646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.389605 32653 tablet_service.cc:1511] Processing CreateTablet for tablet 8bb6e466e80141f88334cafd8db9d24f (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:11:19.390357 32500 tablet_service.cc:1511] Processing CreateTablet for tablet 3825b68a7235435bb84fa43308cfa0ad (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:11:19.391592 32500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3825b68a7235435bb84fa43308cfa0ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.391717 32652 tablet_service.cc:1511] Processing CreateTablet for tablet 24939c381a3346b1b36b86c1ce7b8ceb (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:11:19.393925 32649 tablet_service.cc:1511] Processing CreateTablet for tablet 3825b68a7235435bb84fa43308cfa0ad (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:11:19.394490 32499 tablet_service.cc:1511] Processing CreateTablet for tablet c8fe0a72962740438a86dedeb1009f84 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:11:19.395952 32499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8fe0a72962740438a86dedeb1009f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.396311 32653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bb6e466e80141f88334cafd8db9d24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.396603 32498 tablet_service.cc:1511] Processing CreateTablet for tablet 67690ebee6ca4f21bacd1292263e8112 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:11:19.397817 32498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67690ebee6ca4f21bacd1292263e8112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.398087 32652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24939c381a3346b1b36b86c1ce7b8ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.400421 32497 tablet_service.cc:1511] Processing CreateTablet for tablet 3391c7fa5f274de5a49d3d43752d262b (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:11:19.401677 32497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3391c7fa5f274de5a49d3d43752d262b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.401968 32648 tablet_service.cc:1511] Processing CreateTablet for tablet c8fe0a72962740438a86dedeb1009f84 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:11:19.403246 32648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8fe0a72962740438a86dedeb1009f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.405714 32649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3825b68a7235435bb84fa43308cfa0ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.406476 32647 tablet_service.cc:1511] Processing CreateTablet for tablet 67690ebee6ca4f21bacd1292263e8112 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:11:19.407737 32647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67690ebee6ca4f21bacd1292263e8112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.412866 32646 tablet_service.cc:1511] Processing CreateTablet for tablet 3391c7fa5f274de5a49d3d43752d262b (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:11:19.414141 32646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3391c7fa5f274de5a49d3d43752d262b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.438445 32579 tablet_service.cc:1511] Processing CreateTablet for tablet 8bb6e466e80141f88334cafd8db9d24f (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:11:19.439839 32579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bb6e466e80141f88334cafd8db9d24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.440016 32496 tablet_service.cc:1511] Processing CreateTablet for tablet 53dbd0bff9b847c38724e492a61b09d6 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:11:19.441838 32496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53dbd0bff9b847c38724e492a61b09d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.442559 32578 tablet_service.cc:1511] Processing CreateTablet for tablet 24939c381a3346b1b36b86c1ce7b8ceb (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:11:19.443959 32578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24939c381a3346b1b36b86c1ce7b8ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.446342 32577 tablet_service.cc:1511] Processing CreateTablet for tablet 3884a2add97c4cb197b49ef63fc4fabe (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:11:19.447671 32577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3884a2add97c4cb197b49ef63fc4fabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.450449 32576 tablet_service.cc:1511] Processing CreateTablet for tablet de79c8f3ab0142cfbc6d474965e86646 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:11:19.451650 32576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de79c8f3ab0142cfbc6d474965e86646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.454399 32575 tablet_service.cc:1511] Processing CreateTablet for tablet 3825b68a7235435bb84fa43308cfa0ad (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:11:19.456139 32575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3825b68a7235435bb84fa43308cfa0ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.458350 32574 tablet_service.cc:1511] Processing CreateTablet for tablet c8fe0a72962740438a86dedeb1009f84 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:11:19.459625 32574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8fe0a72962740438a86dedeb1009f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.476977 32645 tablet_service.cc:1511] Processing CreateTablet for tablet 53dbd0bff9b847c38724e492a61b09d6 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:11:19.477564 32495 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdd94ee0820431b9d14406fe6100bb9 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:11:19.478456 32645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53dbd0bff9b847c38724e492a61b09d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.478847 32495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdd94ee0820431b9d14406fe6100bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.489970 32573 tablet_service.cc:1511] Processing CreateTablet for tablet 67690ebee6ca4f21bacd1292263e8112 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:11:19.495434 32572 tablet_service.cc:1511] Processing CreateTablet for tablet 3391c7fa5f274de5a49d3d43752d262b (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:11:19.495890 32704 tablet_bootstrap.cc:492] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
I20260709 13:11:19.496268 32573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67690ebee6ca4f21bacd1292263e8112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.496941 32572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3391c7fa5f274de5a49d3d43752d262b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.498150 32571 tablet_service.cc:1511] Processing CreateTablet for tablet 53dbd0bff9b847c38724e492a61b09d6 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:11:19.499356 32571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53dbd0bff9b847c38724e492a61b09d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.503811 32570 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdd94ee0820431b9d14406fe6100bb9 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:11:19.505024 32570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdd94ee0820431b9d14406fe6100bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.508086 32653 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdd94ee0820431b9d14406fe6100bb9 (DEFAULT_TABLE table=test-workload [id=6c78fef697024706bcbe03752b435339]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:11:19.509508 32653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdd94ee0820431b9d14406fe6100bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.511420 32704 tablet_bootstrap.cc:654] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.551117 32703 tablet_bootstrap.cc:492] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:19.582764 32705 tablet_bootstrap.cc:492] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:19.608417 32703 tablet_bootstrap.cc:654] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:19.612200 32397 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.31.67:46139, user_credentials={real_user=slave}} blocked reactor thread for 87950.6us
W20260709 13:11:19.612389 32398 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.31.66:43781, user_credentials={real_user=slave}} blocked reactor thread for 45360us
W20260709 13:11:19.617682 32399 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.31.65:38411, user_credentials={real_user=slave}} blocked reactor thread for 83834.5us
I20260709 13:11:19.620131 32703 tablet_bootstrap.cc:492] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:19.620813 32703 ts_tablet_manager.cc:1403] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.070s	user 0.009s	sys 0.005s
I20260709 13:11:19.624341 32703 raft_consensus.cc:359] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.628182 32703 raft_consensus.cc:385] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.628536 32703 raft_consensus.cc:740] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.629612 32703 consensus_queue.cc:260] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.646354 32704 tablet_bootstrap.cc:492] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:19.646893 32704 ts_tablet_manager.cc:1403] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.151s	user 0.009s	sys 0.005s
I20260709 13:11:19.650002 32704 raft_consensus.cc:359] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.650666 32704 raft_consensus.cc:385] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.650965 32704 raft_consensus.cc:740] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.651757 32539 heartbeater.cc:499] Master 127.21.31.126:40843 was elected leader, sending a full tablet report...
I20260709 13:11:19.652432 32703 ts_tablet_manager.cc:1434] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.031s	user 0.013s	sys 0.009s
I20260709 13:11:19.653505 32703 tablet_bootstrap.cc:492] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:19.656782 32704 consensus_queue.cc:260] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.660126 32703 tablet_bootstrap.cc:654] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.663822 32688 heartbeater.cc:499] Master 127.21.31.126:40843 was elected leader, sending a full tablet report...
I20260709 13:11:19.664108 32704 ts_tablet_manager.cc:1434] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:11:19.665081 32704 tablet_bootstrap.cc:492] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
I20260709 13:11:19.650683 32705 tablet_bootstrap.cc:654] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.671578 32704 tablet_bootstrap.cc:654] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.676461 32703 tablet_bootstrap.cc:492] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:19.677012 32703 ts_tablet_manager.cc:1403] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.016s
I20260709 13:11:19.679790 32703 raft_consensus.cc:359] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.680559 32703 raft_consensus.cc:385] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.681051 32703 raft_consensus.cc:740] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.682013 32703 consensus_queue.cc:260] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.682705 32704 tablet_bootstrap.cc:492] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:19.683221 32704 ts_tablet_manager.cc:1403] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.003s
I20260709 13:11:19.683818 32703 ts_tablet_manager.cc:1434] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:11:19.684752 32703 tablet_bootstrap.cc:492] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:19.686061 32704 raft_consensus.cc:359] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.686784 32704 raft_consensus.cc:385] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.687063 32704 raft_consensus.cc:740] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.687826 32704 consensus_queue.cc:260] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.690018 32704 ts_tablet_manager.cc:1434] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.005s
I20260709 13:11:19.690956 32705 tablet_bootstrap.cc:492] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:19.690969 32704 tablet_bootstrap.cc:492] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
I20260709 13:11:19.691452 32703 tablet_bootstrap.cc:654] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.691601 32705 ts_tablet_manager.cc:1403] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.109s	user 0.008s	sys 0.004s
I20260709 13:11:19.695000 32705 raft_consensus.cc:359] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.695880 32705 raft_consensus.cc:385] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.696246 32705 raft_consensus.cc:740] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.697296 32705 consensus_queue.cc:260] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.700014 32704 tablet_bootstrap.cc:654] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.702968 32703 tablet_bootstrap.cc:492] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:19.703558 32703 ts_tablet_manager.cc:1403] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.000s
I20260709 13:11:19.707221 32703 raft_consensus.cc:359] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.708001 32703 raft_consensus.cc:385] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.708272 32703 raft_consensus.cc:740] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.708910 32703 consensus_queue.cc:260] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.710692 32705 ts_tablet_manager.cc:1434] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.019s	user 0.012s	sys 0.007s
I20260709 13:11:19.711028 32703 ts_tablet_manager.cc:1434] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:19.711432 32705 tablet_bootstrap.cc:492] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:19.711620 32614 heartbeater.cc:499] Master 127.21.31.126:40843 was elected leader, sending a full tablet report...
I20260709 13:11:19.712001 32703 tablet_bootstrap.cc:492] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:19.716059 32709 raft_consensus.cc:493] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.716321 32704 tablet_bootstrap.cc:492] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:19.716941 32704 ts_tablet_manager.cc:1403] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.009s
I20260709 13:11:19.716588 32709 raft_consensus.cc:515] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.718587 32705 tablet_bootstrap.cc:654] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.718834 32703 tablet_bootstrap.cc:654] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.720180 32704 raft_consensus.cc:359] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.720943 32704 raft_consensus.cc:385] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.721282 32704 raft_consensus.cc:740] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.722397 32704 consensus_queue.cc:260] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.737287 32709 leader_election.cc:290] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
W20260709 13:11:19.756417 32689 tablet.cc:2406] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:11:19.757069 32514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98" is_pre_election: true
W20260709 13:11:19.760915 32629 leader_election.cc:343] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:19.770155 32709 raft_consensus.cc:493] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.773684 32709 raft_consensus.cc:515] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.778975 32708 raft_consensus.cc:493] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.779459 32708 raft_consensus.cc:515] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.779474 32514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98" is_pre_election: true
I20260709 13:11:19.786494 32709 leader_election.cc:290] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:19.786753 32514 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 0.
I20260709 13:11:19.797044 32629 leader_election.cc:304] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:19.794112 32708 leader_election.cc:290] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:19.806087 32709 raft_consensus.cc:2804] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:19.806569 32709 raft_consensus.cc:493] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:19.806983 32709 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:19.828106 32705 tablet_bootstrap.cc:492] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:19.827816 32588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:19.828673 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
W20260709 13:11:19.829674 32628 leader_election.cc:343] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:19.831004 32628 leader_election.cc:343] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:19.831583 32628 leader_election.cc:304] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d; no voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:19.833402 32705 ts_tablet_manager.cc:1403] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.122s	user 0.056s	sys 0.056s
I20260709 13:11:19.833720 32720 raft_consensus.cc:2749] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:19.837893 32705 raft_consensus.cc:359] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.839293 32705 raft_consensus.cc:385] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.841385 32705 raft_consensus.cc:740] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.843178 32705 consensus_queue.cc:260] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.844645 32703 tablet_bootstrap.cc:492] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:19.845224 32703 ts_tablet_manager.cc:1403] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.133s	user 0.031s	sys 0.040s
I20260709 13:11:19.845407 32705 ts_tablet_manager.cc:1434] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:11:19.845438 32704 ts_tablet_manager.cc:1434] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.128s	user 0.017s	sys 0.055s
I20260709 13:11:19.846184 32709 raft_consensus.cc:515] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.847355 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:19.848604 32705 tablet_bootstrap.cc:492] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:19.848968 32704 tablet_bootstrap.cc:492] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
W20260709 13:11:19.853730 32479 leader_election.cc:343] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:19.855792 32705 tablet_bootstrap.cc:654] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.856207 32704 tablet_bootstrap.cc:654] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.856333 32514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:19.857093 32514 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:19.866061 32709 leader_election.cc:290] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:19.867506 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:19.870594 32589 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:19.871714 32514 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:19.873302 32629 leader_election.cc:304] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:19.875244 32709 raft_consensus.cc:2804] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:19.876073 32709 raft_consensus.cc:697] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:19.877200 32709 consensus_queue.cc:237] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.881397 32708 raft_consensus.cc:493] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.881554 32703 raft_consensus.cc:359] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.882102 32708 raft_consensus.cc:515] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.885942 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:19.886045 32588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:19.886956 32588 raft_consensus.cc:2468] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
W20260709 13:11:19.888010 32478 leader_election.cc:343] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:19.888644 32479 leader_election.cc:304] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:19.888667 32478 leader_election.cc:304] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98; no voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:19.890556 32726 raft_consensus.cc:2804] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:11:19.890743 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:19.891034 32726 raft_consensus.cc:493] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:19.891425 32726 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:19.891609 32708 leader_election.cc:290] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:19.892243 32708 raft_consensus.cc:2749] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:19.882601 32703 raft_consensus.cc:385] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.893500 32703 raft_consensus.cc:740] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.894840 32589 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:19.897034 32703 consensus_queue.cc:260] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.899315 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:19.900381 32703 ts_tablet_manager.cc:1434] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
W20260709 13:11:19.900908 32478 leader_election.cc:343] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:19.901656 32703 tablet_bootstrap.cc:492] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:19.906468 32704 tablet_bootstrap.cc:492] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:19.906971 32704 ts_tablet_manager.cc:1403] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.058s	user 0.023s	sys 0.032s
I20260709 13:11:19.909828 32704 raft_consensus.cc:359] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.910502 32704 raft_consensus.cc:385] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.910739 32703 tablet_bootstrap.cc:654] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.910805 32704 raft_consensus.cc:740] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.911861 32704 consensus_queue.cc:260] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.914604 32704 ts_tablet_manager.cc:1434] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:11:19.915817 32704 tablet_bootstrap.cc:492] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
I20260709 13:11:19.918092 32705 tablet_bootstrap.cc:492] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:19.918638 32705 ts_tablet_manager.cc:1403] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.072s	user 0.027s	sys 0.016s
I20260709 13:11:19.920746 32726 raft_consensus.cc:515] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.921749 32705 raft_consensus.cc:359] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.922312 32704 tablet_bootstrap.cc:654] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.922500 32705 raft_consensus.cc:385] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.922827 32705 raft_consensus.cc:740] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.923427 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:19.924088 32589 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:19.925087 32718 raft_consensus.cc:493] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.924536 32412 catalog_manager.cc:5697] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 0 to 1, leader changed from <none> to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 1 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:19.925763 32718 raft_consensus.cc:515] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.929359 32588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3825b68a7235435bb84fa43308cfa0ad" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:19.931206 32514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3825b68a7235435bb84fa43308cfa0ad" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98" is_pre_election: true
W20260709 13:11:19.932749 32628 leader_election.cc:343] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:19.933439 32629 leader_election.cc:343] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:19.933820 32629 leader_election.cc:304] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d; no voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:19.934489 32718 leader_election.cc:290] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:19.935124 32709 raft_consensus.cc:2749] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:19.937341 32589 raft_consensus.cc:2468] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:19.942237 32479 leader_election.cc:304] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:19.943230 32703 tablet_bootstrap.cc:492] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:19.943780 32703 ts_tablet_manager.cc:1403] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.001s
I20260709 13:11:19.944615 32727 raft_consensus.cc:2804] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:19.945130 32727 raft_consensus.cc:697] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Becoming Leader. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Running, Role: LEADER
I20260709 13:11:19.945200 32704 tablet_bootstrap.cc:492] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:19.945825 32704 ts_tablet_manager.cc:1403] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.009s
I20260709 13:11:19.946040 32727 consensus_queue.cc:237] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.946650 32703 raft_consensus.cc:359] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.947405 32703 raft_consensus.cc:385] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.947530 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:19.947736 32703 raft_consensus.cc:740] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.948810 32726 leader_election.cc:290] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Requested vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:19.948496 32703 consensus_queue.cc:260] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.948635 32704 raft_consensus.cc:359] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.923616 32705 consensus_queue.cc:260] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
W20260709 13:11:19.949717 32478 leader_election.cc:343] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:19.952234 32703 ts_tablet_manager.cc:1434] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 13:11:19.952404 32705 ts_tablet_manager.cc:1434] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:11:19.953663 32705 tablet_bootstrap.cc:492] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:19.953684 32703 tablet_bootstrap.cc:492] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:19.949748 32704 raft_consensus.cc:385] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.959689 32704 raft_consensus.cc:740] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.962361 32704 consensus_queue.cc:260] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.965034 32704 ts_tablet_manager.cc:1434] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.003s
I20260709 13:11:19.966033 32704 tablet_bootstrap.cc:492] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
I20260709 13:11:19.968262 32703 tablet_bootstrap.cc:654] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.971123 32412 catalog_manager.cc:5697] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "7edad4013cb44461aad4b40ad080cd98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:19.972977 32705 tablet_bootstrap.cc:654] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.973943 32704 tablet_bootstrap.cc:654] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:19.981992 32703 tablet_bootstrap.cc:492] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:19.982457 32703 ts_tablet_manager.cc:1403] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.005s
I20260709 13:11:19.983801 32705 tablet_bootstrap.cc:492] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:19.984438 32705 ts_tablet_manager.cc:1403] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.010s
I20260709 13:11:19.985241 32703 raft_consensus.cc:359] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.986002 32703 raft_consensus.cc:385] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.986271 32703 raft_consensus.cc:740] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.987082 32703 consensus_queue.cc:260] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.989444 32703 ts_tablet_manager.cc:1434] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:19.990427 32703 tablet_bootstrap.cc:492] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:19.990895 32705 raft_consensus.cc:359] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.991662 32705 raft_consensus.cc:385] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.992007 32705 raft_consensus.cc:740] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:19.993846 32705 consensus_queue.cc:260] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.994956 32704 tablet_bootstrap.cc:492] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:19.995445 32704 ts_tablet_manager.cc:1403] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.011s
I20260709 13:11:19.996284 32705 ts_tablet_manager.cc:1434] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.005s
I20260709 13:11:19.997354 32705 tablet_bootstrap.cc:492] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:19.998436 32704 raft_consensus.cc:359] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:19.999179 32704 raft_consensus.cc:385] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:19.999482 32704 raft_consensus.cc:740] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.000468 32704 consensus_queue.cc:260] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.004609 32705 tablet_bootstrap.cc:654] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.005594 32703 tablet_bootstrap.cc:654] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.009043 32704 ts_tablet_manager.cc:1434] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.002s
I20260709 13:11:20.013999 32704 tablet_bootstrap.cc:492] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
I20260709 13:11:20.015497 32703 tablet_bootstrap.cc:492] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:20.016149 32703 ts_tablet_manager.cc:1403] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.003s
I20260709 13:11:20.018653 32704 tablet_bootstrap.cc:654] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.019960 32703 raft_consensus.cc:359] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.020941 32703 raft_consensus.cc:385] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.021292 32703 raft_consensus.cc:740] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.023656 32705 tablet_bootstrap.cc:492] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:20.023281 32703 consensus_queue.cc:260] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.024108 32705 ts_tablet_manager.cc:1403] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.012s
I20260709 13:11:20.026437 32703 ts_tablet_manager.cc:1434] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.004s
I20260709 13:11:20.026973 32705 raft_consensus.cc:359] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.027679 32705 raft_consensus.cc:385] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.027714 32703 tablet_bootstrap.cc:492] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:20.028087 32705 raft_consensus.cc:740] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.029116 32705 consensus_queue.cc:260] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.033936 32705 ts_tablet_manager.cc:1434] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:11:20.034901 32705 tablet_bootstrap.cc:492] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:20.036726 32703 tablet_bootstrap.cc:654] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.039811 32704 tablet_bootstrap.cc:492] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:20.040315 32704 ts_tablet_manager.cc:1403] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.015s
I20260709 13:11:20.041136 32705 tablet_bootstrap.cc:654] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.043773 32704 raft_consensus.cc:359] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.044597 32703 tablet_bootstrap.cc:492] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:20.044538 32704 raft_consensus.cc:385] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.045267 32703 ts_tablet_manager.cc:1403] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:11:20.045183 32704 raft_consensus.cc:740] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.046518 32704 consensus_queue.cc:260] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.047482 32727 raft_consensus.cc:493] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.047967 32727 raft_consensus.cc:515] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.050109 32727 leader_election.cc:290] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.050304 32704 ts_tablet_manager.cc:1434] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:11:20.050973 32705 tablet_bootstrap.cc:492] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:20.051319 32704 tablet_bootstrap.cc:492] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
I20260709 13:11:20.051453 32705 ts_tablet_manager.cc:1403] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:11:20.052798 32703 raft_consensus.cc:359] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.053536 32703 raft_consensus.cc:385] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.053851 32703 raft_consensus.cc:740] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.054427 32705 raft_consensus.cc:359] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.054774 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3825b68a7235435bb84fa43308cfa0ad" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:20.054633 32703 consensus_queue.cc:260] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.055166 32705 raft_consensus.cc:385] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.058516 32704 tablet_bootstrap.cc:654] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.061758 32703 ts_tablet_manager.cc:1434] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 13:11:20.055572 32705 raft_consensus.cc:740] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.063263 32703 tablet_bootstrap.cc:492] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:20.062956 32705 consensus_queue.cc:260] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.055528 32663 raft_consensus.cc:2468] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.066787 32478 leader_election.cc:304] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:20.067874 32727 raft_consensus.cc:2804] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.068349 32727 raft_consensus.cc:493] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.068763 32727 raft_consensus.cc:3060] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.069573 32705 ts_tablet_manager.cc:1434] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 13:11:20.070252 32703 tablet_bootstrap.cc:654] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.070583 32705 tablet_bootstrap.cc:492] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:20.084794 32705 tablet_bootstrap.cc:654] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.054440 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3825b68a7235435bb84fa43308cfa0ad" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:20.091096 32589 raft_consensus.cc:2468] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.104187 32727 raft_consensus.cc:515] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.104851 32726 raft_consensus.cc:493] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.105309 32705 tablet_bootstrap.cc:492] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:20.105839 32705 ts_tablet_manager.cc:1403] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.002s
I20260709 13:11:20.105459 32726 raft_consensus.cc:515] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.107559 32726 leader_election.cc:290] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.109052 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:20.109658 32589 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.108809 32705 raft_consensus.cc:359] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.110440 32705 raft_consensus.cc:385] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.110769 32705 raft_consensus.cc:740] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.111375 32727 leader_election.cc:290] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Requested vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.111394 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:20.111433 32705 consensus_queue.cc:260] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.112412 32663 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.114082 32662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3825b68a7235435bb84fa43308cfa0ad" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:20.114710 32662 raft_consensus.cc:3060] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.114795 32705 ts_tablet_manager.cc:1434] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:11:20.115829 32705 tablet_bootstrap.cc:492] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:20.117661 32479 leader_election.cc:304] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.119375 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3825b68a7235435bb84fa43308cfa0ad" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:20.119978 32589 raft_consensus.cc:3060] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.121644 32726 raft_consensus.cc:2804] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.122215 32726 raft_consensus.cc:493] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.129539 32726 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.122491 32705 tablet_bootstrap.cc:654] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.132743 32589 raft_consensus.cc:2468] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.133726 32704 tablet_bootstrap.cc:492] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:20.134366 32704 ts_tablet_manager.cc:1403] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.083s	user 0.031s	sys 0.051s
I20260709 13:11:20.134358 32479 leader_election.cc:304] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.135725 32727 raft_consensus.cc:2804] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:20.136210 32703 tablet_bootstrap.cc:492] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:20.136672 32703 ts_tablet_manager.cc:1403] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.028s
I20260709 13:11:20.137117 32704 raft_consensus.cc:359] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.137451 32727 raft_consensus.cc:697] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Becoming Leader. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Running, Role: LEADER
I20260709 13:11:20.137844 32704 raft_consensus.cc:385] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.137773 32726 raft_consensus.cc:515] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.138409 32704 raft_consensus.cc:740] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.139081 32727 consensus_queue.cc:237] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.139178 32703 raft_consensus.cc:359] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.139187 32704 consensus_queue.cc:260] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.140040 32703 raft_consensus.cc:385] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.140323 32703 raft_consensus.cc:740] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.140818 32588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:20.141219 32726 leader_election.cc:290] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Requested vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.141474 32588 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.143460 32705 tablet_bootstrap.cc:492] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:20.143960 32705 ts_tablet_manager.cc:1403] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.028s	user 0.005s	sys 0.007s
I20260709 13:11:20.144069 32704 ts_tablet_manager.cc:1434] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:11:20.145406 32704 tablet_bootstrap.cc:492] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
I20260709 13:11:20.145766 32709 raft_consensus.cc:493] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.146215 32709 raft_consensus.cc:515] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.147615 32705 raft_consensus.cc:359] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.148185 32709 leader_election.cc:290] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:20.149781 32588 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.150095 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:20.127260 32662 raft_consensus.cc:2468] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.140954 32703 consensus_queue.cc:260] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.150799 32587 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 0.
I20260709 13:11:20.148263 32705 raft_consensus.cc:385] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.151732 32705 raft_consensus.cc:740] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.152222 32628 leader_election.cc:304] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.152514 32703 ts_tablet_manager.cc:1434] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:11:20.150832 32704 tablet_bootstrap.cc:654] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.153235 32709 raft_consensus.cc:2804] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.154016 32703 tablet_bootstrap.cc:492] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:20.153990 32709 raft_consensus.cc:493] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.154665 32709 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.200650 32479 leader_election.cc:304] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.209017 32703 tablet_bootstrap.cc:654] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.152524 32705 consensus_queue.cc:260] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.214183 32705 ts_tablet_manager.cc:1434] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.070s	user 0.009s	sys 0.001s
I20260709 13:11:20.214555 32662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:20.215754 32662 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.216521 32705 tablet_bootstrap.cc:492] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:20.229005 32726 raft_consensus.cc:493] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.230078 32726 raft_consensus.cc:515] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.231840 32708 raft_consensus.cc:2804] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:20.233232 32726 leader_election.cc:290] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.234659 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:20.234966 32726 raft_consensus.cc:493] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.235729 32589 raft_consensus.cc:2468] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.235762 32726 raft_consensus.cc:515] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.236195 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:20.237295 32663 raft_consensus.cc:2468] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.237412 32479 leader_election.cc:304] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.238967 32726 leader_election.cc:290] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.239907 32726 raft_consensus.cc:2804] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.204288 32514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98" is_pre_election: true
I20260709 13:11:20.240231 32726 raft_consensus.cc:493] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.245571 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:20.250070 32726 raft_consensus.cc:3060] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.250690 32663 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.244865 32704 tablet_bootstrap.cc:492] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:20.258952 32727 raft_consensus.cc:493] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.259742 32704 ts_tablet_manager.cc:1403] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.115s	user 0.009s	sys 0.005s
I20260709 13:11:20.259763 32727 raft_consensus.cc:515] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.240661 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
W20260709 13:11:20.263902 32479 leader_election.cc:343] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.265131 32704 raft_consensus.cc:359] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.266224 32704 raft_consensus.cc:385] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.266601 32704 raft_consensus.cc:740] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.266995 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdd94ee0820431b9d14406fe6100bb9" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:20.267740 32589 raft_consensus.cc:2468] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.267503 32704 consensus_queue.cc:260] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.269241 32479 leader_election.cc:304] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.269693 32705 tablet_bootstrap.cc:654] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.270015 32704 ts_tablet_manager.cc:1434] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:11:20.270587 32478 leader_election.cc:304] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:20.241693 32514 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 0.
I20260709 13:11:20.271597 32704 tablet_bootstrap.cc:492] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d: Bootstrap starting.
I20260709 13:11:20.246585 32412 catalog_manager.cc:5697] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "7edad4013cb44461aad4b40ad080cd98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:20.258600 32662 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.268563 32709 raft_consensus.cc:515] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.272779 32727 leader_election.cc:290] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.284847 32727 raft_consensus.cc:2804] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.285346 32727 raft_consensus.cc:493] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.285772 32727 raft_consensus.cc:3060] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.288770 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdd94ee0820431b9d14406fe6100bb9" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:20.294147 32708 raft_consensus.cc:697] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Becoming Leader. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Running, Role: LEADER
I20260709 13:11:20.295424 32705 tablet_bootstrap.cc:492] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:20.296094 32705 ts_tablet_manager.cc:1403] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.002s
I20260709 13:11:20.295518 32708 consensus_queue.cc:237] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
W20260709 13:11:20.296646 32478 leader_election.cc:343] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.299631 32726 raft_consensus.cc:515] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.301584 32704 tablet_bootstrap.cc:654] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.301677 32718 raft_consensus.cc:493] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.302512 32726 leader_election.cc:290] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Requested vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.302356 32718 raft_consensus.cc:515] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.303165 32726 raft_consensus.cc:493] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.303742 32726 raft_consensus.cc:515] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.305913 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:20.306479 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:20.307267 32589 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.308794 32703 tablet_bootstrap.cc:492] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: No bootstrap required, opened a new log
I20260709 13:11:20.309455 32703 ts_tablet_manager.cc:1403] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.156s	user 0.015s	sys 0.024s
I20260709 13:11:20.309721 32514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:20.310424 32514 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.310931 32513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3391c7fa5f274de5a49d3d43752d262b" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98" is_pre_election: true
W20260709 13:11:20.312673 32629 leader_election.cc:343] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.311117 32718 leader_election.cc:290] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:20.314628 32709 leader_election.cc:290] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:20.314885 32588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3391c7fa5f274de5a49d3d43752d262b" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:20.315876 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:20.316205 32662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:20.316717 32587 raft_consensus.cc:3060] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.317085 32662 raft_consensus.cc:2468] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.312980 32703 raft_consensus.cc:359] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.318621 32703 raft_consensus.cc:385] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.318889 32478 leader_election.cc:304] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:20.314886 32727 raft_consensus.cc:515] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.318909 32703 raft_consensus.cc:740] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.304379 32753 raft_consensus.cc:2804] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.325615 32753 raft_consensus.cc:493] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.326079 32753 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.334189 32586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:20.306510 32663 raft_consensus.cc:3060] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.314224 32726 leader_election.cc:290] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.339164 32752 raft_consensus.cc:2804] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.339762 32752 raft_consensus.cc:493] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.340195 32752 raft_consensus.cc:3060] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.346643 32585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdd94ee0820431b9d14406fe6100bb9" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:20.347543 32585 raft_consensus.cc:3060] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.336684 32586 raft_consensus.cc:2468] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 0.
I20260709 13:11:20.352911 32752 raft_consensus.cc:515] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.357510 32586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:20.358335 32586 raft_consensus.cc:3060] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.365649 32752 leader_election.cc:290] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Requested vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.367506 32662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:20.368456 32662 raft_consensus.cc:3060] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.377913 32727 leader_election.cc:290] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Requested vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.380278 32661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdd94ee0820431b9d14406fe6100bb9" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
W20260709 13:11:20.382691 32478 leader_election.cc:343] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:20.385855 32628 leader_election.cc:343] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.386487 32628 leader_election.cc:304] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d; no voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:20.387787 32587 raft_consensus.cc:2468] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.388095 32709 raft_consensus.cc:2749] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:20.384842 32662 raft_consensus.cc:2468] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.396199 32703 consensus_queue.cc:260] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.396878 32479 leader_election.cc:304] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.403806 32726 raft_consensus.cc:2804] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:20.402249 32705 raft_consensus.cc:359] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.404959 32705 raft_consensus.cc:385] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.405458 32705 raft_consensus.cc:740] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.406320 32705 consensus_queue.cc:260] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.403205 32478 leader_election.cc:304] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:20.414052 32709 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:20.415884 32708 raft_consensus.cc:2804] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:20.416680 32708 raft_consensus.cc:697] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Becoming Leader. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Running, Role: LEADER
I20260709 13:11:20.419935 32704 tablet_bootstrap.cc:492] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d: No bootstrap required, opened a new log
I20260709 13:11:20.422317 32704 ts_tablet_manager.cc:1403] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d: Time spent bootstrapping tablet: real 0.151s	user 0.046s	sys 0.047s
I20260709 13:11:20.418509 32708 consensus_queue.cc:237] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.423219 32709 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:20.418514 32412 catalog_manager.cc:5697] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "7edad4013cb44461aad4b40ad080cd98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:20.423599 32663 raft_consensus.cc:2468] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.425403 32704 raft_consensus.cc:359] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.426102 32704 raft_consensus.cc:385] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.426357 32704 raft_consensus.cc:740] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.426769 32662 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.427067 32704 consensus_queue.cc:260] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.408452 32586 raft_consensus.cc:2468] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.431746 32704 ts_tablet_manager.cc:1434] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:11:20.421017 32726 raft_consensus.cc:697] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Becoming Leader. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Running, Role: LEADER
I20260709 13:11:20.434873 32514 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:20.435799 32589 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:20.437732 32703 ts_tablet_manager.cc:1434] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.128s	user 0.005s	sys 0.004s
I20260709 13:11:20.434921 32726 consensus_queue.cc:237] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.439446 32753 raft_consensus.cc:515] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.439644 32628 leader_election.cc:304] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.444188 32709 raft_consensus.cc:2804] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:20.444933 32709 raft_consensus.cc:697] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:20.447861 32709 consensus_queue.cc:237] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.449297 32708 consensus_queue.cc:1048] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:20.450353 32718 raft_consensus.cc:493] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.451023 32718 raft_consensus.cc:515] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.454851 32718 leader_election.cc:290] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:20.464360 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:20.465040 32585 raft_consensus.cc:2468] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.465250 32589 raft_consensus.cc:2468] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:20.466559 32479 leader_election.cc:304] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:20.467661 32708 raft_consensus.cc:2804] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:20.468140 32708 raft_consensus.cc:697] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Becoming Leader. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Running, Role: LEADER
I20260709 13:11:20.469333 32514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98" is_pre_election: true
I20260709 13:11:20.470041 32628 leader_election.cc:304] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.469094 32708 consensus_queue.cc:237] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.472136 32705 ts_tablet_manager.cc:1434] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.176s	user 0.011s	sys 0.001s
I20260709 13:11:20.473634 32705 tablet_bootstrap.cc:492] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015: Bootstrap starting.
I20260709 13:11:20.489698 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
W20260709 13:11:20.491577 32479 leader_election.cc:343] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:20.493377 32705 tablet_bootstrap.cc:654] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:20.500063 32753 leader_election.cc:290] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Requested vote from peers c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.506347 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "7edad4013cb44461aad4b40ad080cd98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:20.504805 32718 raft_consensus.cc:2804] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:20.511536 32663 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.514179 32718 raft_consensus.cc:493] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7edad4013cb44461aad4b40ad080cd98)
I20260709 13:11:20.514676 32718 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.517571 32708 consensus_queue.cc:1048] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:20.499606 32411 catalog_manager.cc:5697] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "7edad4013cb44461aad4b40ad080cd98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:20.519596 32411 catalog_manager.cc:5697] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "7edad4013cb44461aad4b40ad080cd98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:20.529007 32718 raft_consensus.cc:515] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.530997 32718 leader_election.cc:290] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:20.532023 32514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:20.532925 32663 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edad4013cb44461aad4b40ad080cd98 in term 1.
I20260709 13:11:20.533774 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:20.535705 32628 leader_election.cc:304] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d; no voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:20.536218 32478 leader_election.cc:304] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:20.537863 32753 raft_consensus.cc:2804] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:20.538406 32753 raft_consensus.cc:697] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Becoming Leader. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Running, Role: LEADER
I20260709 13:11:20.539738 32663 raft_consensus.cc:1240] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Rejecting Update request from peer 7edad4013cb44461aad4b40ad080cd98 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:20.543418 32708 consensus_queue.cc:1059] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:20.544606 32708 raft_consensus.cc:3055] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:20.544917 32708 raft_consensus.cc:740] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Running, Role: LEADER
I20260709 13:11:20.546196 32410 catalog_manager.cc:5697] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 0 to 1, leader changed from <none> to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 1 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:20.551183 32767 raft_consensus.cc:2749] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:20.550752 32718 raft_consensus.cc:493] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.552018 32718 raft_consensus.cc:515] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.554536 32718 leader_election.cc:290] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:20.546005 32708 consensus_queue.cc:260] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.555703 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdd94ee0820431b9d14406fe6100bb9" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:20.556308 32708 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.556501 32587 raft_consensus.cc:2393] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b50e45fe51014825a6cb95a1dac5603d in current term 1: Already voted for candidate 7edad4013cb44461aad4b40ad080cd98 in this term.
I20260709 13:11:20.557916 32513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdd94ee0820431b9d14406fe6100bb9" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98" is_pre_election: true
I20260709 13:11:20.560734 32628 leader_election.cc:304] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d; no voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:20.539639 32753 consensus_queue.cc:237] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.561836 32718 raft_consensus.cc:3060] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.574797 32718 raft_consensus.cc:2749] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:20.581017 32705 tablet_bootstrap.cc:492] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015: No bootstrap required, opened a new log
I20260709 13:11:20.581575 32705 ts_tablet_manager.cc:1403] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015: Time spent bootstrapping tablet: real 0.108s	user 0.009s	sys 0.018s
I20260709 13:11:20.582145 32410 catalog_manager.cc:5697] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "7edad4013cb44461aad4b40ad080cd98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:20.583541 32410 catalog_manager.cc:5697] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "7edad4013cb44461aad4b40ad080cd98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:20.584120 32705 raft_consensus.cc:359] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.584689 32705 raft_consensus.cc:385] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:20.584949 32705 raft_consensus.cc:740] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Initialized, Role: FOLLOWER
I20260709 13:11:20.585680 32705 consensus_queue.cc:260] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.587368 32705 ts_tablet_manager.cc:1434] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:20.677052 32714 raft_consensus.cc:493] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.677676 32714 raft_consensus.cc:515] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.680378 32714 leader_election.cc:290] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.694805 32513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3391c7fa5f274de5a49d3d43752d262b" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98" is_pre_election: true
I20260709 13:11:20.694865 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3391c7fa5f274de5a49d3d43752d262b" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:20.695483 32513 raft_consensus.cc:2468] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 0.
I20260709 13:11:20.695695 32663 raft_consensus.cc:2468] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 0.
I20260709 13:11:20.696615 32555 leader_election.cc:304] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.697463 32714 raft_consensus.cc:2804] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:20.697744 32714 raft_consensus.cc:493] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.698020 32714 raft_consensus.cc:3060] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.703189 32714 raft_consensus.cc:515] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.705170 32714 leader_election.cc:290] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 1 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:20.705796 32513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3391c7fa5f274de5a49d3d43752d262b" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:20.706115 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3391c7fa5f274de5a49d3d43752d262b" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:20.706408 32513 raft_consensus.cc:3060] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.706670 32663 raft_consensus.cc:3060] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:20.713546 32513 raft_consensus.cc:2468] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 1.
I20260709 13:11:20.713546 32663 raft_consensus.cc:2468] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 1.
I20260709 13:11:20.714820 32555 leader_election.cc:304] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:20.715577 32714 raft_consensus.cc:2804] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:20.716843 32714 raft_consensus.cc:697] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 1 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:20.717736 32714 consensus_queue.cc:237] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:20.725080 32412 catalog_manager.cc:5697] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 0 to 1, leader changed from <none> to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:20.740736 32753 consensus_queue.cc:1048] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:20.741459 32708 consensus_queue.cc:1048] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:20.747771 32708 consensus_queue.cc:1048] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:20.766681 32753 consensus_queue.cc:1048] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:20.803736 21629 ts_manager.cc:295] Set tserver state for 7edad4013cb44461aad4b40ad080cd98 to MAINTENANCE_MODE
I20260709 13:11:20.804533 21629 tablet_server.cc:179] TabletServer@127.21.31.65:0 shutting down...
I20260709 13:11:20.831483 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:20.832351 21629 tablet_replica.cc:333] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.833403 21629 raft_consensus.cc:2243] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.834043 21629 pending_rounds.cc:70] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Trying to abort 1 pending ops.
I20260709 13:11:20.834256 21629 pending_rounds.cc:77] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305636579030822912 op_type: NO_OP noop_request { }
I20260709 13:11:20.834539 21629 raft_consensus.cc:2889] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:20.834826 21629 raft_consensus.cc:2272] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.837550 21629 tablet_replica.cc:333] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.838377 21629 raft_consensus.cc:2243] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.839165 21629 pending_rounds.cc:70] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Trying to abort 1 pending ops.
I20260709 13:11:20.839372 21629 pending_rounds.cc:77] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305636579217022976 op_type: NO_OP noop_request { }
I20260709 13:11:20.839659 21629 raft_consensus.cc:2889] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:20.839929 21629 raft_consensus.cc:2272] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.842573 21629 tablet_replica.cc:333] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.843230 21629 raft_consensus.cc:2243] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.843799 21629 pending_rounds.cc:70] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Trying to abort 1 pending ops.
I20260709 13:11:20.844007 21629 pending_rounds.cc:77] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305636579588530176 op_type: NO_OP noop_request { }
I20260709 13:11:20.844264 21629 raft_consensus.cc:2889] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:20.844521 21629 raft_consensus.cc:2272] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.846742 21629 tablet_replica.cc:333] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.847278 21629 raft_consensus.cc:2243] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.847878 21629 pending_rounds.cc:70] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Trying to abort 1 pending ops.
I20260709 13:11:20.848095 21629 pending_rounds.cc:77] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305636579113521152 op_type: NO_OP noop_request { }
I20260709 13:11:20.848338 21629 raft_consensus.cc:2889] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:20.848589 21629 raft_consensus.cc:2272] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.850646 21629 tablet_replica.cc:333] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.851141 21629 raft_consensus.cc:2243] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.851855 21629 raft_consensus.cc:2272] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.854132 21629 tablet_replica.cc:333] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.854661 21629 raft_consensus.cc:2243] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.855371 21629 raft_consensus.cc:2272] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.857770 21629 tablet_replica.cc:333] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.858330 21629 raft_consensus.cc:2243] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.858729 21629 raft_consensus.cc:2272] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.861071 21629 tablet_replica.cc:333] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.861601 21629 raft_consensus.cc:2243] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.862022 21629 raft_consensus.cc:2272] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.864305 21629 tablet_replica.cc:333] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.864771 21629 raft_consensus.cc:2243] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.865144 21629 raft_consensus.cc:2272] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.866978 21629 tablet_replica.cc:333] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: stopping tablet replica
I20260709 13:11:20.867434 21629 raft_consensus.cc:2243] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.867794 21629 raft_consensus.cc:2272] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:20.891988 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:20.899463 21629 tablet_server.cc:196] TabletServer@127.21.31.65:0 shutdown complete.
I20260709 13:11:20.900116 32718 consensus_queue.cc:1048] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:20.922359 32629 consensus_peers.cc:597] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:21.055811 32714 raft_consensus.cc:493] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.056326 32714 raft_consensus.cc:515] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:21.058658 32714 leader_election.cc:290] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:21.059928 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:21.060775 32663 raft_consensus.cc:2393] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c372ed6a9e6546f591cd1c6ab584f015 in current term 1: Already voted for candidate 7edad4013cb44461aad4b40ad080cd98 in this term.
W20260709 13:11:21.063031 32555 leader_election.cc:336] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:21.063412 32555 leader_election.cc:304] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:21.064013 32714 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:21.069729 32714 raft_consensus.cc:2749] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:21.122217 32629 consensus_peers.cc:597] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:21.265411 32555 consensus_peers.cc:597] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:21.269814 32714 consensus_queue.cc:1048] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:21.893061 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:22.021683   319 raft_consensus.cc:493] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7edad4013cb44461aad4b40ad080cd98)
I20260709 13:11:22.022277   319 raft_consensus.cc:515] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.023911   319 leader_election.cc:290] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:22.025213 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:22.026036 32663 raft_consensus.cc:2393] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c372ed6a9e6546f591cd1c6ab584f015 in current term 2: Already voted for candidate b50e45fe51014825a6cb95a1dac5603d in this term.
W20260709 13:11:22.028326 32555 leader_election.cc:336] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.028821 32555 leader_election.cc:304] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:22.029613   319 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.035437   319 raft_consensus.cc:2749] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:22.202139   321 raft_consensus.cc:493] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.202639   321 raft_consensus.cc:515] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.204206   321 leader_election.cc:290] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.205268 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:22.206009 32587 raft_consensus.cc:2468] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:22.207195 32628 leader_election.cc:304] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:22.207890   321 raft_consensus.cc:2804] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:22.208124 32629 leader_election.cc:336] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.208226   321 raft_consensus.cc:493] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.208738   321 raft_consensus.cc:3060] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.215190   321 raft_consensus.cc:515] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.217253   321 leader_election.cc:290] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.219026 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:22.219612 32587 raft_consensus.cc:3060] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:22.220685 32629 leader_election.cc:336] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.224996 32587 raft_consensus.cc:2468] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 2.
I20260709 13:11:22.226114 32628 leader_election.cc:304] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98
I20260709 13:11:22.226866   321 raft_consensus.cc:2804] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:22.227990   321 raft_consensus.cc:697] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:22.229035   321 consensus_queue.cc:237] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.236735 32412 catalog_manager.cc:5697] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 1 to 2, leader changed from 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 2 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:22.252883   319 raft_consensus.cc:493] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7edad4013cb44461aad4b40ad080cd98)
I20260709 13:11:22.253396   319 raft_consensus.cc:515] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.254941   319 leader_election.cc:290] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:22.256044 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
W20260709 13:11:22.256250 32555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111) [suppressed 102 similar messages]
I20260709 13:11:22.257267   319 raft_consensus.cc:493] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7edad4013cb44461aad4b40ad080cd98)
I20260709 13:11:22.257848   319 raft_consensus.cc:515] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
W20260709 13:11:22.259006 32555 leader_election.cc:336] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.259403 32555 leader_election.cc:304] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:22.259609   319 leader_election.cc:290] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:22.260242   319 raft_consensus.cc:2749] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:22.260823 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3825b68a7235435bb84fa43308cfa0ad" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
W20260709 13:11:22.264823 32555 leader_election.cc:336] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.265175 32555 leader_election.cc:304] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:22.265524   321 raft_consensus.cc:493] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7edad4013cb44461aad4b40ad080cd98)
I20260709 13:11:22.265938   328 raft_consensus.cc:2749] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:22.266069   321 raft_consensus.cc:515] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.268038   321 leader_election.cc:290] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.269143 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3825b68a7235435bb84fa43308cfa0ad" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:22.269804 32587 raft_consensus.cc:2468] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:22.270561   321 raft_consensus.cc:493] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7edad4013cb44461aad4b40ad080cd98)
I20260709 13:11:22.270995   321 raft_consensus.cc:515] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
W20260709 13:11:22.271309 32629 leader_election.cc:336] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.272754   321 leader_election.cc:290] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.273672 32628 leader_election.cc:304] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98
I20260709 13:11:22.274281 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:22.274942 32587 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:22.274950   321 raft_consensus.cc:2804] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:22.276098 32628 leader_election.cc:304] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:22.276247   321 raft_consensus.cc:493] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7edad4013cb44461aad4b40ad080cd98)
I20260709 13:11:22.276750   321 raft_consensus.cc:3060] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:22.276758 32629 leader_election.cc:336] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.276995   322 raft_consensus.cc:2804] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:22.277513   322 raft_consensus.cc:493] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7edad4013cb44461aad4b40ad080cd98)
I20260709 13:11:22.277877   322 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.283037   321 raft_consensus.cc:515] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.283238   322 raft_consensus.cc:515] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.285840   322 leader_election.cc:290] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.286294   321 leader_election.cc:290] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.286536 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3825b68a7235435bb84fa43308cfa0ad" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:22.287151 32587 raft_consensus.cc:3060] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:22.291461 32629 leader_election.cc:336] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
W20260709 13:11:22.291908 32629 leader_election.cc:336] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.292328 32587 raft_consensus.cc:2468] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 2.
I20260709 13:11:22.293543 32628 leader_election.cc:304] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98
I20260709 13:11:22.294039 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:22.294344   329 raft_consensus.cc:2804] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:22.294674 32587 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.295183   329 raft_consensus.cc:697] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:22.296101   329 consensus_queue.cc:237] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.300688 32587 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 2.
I20260709 13:11:22.301751 32628 leader_election.cc:304] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98
I20260709 13:11:22.302508   329 raft_consensus.cc:2804] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:22.302977   329 raft_consensus.cc:697] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:22.303797   329 consensus_queue.cc:237] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.303843 32412 catalog_manager.cc:5697] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 1 to 2, leader changed from 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 2 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:22.315725 32412 catalog_manager.cc:5697] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 1 to 2, leader changed from 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 2 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:22.347234   328 raft_consensus.cc:493] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.347714   328 raft_consensus.cc:515] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.349376   328 leader_election.cc:290] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:22.350368 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d" is_pre_election: true
I20260709 13:11:22.351012 32663 raft_consensus.cc:2468] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 1.
I20260709 13:11:22.352005 32553 leader_election.cc:304] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:22.352857   328 raft_consensus.cc:2804] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:22.353355   328 raft_consensus.cc:493] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.353812   328 raft_consensus.cc:3060] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:22.353967 32555 leader_election.cc:336] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.358938   328 raft_consensus.cc:515] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.360448   328 leader_election.cc:290] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:22.361462 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:22.362020 32663 raft_consensus.cc:3060] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:22.363682 32555 leader_election.cc:336] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.367146 32663 raft_consensus.cc:2468] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 2.
I20260709 13:11:22.368026 32553 leader_election.cc:304] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98
I20260709 13:11:22.368763   328 raft_consensus.cc:2804] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:22.369252   328 raft_consensus.cc:697] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:22.370069   328 consensus_queue.cc:237] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.522145 32412 catalog_manager.cc:5697] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 1 to 2, leader changed from 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:22.593935   329 raft_consensus.cc:493] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.594578   329 raft_consensus.cc:515] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.596431   329 leader_election.cc:290] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.597622 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdd94ee0820431b9d14406fe6100bb9" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:22.598295 32587 raft_consensus.cc:2468] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:22.599241 32628 leader_election.cc:304] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:22.600052   329 raft_consensus.cc:2804] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:22.600452 32629 leader_election.cc:336] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.600481   329 raft_consensus.cc:493] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.600979   329 raft_consensus.cc:3060] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.606027   329 raft_consensus.cc:515] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.607903   329 leader_election.cc:290] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.608960 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdd94ee0820431b9d14406fe6100bb9" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:22.609483 32587 raft_consensus.cc:3060] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:22.610913 32629 leader_election.cc:336] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.614151 32587 raft_consensus.cc:2468] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 2.
I20260709 13:11:22.615110 32628 leader_election.cc:304] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98
I20260709 13:11:22.615839   329 raft_consensus.cc:2804] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:22.616308   329 raft_consensus.cc:697] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:22.617079   329 consensus_queue.cc:237] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.626019 32412 catalog_manager.cc:5697] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 1 to 2, leader changed from 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 2 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:22.630174   329 raft_consensus.cc:493] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.630671   329 raft_consensus.cc:515] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.633755   329 leader_election.cc:290] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.633870 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:22.634600 32587 raft_consensus.cc:2468] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 2.
I20260709 13:11:22.635824 32628 leader_election.cc:304] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:22.636582   329 raft_consensus.cc:2804] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:22.636723 32629 leader_election.cc:336] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.637034   329 raft_consensus.cc:493] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.637420   329 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:22.637955 32587 raft_consensus.cc:1275] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:11:22.639434   322 consensus_queue.cc:1048] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:22.644021   329 raft_consensus.cc:515] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:22.646242   329 leader_election.cc:290] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:22.647559 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:22.648219 32587 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:22.652411 32629 leader_election.cc:336] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:22.655634 32587 raft_consensus.cc:2468] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 3.
I20260709 13:11:22.657188 32628 leader_election.cc:304] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98
I20260709 13:11:22.658247   322 raft_consensus.cc:2804] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:22.658725   322 raft_consensus.cc:697] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:22.659677   322 consensus_queue.cc:237] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
W20260709 13:11:22.661545 32629 consensus_peers.cc:597] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:22.667079 32412 catalog_manager.cc:5697] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 1 to 3, leader changed from 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 3 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:22.690289 32587 raft_consensus.cc:1275] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.691731   322 consensus_queue.cc:1048] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:22.705096 32629 consensus_peers.cc:597] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:22.794265 32587 raft_consensus.cc:1275] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:22.796048   329 consensus_queue.cc:1048] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:22.809895 32629 consensus_peers.cc:597] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:22.815148 32555 consensus_peers.cc:597] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:22.894361 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:22.938426 32663 raft_consensus.cc:1275] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:11:22.939945   328 consensus_queue.cc:1048] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:23.052023 32629 consensus_peers.cc:597] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:23.142058   322 raft_consensus.cc:493] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.142561   322 raft_consensus.cc:515] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:23.144263   322 leader_election.cc:290] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:23.145241 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015" is_pre_election: true
I20260709 13:11:23.145957 32587 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 1.
I20260709 13:11:23.146864 32628 leader_election.cc:304] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
W20260709 13:11:23.147459 32629 leader_election.cc:336] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:23.147501   322 raft_consensus.cc:2804] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:23.147946   322 raft_consensus.cc:493] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.148206   322 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:23.153002   322 raft_consensus.cc:515] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:23.154549   322 leader_election.cc:290] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:23.155447 32587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:23.155939 32587 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:23.158205 32629 leader_election.cc:336] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111)
I20260709 13:11:23.160482 32587 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 2.
I20260709 13:11:23.161523 32628 leader_election.cc:304] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 7edad4013cb44461aad4b40ad080cd98
I20260709 13:11:23.162084   322 raft_consensus.cc:2804] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:23.162472   322 raft_consensus.cc:697] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:23.163136   322 consensus_queue.cc:237] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:23.170594 32411 catalog_manager.cc:5697] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 1 to 2, leader changed from 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 2 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:23.220990 32587 raft_consensus.cc:1275] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:11:23.222185   322 consensus_queue.cc:1048] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:23.234901 32629 consensus_peers.cc:597] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:23.241274 32587 raft_consensus.cc:1275] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:23.243796   322 consensus_queue.cc:1048] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:23.416280 32629 consensus_peers.cc:597] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.477002 32629 consensus_peers.cc:597] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.480365 32610 debug-util.cc:398] Leaking SignalData structure 0x7b080009fda0 after lost signal to thread 21632
W20260709 13:11:23.481391 32610 debug-util.cc:398] Leaking SignalData structure 0x7b080021b6a0 after lost signal to thread 32446
W20260709 13:11:23.482414 32610 debug-util.cc:398] Leaking SignalData structure 0x7b0800138e20 after lost signal to thread 32613
W20260709 13:11:23.483318 32610 debug-util.cc:398] Leaking SignalData structure 0x7b080004c820 after lost signal to thread 32687
W20260709 13:11:23.658439 32629 consensus_peers.cc:597] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:23.742662 32555 consensus_peers.cc:597] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:23.765913 32587 raft_consensus.cc:1275] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:11:23.767459   322 consensus_queue.cc:1048] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:23.895519 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:11:24.896386 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:11:25.164609 32629 consensus_peers.cc:597] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.172312 32629 consensus_peers.cc:597] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.390872 32555 consensus_peers.cc:597] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.415752 32629 consensus_peers.cc:597] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.638644 32629 consensus_peers.cc:597] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.801734 32629 consensus_peers.cc:597] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.843549 32629 consensus_peers.cc:597] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.897460 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:25.968299 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:25.975327   357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:25.975919   356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:25.980651 21629 server_base.cc:1061] running on GCE node
W20260709 13:11:25.981637   359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:25.983027 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:25.983340 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:25.983582 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602685983562 us; error 0 us; skew 500 ppm
I20260709 13:11:25.984365 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:25.987792 21629 webserver.cc:533] Webserver started at http://127.21.31.65:43653/ using document root <none> and password file <none>
I20260709 13:11:25.988468 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:25.988765 21629 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:11:25.994083 21629 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:11:25.999364   364 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:26.000382 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:11:26.000690 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "7edad4013cb44461aad4b40ad080cd98"
format_stamp: "Formatted at 2026-07-09 13:11:18 on dist-test-slave-9s6w"
I20260709 13:11:26.001207 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:11:26.037638 32629 consensus_peers.cc:597] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.038144 32629 consensus_peers.cc:597] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:11:26.051450 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:26.052819 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:26.054572 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:26.058935   372 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:11:26.107520 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:11:26.107829 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.050s	user 0.003s	sys 0.000s
I20260709 13:11:26.108188 21629 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:11:26.113111   372 tablet_bootstrap.cc:492] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.126880   372 tablet_bootstrap.cc:492] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:26.127779   372 tablet_bootstrap.cc:492] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.128799   372 ts_tablet_manager.cc:1403] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:11:26.132548   372 raft_consensus.cc:359] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.133390   372 raft_consensus.cc:740] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.134202   372 consensus_queue.cc:260] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.137292   372 ts_tablet_manager.cc:1434] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:11:26.138154   372 tablet_bootstrap.cc:492] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.154492   372 tablet_bootstrap.cc:492] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:26.155406   372 tablet_bootstrap.cc:492] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.156041   372 ts_tablet_manager.cc:1403] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.018s	user 0.002s	sys 0.013s
I20260709 13:11:26.158712   372 raft_consensus.cc:359] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.159379   372 raft_consensus.cc:740] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.160118   372 consensus_queue.cc:260] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.162107   372 ts_tablet_manager.cc:1434] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:11:26.162865   372 tablet_bootstrap.cc:492] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.164853 21629 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:11:26.165202 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.057s	user 0.053s	sys 0.000s
I20260709 13:11:26.183076   372 tablet_bootstrap.cc:492] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:26.183998   372 tablet_bootstrap.cc:492] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.184638   372 ts_tablet_manager.cc:1403] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.001s
I20260709 13:11:26.187225   372 raft_consensus.cc:359] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.187929   372 raft_consensus.cc:740] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.188854   372 consensus_queue.cc:260] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
W20260709 13:11:26.193274 32555 consensus_peers.cc:597] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:11:26.193645   372 ts_tablet_manager.cc:1434] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:11:26.194356   372 tablet_bootstrap.cc:492] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.208442   372 tablet_bootstrap.cc:492] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:26.209673   372 tablet_bootstrap.cc:492] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.210387   372 ts_tablet_manager.cc:1403] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:11:26.213403   372 raft_consensus.cc:359] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.214254   372 raft_consensus.cc:740] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.215237   372 consensus_queue.cc:260] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.217669   372 ts_tablet_manager.cc:1434] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:26.218585   372 tablet_bootstrap.cc:492] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.237867   372 tablet_bootstrap.cc:492] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:11:26.238973   372 tablet_bootstrap.cc:492] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.239642   372 ts_tablet_manager.cc:1403] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.007s
I20260709 13:11:26.241510   372 raft_consensus.cc:359] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.242136   372 raft_consensus.cc:740] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.242988   372 consensus_queue.cc:260] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.245272   372 ts_tablet_manager.cc:1434] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:26.246172   372 tablet_bootstrap.cc:492] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.253886 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:38411
I20260709 13:11:26.254009   434 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:38411 every 8 connection(s)
I20260709 13:11:26.258010   372 tablet_bootstrap.cc:492] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:26.258882   372 tablet_bootstrap.cc:492] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.259510   372 ts_tablet_manager.cc:1403] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:11:26.261205 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:26.268702   372 raft_consensus.cc:359] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.272692   372 raft_consensus.cc:740] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.274140   372 consensus_queue.cc:260] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.280432   372 ts_tablet_manager.cc:1434] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:11:26.281486   372 tablet_bootstrap.cc:492] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.306752 32663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
mode: GRACEFUL
new_leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:47880
I20260709 13:11:26.307662 32663 raft_consensus.cc:606] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Received request to transfer leadership to TS c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:26.310468   372 tablet_bootstrap.cc:492] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:11:26.311630   372 tablet_bootstrap.cc:492] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.311494 32663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
mode: GRACEFUL
new_leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:47880
I20260709 13:11:26.312163 32663 raft_consensus.cc:606] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Received request to transfer leadership to TS c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:26.312441   372 ts_tablet_manager.cc:1403] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.008s
I20260709 13:11:26.315901 32663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
mode: GRACEFUL
new_leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:47880
I20260709 13:11:26.315694   372 raft_consensus.cc:359] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.316336   436 heartbeater.cc:344] Connected to a master server at 127.21.31.126:40843
I20260709 13:11:26.316704 32663 raft_consensus.cc:606] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Received request to transfer leadership to TS c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:26.316881   372 raft_consensus.cc:740] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.316969   436 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:26.318308   436 heartbeater.cc:507] Master 127.21.31.126:40843 requested a full tablet report, sending...
I20260709 13:11:26.318018   372 consensus_queue.cc:260] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.320269 32663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
mode: GRACEFUL
new_leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:47880
I20260709 13:11:26.320587   372 ts_tablet_manager.cc:1434] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:11:26.320890 32663 raft_consensus.cc:606] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Received request to transfer leadership to TS c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:26.321429   372 tablet_bootstrap.cc:492] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.322327 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:11:26.322981 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:26.323659 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:26.327178 32411 ts_manager.cc:194] Re-registered known tserver with Master: 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411)
I20260709 13:11:26.340819 32411 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33108
I20260709 13:11:26.344455   436 heartbeater.cc:499] Master 127.21.31.126:40843 was elected leader, sending a full tablet report...
I20260709 13:11:26.363855   372 tablet_bootstrap.cc:492] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:26.364956   372 tablet_bootstrap.cc:492] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.365065   410 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:26.365762   372 ts_tablet_manager.cc:1403] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.045s	user 0.026s	sys 0.013s
I20260709 13:11:26.367776   409 raft_consensus.cc:3060] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.368541   372 raft_consensus.cc:359] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.369199   372 raft_consensus.cc:740] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.369984   372 consensus_queue.cc:260] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.372357   355 raft_consensus.cc:993] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d: : Instructing follower c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:26.372864   369 raft_consensus.cc:1081] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Signalling peer c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:26.374377   372 ts_tablet_manager.cc:1434] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:11:26.374570 32587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:42544
I20260709 13:11:26.375118 32587 raft_consensus.cc:493] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.375239   372 tablet_bootstrap.cc:492] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.375496 32587 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.376293   409 pending_rounds.cc:85] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Aborting all ops after (but not including) 0
I20260709 13:11:26.376614   409 pending_rounds.cc:107] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:11:26.376885   409 raft_consensus.cc:2889] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:26.393347 32587 raft_consensus.cc:515] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.399003   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:26.399827   407 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.407616   369 raft_consensus.cc:993] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d: : Instructing follower c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:26.408272   369 raft_consensus.cc:1081] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Signalling peer c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:26.411000 32585 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:42544
I20260709 13:11:26.411736 32585 raft_consensus.cc:493] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.412175 32585 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.424988   406 raft_consensus.cc:3060] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.433185   355 raft_consensus.cc:993] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d: : Instructing follower c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:26.433902   355 raft_consensus.cc:1081] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Signalling peer c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:26.437749 32589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:42544
I20260709 13:11:26.438450 32589 raft_consensus.cc:493] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.438930 32589 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:26.445196   369 raft_consensus.cc:993] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d: : Instructing follower c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:26.445964   369 raft_consensus.cc:1081] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Signalling peer c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:26.448185 32586 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:42544
I20260709 13:11:26.448868 32586 raft_consensus.cc:493] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.449296 32586 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:26.449949 32587 leader_election.cc:290] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:26.451260   407 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 2.
I20260709 13:11:26.453293 32585 raft_consensus.cc:515] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.456107 32585 leader_election.cc:290] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:26.457845 32663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:26.458702 32663 raft_consensus.cc:3055] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.459088 32663 raft_consensus.cc:740] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:26.460124 32663 consensus_queue.cc:260] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.461972 32663 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.466909   372 tablet_bootstrap.cc:492] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:11:26.452971 32555 leader_election.cc:304] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:26.467998   372 tablet_bootstrap.cc:492] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.468321   341 raft_consensus.cc:2804] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.468701   372 ts_tablet_manager.cc:1403] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.094s	user 0.053s	sys 0.040s
I20260709 13:11:26.471648   372 raft_consensus.cc:359] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.472700   372 raft_consensus.cc:740] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.473156   410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:26.473621   372 consensus_queue.cc:260] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.474606   410 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.476259   372 ts_tablet_manager.cc:1434] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:11:26.477137   372 tablet_bootstrap.cc:492] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Bootstrap starting.
I20260709 13:11:26.481838 32662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:26.482688 32662 raft_consensus.cc:3055] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.483121 32662 raft_consensus.cc:740] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:26.487540 32662 consensus_queue.cc:260] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.489040 32662 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.494594 32586 raft_consensus.cc:515] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.492159 32589 raft_consensus.cc:515] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.498663   372 tablet_bootstrap.cc:492] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:11:26.499733   372 tablet_bootstrap.cc:492] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Bootstrap complete.
I20260709 13:11:26.500646   372 ts_tablet_manager.cc:1403] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 13:11:26.501446   405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:26.502089   405 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:26.503504   372 raft_consensus.cc:359] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.503837 32586 leader_election.cc:290] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 3 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:26.504472   372 raft_consensus.cc:740] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edad4013cb44461aad4b40ad080cd98, State: Initialized, Role: FOLLOWER
I20260709 13:11:26.505205   372 consensus_queue.cc:260] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.505925   404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:26.506524   404 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:26.506632 32589 leader_election.cc:290] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 3 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:26.507503   372 ts_tablet_manager.cc:1434] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:26.513077   405 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 3.
I20260709 13:11:26.517133   404 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 3.
I20260709 13:11:26.517973 32661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:26.518311 32660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:26.518569 32661 raft_consensus.cc:3055] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:26.518657   405 raft_consensus.cc:3060] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.518955 32660 raft_consensus.cc:3055] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:26.518972 32661 raft_consensus.cc:740] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:26.519443 32660 raft_consensus.cc:740] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:26.520031 32661 consensus_queue.cc:260] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.520375 32660 consensus_queue.cc:260] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.521277 32661 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:26.521868 32660 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:26.528694 32661 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 3.
I20260709 13:11:26.529254 32660 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 3.
I20260709 13:11:26.530187 32553 leader_election.cc:304] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:26.529767 32663 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 2.
I20260709 13:11:26.532552 32553 leader_election.cc:304] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:26.536151   452 raft_consensus.cc:2804] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:26.536798   452 raft_consensus.cc:697] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:26.538046   452 consensus_queue.cc:237] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.540385   410 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 2.
I20260709 13:11:26.541390   407 raft_consensus.cc:1240] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Rejecting Update request from peer b50e45fe51014825a6cb95a1dac5603d for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:26.541435   341 raft_consensus.cc:697] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:26.542194   457 raft_consensus.cc:2804] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:26.542975   341 consensus_queue.cc:237] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.545503   457 raft_consensus.cc:697] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:26.546326   457 consensus_queue.cc:237] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.546553   405 pending_rounds.cc:85] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Aborting all ops after (but not including) 0
I20260709 13:11:26.544943 32662 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 2.
I20260709 13:11:26.547098   405 pending_rounds.cc:107] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:11:26.547776   405 raft_consensus.cc:2889] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:26.548089 32553 leader_election.cc:304] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50e45fe51014825a6cb95a1dac5603d, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:26.549793   456 raft_consensus.cc:2804] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.551280   456 raft_consensus.cc:697] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:26.552211   456 consensus_queue.cc:237] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:26.567976 32412 catalog_manager.cc:5697] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 1 to 2, leader changed from b50e45fe51014825a6cb95a1dac5603d (127.21.31.67) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.603893 32412 catalog_manager.cc:5697] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 2 to 3, leader changed from b50e45fe51014825a6cb95a1dac5603d (127.21.31.67) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.604988 32412 catalog_manager.cc:5697] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 1 to 2, leader changed from b50e45fe51014825a6cb95a1dac5603d (127.21.31.67) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.606040 32412 catalog_manager.cc:5697] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 2 to 3, leader changed from b50e45fe51014825a6cb95a1dac5603d (127.21.31.67) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.629825   405 raft_consensus.cc:3060] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.634688   405 pending_rounds.cc:85] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Aborting all ops after (but not including) 0
I20260709 13:11:26.634922   405 pending_rounds.cc:107] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:11:26.635140   405 raft_consensus.cc:2889] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:26.898590 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:26.992733 32662 raft_consensus.cc:1275] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:26.993911   457 consensus_queue.cc:1048] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:26.997964   405 raft_consensus.cc:1275] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:27.000504 32662 raft_consensus.cc:1275] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:27.000548   457 consensus_queue.cc:1048] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:27.005043   341 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.007225   407 raft_consensus.cc:1275] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:27.010799   452 consensus_queue.cc:1048] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.012499   407 raft_consensus.cc:1275] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:27.019949   452 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:27.042084 32661 raft_consensus.cc:1275] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:27.045495   457 consensus_queue.cc:1048] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.144954 32661 raft_consensus.cc:1275] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:27.146199   452 consensus_queue.cc:1048] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.154452   405 raft_consensus.cc:1275] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:27.156613   457 consensus_queue.cc:1048] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:27.157234   457 consensus_queue.cc:1243] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Peer 7edad4013cb44461aad4b40ad080cd98 log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 13:11:27.159735   405 pending_rounds.cc:85] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Aborting all ops after (but not including) 0
I20260709 13:11:27.160050   405 pending_rounds.cc:107] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:11:27.160303   405 raft_consensus.cc:2889] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:27.336277 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:27.352623 32586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
mode: GRACEFUL
new_leader_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:42566
I20260709 13:11:27.353158 32586 raft_consensus.cc:606] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Received request to transfer leadership to TS b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:27.355707 32586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
mode: GRACEFUL
new_leader_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:42566
I20260709 13:11:27.356171 32586 raft_consensus.cc:606] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Received request to transfer leadership to TS b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:27.357395 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:11:27.357838 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:27.358371 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:27.558521   457 raft_consensus.cc:993] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015: : Instructing follower b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:27.559127   457 raft_consensus.cc:1081] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Signalling peer b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:27.560819 32661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:47876
I20260709 13:11:27.561493 32661 raft_consensus.cc:493] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:27.561842 32661 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:27.566809 32661 raft_consensus.cc:515] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:27.568495 32661 leader_election.cc:290] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:27.569479 32586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:27.569613   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:27.570142 32586 raft_consensus.cc:3055] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:27.570869 32586 raft_consensus.cc:740] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:27.570948   407 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:27.571825 32586 consensus_queue.cc:260] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:27.573073 32586 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:27.578011   407 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 3.
I20260709 13:11:27.579244 32629 leader_election.cc:304] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:27.579952 32586 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 3.
I20260709 13:11:27.579981   369 raft_consensus.cc:2804] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:27.580652   369 raft_consensus.cc:697] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:27.581465   369 consensus_queue.cc:237] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:27.587088 32411 catalog_manager.cc:5697] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 2 to 3, leader changed from c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 3 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:27.735757   456 raft_consensus.cc:993] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015: : Instructing follower b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:27.736243   452 raft_consensus.cc:1081] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Signalling peer b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:27.737716 32661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:47876
I20260709 13:11:27.738179 32661 raft_consensus.cc:493] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:27.738456 32661 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:27.743124 32661 raft_consensus.cc:515] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:27.744571 32661 leader_election.cc:290] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 4 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:27.745534   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:27.745795 32586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:27.746223   407 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:27.746456 32586 raft_consensus.cc:3055] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:27.746856 32586 raft_consensus.cc:740] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:27.747464 32586 consensus_queue.cc:260] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:27.748370 32586 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:27.752168   407 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 4.
I20260709 13:11:27.753050 32586 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 4.
I20260709 13:11:27.753229 32629 leader_election.cc:304] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:27.754061   350 raft_consensus.cc:2804] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:27.755162   350 raft_consensus.cc:697] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 4 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:27.756052   350 consensus_queue.cc:237] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:27.762391 32411 catalog_manager.cc:5697] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 3 to 4, leader changed from c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 4 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:27.899501 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:28.081458 32586 raft_consensus.cc:1275] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:28.082772   350 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:28.093052   405 raft_consensus.cc:1275] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:28.095362   485 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:28.181051 32586 raft_consensus.cc:1275] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:28.182415   485 consensus_queue.cc:1048] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:28.192157   407 raft_consensus.cc:1275] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:28.193759   485 consensus_queue.cc:1048] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:28.364214 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:28.371272 32661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
mode: GRACEFUL
new_leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:47880
I20260709 13:11:28.371937 32661 raft_consensus.cc:606] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Received request to transfer leadership to TS c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:28.374747 32661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
mode: GRACEFUL
new_leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:47880
I20260709 13:11:28.375296 32661 raft_consensus.cc:606] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 4 LEADER]: Received request to transfer leadership to TS c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:28.377537 32586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
mode: GRACEFUL
new_leader_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:42566
I20260709 13:11:28.378005 32586 raft_consensus.cc:606] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Received request to transfer leadership to TS b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:28.379020 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:11:28.379415 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:28.379940 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:28.636004   456 raft_consensus.cc:993] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015: : Instructing follower b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:28.637075   494 raft_consensus.cc:1081] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Signalling peer b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:28.638909 32661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:47876
I20260709 13:11:28.639490 32661 raft_consensus.cc:493] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:28.639786 32661 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:28.644459 32661 raft_consensus.cc:515] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:28.645905 32661 leader_election.cc:290] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:28.647176 32586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:28.646965   405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:28.648138 32586 raft_consensus.cc:3055] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:28.648211   405 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:28.648669 32586 raft_consensus.cc:740] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:28.649601 32586 consensus_queue.cc:260] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:28.650840 32586 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:28.655485   405 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 3.
I20260709 13:11:28.656745 32629 leader_election.cc:304] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:28.657400   350 raft_consensus.cc:2804] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:28.657497 32586 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 3.
I20260709 13:11:28.657852   350 raft_consensus.cc:697] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:28.658715   350 consensus_queue.cc:237] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:28.665581 32411 catalog_manager.cc:5697] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 2 to 3, leader changed from c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 3 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:28.671166   485 raft_consensus.cc:993] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d: : Instructing follower c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:28.671643   350 raft_consensus.cc:1081] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Signalling peer c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:28.673516 32586 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:42544
I20260709 13:11:28.673997 32586 raft_consensus.cc:493] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:28.674312 32586 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:28.680083 32586 raft_consensus.cc:515] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:28.681640 32586 leader_election.cc:290] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 4 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:28.682228   405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:28.682624 32661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:28.682812   405 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:28.683184 32661 raft_consensus.cc:3055] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:28.683461 32661 raft_consensus.cc:740] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:28.684218 32661 consensus_queue.cc:260] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:28.685248 32661 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:28.687821   405 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 4.
I20260709 13:11:28.688839 32555 leader_election.cc:304] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:28.689632   494 raft_consensus.cc:2804] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:28.690079   494 raft_consensus.cc:697] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 4 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:28.690492 32661 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 4.
I20260709 13:11:28.690798   494 consensus_queue.cc:237] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:28.697119 32411 catalog_manager.cc:5697] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 3 to 4, leader changed from b50e45fe51014825a6cb95a1dac5603d (127.21.31.67) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:28.790865   485 raft_consensus.cc:993] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d: : Instructing follower c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:28.791522   369 raft_consensus.cc:1081] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 4 LEADER]: Signalling peer c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:28.793205 32586 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:42544
I20260709 13:11:28.793816 32586 raft_consensus.cc:493] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:28.794083 32586 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.800366 32586 raft_consensus.cc:515] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:28.802388 32586 leader_election.cc:290] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 5 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:28.803143   405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:28.803632 32661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bb6e466e80141f88334cafd8db9d24f" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:28.803937   405 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.804240 32661 raft_consensus.cc:3055] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:28.804591 32661 raft_consensus.cc:740] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:28.805476 32661 consensus_queue.cc:260] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:28.806769 32661 raft_consensus.cc:3060] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.811503   405 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 5.
I20260709 13:11:28.812875 32555 leader_election.cc:304] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:28.813520 32661 raft_consensus.cc:2468] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 5.
I20260709 13:11:28.813674   494 raft_consensus.cc:2804] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:28.814332   494 raft_consensus.cc:697] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 5 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:28.815110   494 consensus_queue.cc:237] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:28.820779 32411 catalog_manager.cc:5697] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 4 to 5, leader changed from b50e45fe51014825a6cb95a1dac5603d (127.21.31.67) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 5 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:28.900431 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:29.061767   405 raft_consensus.cc:1275] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:29.063242   369 consensus_queue.cc:1048] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:29.074278 32586 raft_consensus.cc:1275] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:29.075845   369 consensus_queue.cc:1048] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:29.160934   405 raft_consensus.cc:1275] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:29.162166   494 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:29.171219 32661 raft_consensus.cc:1275] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:29.172627   494 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:29.196823 32661 raft_consensus.cc:1275] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 5 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:29.198256   494 consensus_queue.cc:1048] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:29.207052   405 raft_consensus.cc:1275] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 5 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:29.208990   509 consensus_queue.cc:1048] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:29.386139 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:29.393582 32586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
mode: GRACEFUL
new_leader_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:42566
I20260709 13:11:29.394104 32586 raft_consensus.cc:606] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 4 LEADER]: Received request to transfer leadership to TS b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:29.396732 32661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
mode: GRACEFUL
new_leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:47880
I20260709 13:11:29.397213 32661 raft_consensus.cc:606] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Received request to transfer leadership to TS c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:29.399477 32586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
mode: GRACEFUL
new_leader_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:42566
I20260709 13:11:29.399940 32586 raft_consensus.cc:606] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Received request to transfer leadership to TS b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:29.400938 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:11:29.401382 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:29.401885 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:29.541968   495 raft_consensus.cc:993] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015: : Instructing follower b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:29.542464   509 raft_consensus.cc:1081] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Signalling peer b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:29.543769 32661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:47876
I20260709 13:11:29.544241 32661 raft_consensus.cc:493] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.544544 32661 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:29.549324 32661 raft_consensus.cc:515] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:29.551257 32661 leader_election.cc:290] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 4 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:29.552106   410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:29.552662 32586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de79c8f3ab0142cfbc6d474965e86646" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:29.552870   410 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:29.553277 32586 raft_consensus.cc:3055] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:29.553660 32586 raft_consensus.cc:740] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:29.554549 32586 consensus_queue.cc:260] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:29.555912 32586 raft_consensus.cc:3060] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:29.560377   410 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 4.
I20260709 13:11:29.561578 32629 leader_election.cc:304] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:29.562508   485 raft_consensus.cc:2804] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:29.562979   485 raft_consensus.cc:697] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 4 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:29.562866 32586 raft_consensus.cc:2468] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 4.
I20260709 13:11:29.563916   485 consensus_queue.cc:237] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:29.565671   369 raft_consensus.cc:993] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d: : Instructing follower c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:29.566164   350 raft_consensus.cc:1081] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Signalling peer c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:29.567830 32589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:42544
I20260709 13:11:29.568310 32589 raft_consensus.cc:493] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.568617 32589 raft_consensus.cc:3060] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:29.571888 32411 catalog_manager.cc:5697] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 3 to 4, leader changed from c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 4 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.573437 32589 raft_consensus.cc:515] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:29.575397 32589 leader_election.cc:290] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 3 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:29.576189   410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:29.576578 32661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67690ebee6ca4f21bacd1292263e8112" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:29.576774   410 raft_consensus.cc:3060] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:29.577191 32661 raft_consensus.cc:3055] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:29.577625 32661 raft_consensus.cc:740] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:29.578332 32661 consensus_queue.cc:260] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:29.579499 32661 raft_consensus.cc:3060] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:29.583004   410 raft_consensus.cc:2468] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 3.
I20260709 13:11:29.584429 32555 leader_election.cc:304] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:29.585044   509 raft_consensus.cc:2804] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:29.585264 32661 raft_consensus.cc:2468] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 3.
I20260709 13:11:29.585558   509 raft_consensus.cc:697] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:29.586877   509 consensus_queue.cc:237] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:29.593381 32411 catalog_manager.cc:5697] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 2 to 3, leader changed from b50e45fe51014825a6cb95a1dac5603d (127.21.31.67) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:29.749941   494 raft_consensus.cc:993] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015: : Instructing follower b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:29.750471   509 raft_consensus.cc:1081] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 4 LEADER]: Signalling peer b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:29.752187 32661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:47876
I20260709 13:11:29.752842 32661 raft_consensus.cc:493] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.753180 32661 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:29.759403 32661 raft_consensus.cc:515] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:29.761114 32661 leader_election.cc:290] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 5 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:29.761811   410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:29.762107 32589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884a2add97c4cb197b49ef63fc4fabe" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:29.762462   410 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:29.762642 32589 raft_consensus.cc:3055] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:29.763034 32589 raft_consensus.cc:740] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:29.763864 32589 consensus_queue.cc:260] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:29.765111 32589 raft_consensus.cc:3060] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:29.769345   410 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 5.
I20260709 13:11:29.770581 32629 leader_election.cc:304] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:29.771253   485 raft_consensus.cc:2804] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:29.771763   485 raft_consensus.cc:697] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 5 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:29.771944 32589 raft_consensus.cc:2468] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 5.
I20260709 13:11:29.772557   485 consensus_queue.cc:237] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:29.778641 32411 catalog_manager.cc:5697] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 4 to 5, leader changed from c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 5 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:29.901291 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:29.961186   410 raft_consensus.cc:1275] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:29.962636   485 consensus_queue.cc:1048] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:29.972025 32589 raft_consensus.cc:1275] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:29.973526   350 consensus_queue.cc:1048] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:30.036404   410 raft_consensus.cc:1275] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:30.037657   509 consensus_queue.cc:1048] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:30.045928 32661 raft_consensus.cc:1275] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:30.047484   509 consensus_queue.cc:1048] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:30.182149   410 raft_consensus.cc:1275] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 5 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:11:30.183331   350 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:30.192770 32589 raft_consensus.cc:1275] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 5 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:11:30.194844   525 consensus_queue.cc:1048] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:30.408259 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:30.419358 32661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
mode: GRACEFUL
new_leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:47880
I20260709 13:11:30.420181 32661 raft_consensus.cc:606] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Received request to transfer leadership to TS c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:30.423388 32589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
mode: GRACEFUL
new_leader_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:42566
I20260709 13:11:30.423907 32589 raft_consensus.cc:606] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Received request to transfer leadership to TS b50e45fe51014825a6cb95a1dac5603d
I20260709 13:11:30.427045 32661 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
mode: GRACEFUL
new_leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:47880
I20260709 13:11:30.427727 32661 raft_consensus.cc:606] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Received request to transfer leadership to TS c372ed6a9e6546f591cd1c6ab584f015
I20260709 13:11:30.429163 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:11:30.429769 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:30.430562 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:30.751461   525 raft_consensus.cc:993] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d: : Instructing follower c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:30.752177   369 raft_consensus.cc:1081] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Signalling peer c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:30.753785 32589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:42544
I20260709 13:11:30.754356 32589 raft_consensus.cc:493] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:30.754698 32589 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:30.762312 32589 raft_consensus.cc:515] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:30.764616 32589 leader_election.cc:290] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 4 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:30.765697   528 raft_consensus.cc:993] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015: : Instructing follower b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:30.765668   410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:30.765882 32661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8fe0a72962740438a86dedeb1009f84" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:30.766501   528 raft_consensus.cc:1081] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Signalling peer b50e45fe51014825a6cb95a1dac5603d to start an election
I20260709 13:11:30.766723 32661 raft_consensus.cc:3055] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:30.766993 32661 raft_consensus.cc:740] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:30.767482   410 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:30.768541 32662 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6"
dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
 from {username='slave'} at 127.0.0.1:47876
I20260709 13:11:30.768115 32661 consensus_queue.cc:260] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:30.769189 32662 raft_consensus.cc:493] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:30.769639 32662 raft_consensus.cc:3060] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:30.769949 32661 raft_consensus.cc:3060] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:30.774981   410 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 4.
I20260709 13:11:30.776690 32555 leader_election.cc:304] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:30.777523   528 raft_consensus.cc:2804] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:30.777979   528 raft_consensus.cc:697] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 4 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:30.779294   528 consensus_queue.cc:237] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:30.780557 32662 raft_consensus.cc:515] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:30.783041 32662 leader_election.cc:290] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781)
I20260709 13:11:30.780607 32661 raft_consensus.cc:2468] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 4.
I20260709 13:11:30.785715 32586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
I20260709 13:11:30.786368 32586 raft_consensus.cc:3055] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:30.786641 32586 raft_consensus.cc:740] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:30.787604 32586 consensus_queue.cc:260] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:30.788635   410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dbd0bff9b847c38724e492a61b09d6" candidate_uuid: "b50e45fe51014825a6cb95a1dac5603d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:30.789201 32586 raft_consensus.cc:3060] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:30.789578   410 raft_consensus.cc:3060] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:30.793387 32412 catalog_manager.cc:5697] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 3 to 4, leader changed from b50e45fe51014825a6cb95a1dac5603d (127.21.31.67) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:30.797901   410 raft_consensus.cc:2468] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 3.
I20260709 13:11:30.799161 32629 leader_election.cc:304] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, b50e45fe51014825a6cb95a1dac5603d; no voters: 
I20260709 13:11:30.799588 32586 raft_consensus.cc:2468] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50e45fe51014825a6cb95a1dac5603d in term 3.
I20260709 13:11:30.800084   369 raft_consensus.cc:2804] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:30.800678   369 raft_consensus.cc:697] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Becoming Leader. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:30.801903   369 consensus_queue.cc:237] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:30.813256   525 raft_consensus.cc:993] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d: : Instructing follower c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:30.813030 32412 catalog_manager.cc:5697] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d reported cstate change: term changed from 2 to 3, leader changed from c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66) to b50e45fe51014825a6cb95a1dac5603d (127.21.31.67). New cstate: current_term: 3 leader_uuid: "b50e45fe51014825a6cb95a1dac5603d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:30.813874   369 raft_consensus.cc:1081] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Signalling peer c372ed6a9e6546f591cd1c6ab584f015 to start an election
I20260709 13:11:30.815259 32586 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb"
dest_uuid: "c372ed6a9e6546f591cd1c6ab584f015"
 from {username='slave'} at 127.0.0.1:42544
I20260709 13:11:30.815690 32586 raft_consensus.cc:493] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:30.815990 32586 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:30.821439 32586 raft_consensus.cc:515] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:30.822683 32586 leader_election.cc:290] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 4 election: Requested vote from peers 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411), b50e45fe51014825a6cb95a1dac5603d (127.21.31.67:46139)
I20260709 13:11:30.823601   410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "7edad4013cb44461aad4b40ad080cd98"
I20260709 13:11:30.823799 32661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24939c381a3346b1b36b86c1ce7b8ceb" candidate_uuid: "c372ed6a9e6546f591cd1c6ab584f015" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b50e45fe51014825a6cb95a1dac5603d"
I20260709 13:11:30.824172   410 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:30.824411 32661 raft_consensus.cc:3055] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:30.824699 32661 raft_consensus.cc:740] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b50e45fe51014825a6cb95a1dac5603d, State: Running, Role: LEADER
I20260709 13:11:30.825564 32661 consensus_queue.cc:260] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:30.826519 32661 raft_consensus.cc:3060] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:30.828979   410 raft_consensus.cc:2468] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 4.
I20260709 13:11:30.830307 32555 leader_election.cc:304] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edad4013cb44461aad4b40ad080cd98, c372ed6a9e6546f591cd1c6ab584f015; no voters: 
I20260709 13:11:30.831159   528 raft_consensus.cc:2804] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:30.831645   528 raft_consensus.cc:697] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 4 LEADER]: Becoming Leader. State: Replica: c372ed6a9e6546f591cd1c6ab584f015, State: Running, Role: LEADER
I20260709 13:11:30.832583 32661 raft_consensus.cc:2468] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c372ed6a9e6546f591cd1c6ab584f015 in term 4.
I20260709 13:11:30.832676   528 consensus_queue.cc:237] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } }
I20260709 13:11:30.840500 32412 catalog_manager.cc:5697] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 reported cstate change: term changed from 3 to 4, leader changed from b50e45fe51014825a6cb95a1dac5603d (127.21.31.67) to c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66). New cstate: current_term: 4 leader_uuid: "c372ed6a9e6546f591cd1c6ab584f015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:30.902338 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:31.216543   410 raft_consensus.cc:1275] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:31.216755 32586 raft_consensus.cc:1275] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Refusing update from remote peer b50e45fe51014825a6cb95a1dac5603d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:31.217943   525 consensus_queue.cc:1048] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:31.218498   369 consensus_queue.cc:1048] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372ed6a9e6546f591cd1c6ab584f015" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:31.259817 32661 raft_consensus.cc:1275] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:31.261010   495 consensus_queue.cc:1048] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:31.271142   405 raft_consensus.cc:1275] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:31.272897   495 consensus_queue.cc:1048] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:31.332115 32661 raft_consensus.cc:1275] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:31.333437   495 consensus_queue.cc:1048] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e45fe51014825a6cb95a1dac5603d" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:31.343020   410 raft_consensus.cc:1275] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Refusing update from remote peer c372ed6a9e6546f591cd1c6ab584f015: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:31.344508   528 consensus_queue.cc:1048] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7edad4013cb44461aad4b40ad080cd98" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:31.437528 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:31.468230 21629 tablet_server.cc:179] TabletServer@127.21.31.65:38411 shutting down...
W20260709 13:11:31.485543 32629 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 86 similar messages]
W20260709 13:11:31.490065 32629 consensus_peers.cc:597] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.496878 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:31.497905 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.499094 21629 raft_consensus.cc:2243] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.499804 21629 raft_consensus.cc:2272] T 8bb6e466e80141f88334cafd8db9d24f P 7edad4013cb44461aad4b40ad080cd98 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.503324 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.504037 21629 raft_consensus.cc:2243] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.504576 21629 raft_consensus.cc:2272] T 1fdd94ee0820431b9d14406fe6100bb9 P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.506974 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.507582 21629 raft_consensus.cc:2243] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.508078 21629 raft_consensus.cc:2272] T 67690ebee6ca4f21bacd1292263e8112 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.510613 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.511178 21629 raft_consensus.cc:2243] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.511613 21629 raft_consensus.cc:2272] T 24939c381a3346b1b36b86c1ce7b8ceb P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.514689 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.515332 21629 raft_consensus.cc:2243] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.515861 21629 raft_consensus.cc:2272] T 3884a2add97c4cb197b49ef63fc4fabe P 7edad4013cb44461aad4b40ad080cd98 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.518363 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.518908 21629 raft_consensus.cc:2243] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.519345 21629 raft_consensus.cc:2272] T 53dbd0bff9b847c38724e492a61b09d6 P 7edad4013cb44461aad4b40ad080cd98 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.522964 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.524019 21629 raft_consensus.cc:2243] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.524724 21629 raft_consensus.cc:2272] T de79c8f3ab0142cfbc6d474965e86646 P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.526283 32555 consensus_peers.cc:597] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.527595 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.528103 21629 raft_consensus.cc:2243] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.528733 21629 raft_consensus.cc:2272] T 3825b68a7235435bb84fa43308cfa0ad P 7edad4013cb44461aad4b40ad080cd98 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.531549 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.532096 21629 raft_consensus.cc:2243] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.532547 21629 raft_consensus.cc:2272] T c8fe0a72962740438a86dedeb1009f84 P 7edad4013cb44461aad4b40ad080cd98 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.535935 21629 tablet_replica.cc:333] stopping tablet replica
I20260709 13:11:31.536480 21629 raft_consensus.cc:2243] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.537084 21629 raft_consensus.cc:2272] T 3391c7fa5f274de5a49d3d43752d262b P 7edad4013cb44461aad4b40ad080cd98 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.545501 32629 consensus_peers.cc:597] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:31.546496 32555 consensus_peers.cc:597] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.571614 21629 tablet_server.cc:196] TabletServer@127.21.31.65:38411 shutdown complete.
I20260709 13:11:31.601414 21629 tablet_server.cc:179] TabletServer@127.21.31.66:0 shutting down...
W20260709 13:11:31.606927 32629 consensus_peers.cc:597] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.630285 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:31.631160 21629 tablet_replica.cc:333] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.632340 21629 raft_consensus.cc:2243] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.633548 21629 raft_consensus.cc:2272] T 8bb6e466e80141f88334cafd8db9d24f P c372ed6a9e6546f591cd1c6ab584f015 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.635818 21629 tablet_replica.cc:333] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.636351 21629 raft_consensus.cc:2243] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.636770 21629 raft_consensus.cc:2272] T 1fdd94ee0820431b9d14406fe6100bb9 P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.639053 21629 tablet_replica.cc:333] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.639539 21629 raft_consensus.cc:2243] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.640246 21629 raft_consensus.cc:2272] T 67690ebee6ca4f21bacd1292263e8112 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.642345 21629 tablet_replica.cc:333] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.642777 21629 raft_consensus.cc:2243] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.643118 21629 raft_consensus.cc:2272] T 53dbd0bff9b847c38724e492a61b09d6 P c372ed6a9e6546f591cd1c6ab584f015 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.646375 21629 tablet_replica.cc:333] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.646895 21629 raft_consensus.cc:2243] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.647504 21629 raft_consensus.cc:2272] T de79c8f3ab0142cfbc6d474965e86646 P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.648419 32629 consensus_peers.cc:597] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.650007 21629 tablet_replica.cc:333] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.650650 21629 raft_consensus.cc:2243] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.651108 21629 raft_consensus.cc:2272] T 3825b68a7235435bb84fa43308cfa0ad P c372ed6a9e6546f591cd1c6ab584f015 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.653385 21629 tablet_replica.cc:333] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.653892 21629 raft_consensus.cc:2243] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.654749 21629 raft_consensus.cc:2272] T 3391c7fa5f274de5a49d3d43752d262b P c372ed6a9e6546f591cd1c6ab584f015 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.656831 21629 tablet_replica.cc:333] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.657398 21629 raft_consensus.cc:2243] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.658128 21629 raft_consensus.cc:2272] T c8fe0a72962740438a86dedeb1009f84 P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.661178 21629 tablet_replica.cc:333] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.661937 21629 raft_consensus.cc:2243] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.662434 21629 raft_consensus.cc:2272] T 3884a2add97c4cb197b49ef63fc4fabe P c372ed6a9e6546f591cd1c6ab584f015 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.664526 21629 tablet_replica.cc:333] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015: stopping tablet replica
I20260709 13:11:31.665146 21629 raft_consensus.cc:2243] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.665916 21629 raft_consensus.cc:2272] T 24939c381a3346b1b36b86c1ce7b8ceb P c372ed6a9e6546f591cd1c6ab584f015 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.684697 32628 consensus_peers.cc:597] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d -> Peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Couldn't send request to peer c372ed6a9e6546f591cd1c6ab584f015. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:31.707191 32629 consensus_peers.cc:597] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d -> Peer 7edad4013cb44461aad4b40ad080cd98 (127.21.31.65:38411): Couldn't send request to peer 7edad4013cb44461aad4b40ad080cd98. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:38411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:31.713423 32628 consensus_peers.cc:597] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d -> Peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Couldn't send request to peer c372ed6a9e6546f591cd1c6ab584f015. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:43781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:31.721189 32628 consensus_peers.cc:597] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d -> Peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Couldn't send request to peer c372ed6a9e6546f591cd1c6ab584f015. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:43781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.725251 21629 tablet_server.cc:196] TabletServer@127.21.31.66:0 shutdown complete.
W20260709 13:11:31.733420 32628 consensus_peers.cc:597] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d -> Peer c372ed6a9e6546f591cd1c6ab584f015 (127.21.31.66:43781): Couldn't send request to peer c372ed6a9e6546f591cd1c6ab584f015. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:43781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.753729 21629 tablet_server.cc:179] TabletServer@127.21.31.67:0 shutting down...
I20260709 13:11:31.786017 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:31.786765 21629 tablet_replica.cc:333] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.787823 21629 raft_consensus.cc:2243] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.788842 21629 raft_consensus.cc:2272] T 53dbd0bff9b847c38724e492a61b09d6 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.791736 21629 tablet_replica.cc:333] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.792279 21629 raft_consensus.cc:2243] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.792703 21629 raft_consensus.cc:2272] T 67690ebee6ca4f21bacd1292263e8112 P b50e45fe51014825a6cb95a1dac5603d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.794852 21629 tablet_replica.cc:333] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.795727 21629 raft_consensus.cc:2243] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.796407 21629 raft_consensus.cc:2272] T 3391c7fa5f274de5a49d3d43752d262b P b50e45fe51014825a6cb95a1dac5603d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.798544 21629 tablet_replica.cc:333] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.799099 21629 raft_consensus.cc:2243] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.799495 21629 raft_consensus.cc:2272] T c8fe0a72962740438a86dedeb1009f84 P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.802393 21629 tablet_replica.cc:333] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.802953 21629 raft_consensus.cc:2243] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.803783 21629 raft_consensus.cc:2272] T de79c8f3ab0142cfbc6d474965e86646 P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.805756 21629 tablet_replica.cc:333] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.806252 21629 raft_consensus.cc:2243] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.807075 21629 raft_consensus.cc:2272] T 3825b68a7235435bb84fa43308cfa0ad P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.808981 21629 tablet_replica.cc:333] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.809537 21629 raft_consensus.cc:2243] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.810364 21629 raft_consensus.cc:2272] T 1fdd94ee0820431b9d14406fe6100bb9 P b50e45fe51014825a6cb95a1dac5603d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.812327 21629 tablet_replica.cc:333] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.812848 21629 raft_consensus.cc:2243] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.813280 21629 raft_consensus.cc:2272] T 8bb6e466e80141f88334cafd8db9d24f P b50e45fe51014825a6cb95a1dac5603d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.815408 21629 tablet_replica.cc:333] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.815876 21629 raft_consensus.cc:2243] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.816273 21629 raft_consensus.cc:2272] T 24939c381a3346b1b36b86c1ce7b8ceb P b50e45fe51014825a6cb95a1dac5603d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.820274 21629 tablet_replica.cc:333] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d: stopping tablet replica
I20260709 13:11:31.820771 21629 raft_consensus.cc:2243] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.821576 21629 raft_consensus.cc:2272] T 3884a2add97c4cb197b49ef63fc4fabe P b50e45fe51014825a6cb95a1dac5603d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.855933 21629 tablet_server.cc:196] TabletServer@127.21.31.67:0 shutdown complete.
I20260709 13:11:31.880683 21629 master.cc:562] Master@127.21.31.126:40843 shutting down...
W20260709 13:11:31.904080 32460 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:11:31.908747 21629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.909264 21629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.909647 21629 tablet_replica.cc:333] T 00000000000000000000000000000000 P 76af26a549894d93aa89efc93864862b: stopping tablet replica
I20260709 13:11:31.930490 21629 master.cc:584] Master@127.21.31.126:40843 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13226 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:11:31.982434 21629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.31.126:41005
I20260709 13:11:31.983484 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:31.989365   553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:31.989367   552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:31.989776   555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:31.990531 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:31.991395 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:31.991564 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:31.991677 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602691991666 us; error 0 us; skew 500 ppm
I20260709 13:11:31.992178 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:31.994697 21629 webserver.cc:533] Webserver started at http://127.21.31.126:44641/ using document root <none> and password file <none>
I20260709 13:11:31.995150 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:31.995302 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:31.995513 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:31.996515 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/master-0-root/instance:
uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9"
format_stamp: "Formatted at 2026-07-09 13:11:31 on dist-test-slave-9s6w"
I20260709 13:11:32.000717 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:11:32.004299   560 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.005031 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:11:32.005292 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/master-0-root
uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9"
format_stamp: "Formatted at 2026-07-09 13:11:31 on dist-test-slave-9s6w"
I20260709 13:11:32.005596 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:32.026784 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:32.028090 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:32.071169 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.126:41005
I20260709 13:11:32.071341   611 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.126:41005 every 8 connection(s)
I20260709 13:11:32.075619   612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.086954   612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9: Bootstrap starting.
I20260709 13:11:32.091781   612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.095773   612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9: No bootstrap required, opened a new log
I20260709 13:11:32.097890   612 raft_consensus.cc:359] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9" member_type: VOTER }
I20260709 13:11:32.098235   612 raft_consensus.cc:385] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.098417   612 raft_consensus.cc:740] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc9fa93f3e84facb13d88e2f0ccf0f9, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.098977   612 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9" member_type: VOTER }
I20260709 13:11:32.099368   612 raft_consensus.cc:399] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:11:32.099562   612 raft_consensus.cc:493] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:11:32.099764   612 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:32.103938   612 raft_consensus.cc:515] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9" member_type: VOTER }
I20260709 13:11:32.104386   612 leader_election.cc:304] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdc9fa93f3e84facb13d88e2f0ccf0f9; no voters: 
I20260709 13:11:32.105641   612 leader_election.cc:290] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:11:32.106072   615 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:32.107689   615 raft_consensus.cc:697] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 1 LEADER]: Becoming Leader. State: Replica: bdc9fa93f3e84facb13d88e2f0ccf0f9, State: Running, Role: LEADER
I20260709 13:11:32.108412   615 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9" member_type: VOTER }
I20260709 13:11:32.108630   612 sys_catalog.cc:565] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:11:32.113415   616 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9" member_type: VOTER } }
I20260709 13:11:32.113816   617 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdc9fa93f3e84facb13d88e2f0ccf0f9. Latest consensus state: current_term: 1 leader_uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc9fa93f3e84facb13d88e2f0ccf0f9" member_type: VOTER } }
I20260709 13:11:32.114653   617 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:32.114212   616 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:32.116664   623 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:11:32.121364   623 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:11:32.125003 21629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:11:32.129518   623 catalog_manager.cc:1383] Generated new cluster ID: 16fa3f481e1e49fb94a8d4101a622f4f
I20260709 13:11:32.129873   623 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:11:32.155123   623 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:11:32.156319   623 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:11:32.178418   623 catalog_manager.cc:6070] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9: Generated new TSK 0
I20260709 13:11:32.178974   623 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:11:32.191823 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:32.199012   633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:32.200054   634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:32.201231   636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:32.202209 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:32.203261 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:32.203451 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:32.203680 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602692203663 us; error 0 us; skew 500 ppm
I20260709 13:11:32.204312 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:32.207265 21629 webserver.cc:533] Webserver started at http://127.21.31.65:40905/ using document root <none> and password file <none>
I20260709 13:11:32.207989 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:32.208209 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:32.208506 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:32.209640 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-0-root/instance:
uuid: "bbe54e21512d4420b6c069a7908c5bc5"
format_stamp: "Formatted at 2026-07-09 13:11:32 on dist-test-slave-9s6w"
I20260709 13:11:32.215065 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:11:32.218511   641 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.219321 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:11:32.219619 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "bbe54e21512d4420b6c069a7908c5bc5"
format_stamp: "Formatted at 2026-07-09 13:11:32 on dist-test-slave-9s6w"
I20260709 13:11:32.219913 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:32.236130 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:32.237525 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:32.239063 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:32.241514 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:32.241727 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.241972 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:32.242166 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.284160 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:40743
I20260709 13:11:32.284293   703 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:40743 every 8 connection(s)
I20260709 13:11:32.289520 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:32.296401   708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:32.297273   709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:32.299816   711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:32.300700 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:32.301942 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:32.302206 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:32.302433 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602692302415 us; error 0 us; skew 500 ppm
I20260709 13:11:32.303068 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:32.303426   704 heartbeater.cc:344] Connected to a master server at 127.21.31.126:41005
I20260709 13:11:32.303821   704 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:32.304551   704 heartbeater.cc:507] Master 127.21.31.126:41005 requested a full tablet report, sending...
I20260709 13:11:32.305896 21629 webserver.cc:533] Webserver started at http://127.21.31.66:39721/ using document root <none> and password file <none>
I20260709 13:11:32.306514 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:32.306722 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:32.306813   577 ts_manager.cc:194] Registered new tserver with Master: bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65:40743)
I20260709 13:11:32.307040 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:32.308701 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-1-root/instance:
uuid: "701ce531e2504f12943e0efcb869e403"
format_stamp: "Formatted at 2026-07-09 13:11:32 on dist-test-slave-9s6w"
I20260709 13:11:32.309290   577 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35822
I20260709 13:11:32.314110 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:11:32.317430   716 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.318274 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:11:32.318578 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-1-root
uuid: "701ce531e2504f12943e0efcb869e403"
format_stamp: "Formatted at 2026-07-09 13:11:32 on dist-test-slave-9s6w"
I20260709 13:11:32.318856 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:32.343266 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:32.344380 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:32.345832 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:32.347893 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:32.348115 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.348345 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:32.348507 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.388300 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.66:41507
I20260709 13:11:32.388477   778 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.66:41507 every 8 connection(s)
I20260709 13:11:32.393291 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:32.400849   782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:32.402292   783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:32.404148   785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:32.405035 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:32.406060 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:32.406348 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:32.406556 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602692406539 us; error 0 us; skew 500 ppm
I20260709 13:11:32.406975   779 heartbeater.cc:344] Connected to a master server at 127.21.31.126:41005
I20260709 13:11:32.407291 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:32.407397   779 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:32.408483   779 heartbeater.cc:507] Master 127.21.31.126:41005 requested a full tablet report, sending...
I20260709 13:11:32.410283 21629 webserver.cc:533] Webserver started at http://127.21.31.67:35911/ using document root <none> and password file <none>
I20260709 13:11:32.410717   577 ts_manager.cc:194] Registered new tserver with Master: 701ce531e2504f12943e0efcb869e403 (127.21.31.66:41507)
I20260709 13:11:32.410817 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:32.411190 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:32.411464 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:32.412186   577 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35830
I20260709 13:11:32.412614 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-2-root/instance:
uuid: "44476e19129f41b0bb33a0d9982c7410"
format_stamp: "Formatted at 2026-07-09 13:11:32 on dist-test-slave-9s6w"
I20260709 13:11:32.418254 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 13:11:32.421790   790 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.422659 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:11:32.422899 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-2-root
uuid: "44476e19129f41b0bb33a0d9982c7410"
format_stamp: "Formatted at 2026-07-09 13:11:32 on dist-test-slave-9s6w"
I20260709 13:11:32.423132 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602359137460-21629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:32.433195 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:32.434332 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:32.435997 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:32.438939 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:32.439194 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.439433 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:32.439602 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:32.480742 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.67:37447
I20260709 13:11:32.480854   852 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.67:37447 every 8 connection(s)
I20260709 13:11:32.495730   853 heartbeater.cc:344] Connected to a master server at 127.21.31.126:41005
I20260709 13:11:32.496196   853 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:32.496966   853 heartbeater.cc:507] Master 127.21.31.126:41005 requested a full tablet report, sending...
I20260709 13:11:32.498909   577 ts_manager.cc:194] Registered new tserver with Master: 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67:37447)
I20260709 13:11:32.499482 21629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015086375s
I20260709 13:11:32.500615   577 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35834
I20260709 13:11:32.528337   577 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35844:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:32.530521   577 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:32.580739   669 tablet_service.cc:1511] Processing CreateTablet for tablet 909d267da7b1425ca840557610effccf (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5b23ef99ce5840d6a9915924f516930f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:32.581080   668 tablet_service.cc:1511] Processing CreateTablet for tablet 555fe9146bc04fd1b989bd4103b0e919 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5b23ef99ce5840d6a9915924f516930f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:32.582224   669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909d267da7b1425ca840557610effccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.583139   668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 555fe9146bc04fd1b989bd4103b0e919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.587232   744 tablet_service.cc:1511] Processing CreateTablet for tablet 909d267da7b1425ca840557610effccf (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5b23ef99ce5840d6a9915924f516930f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:32.588544   744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909d267da7b1425ca840557610effccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.589226   743 tablet_service.cc:1511] Processing CreateTablet for tablet 555fe9146bc04fd1b989bd4103b0e919 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5b23ef99ce5840d6a9915924f516930f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:32.590946   743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 555fe9146bc04fd1b989bd4103b0e919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.592585   818 tablet_service.cc:1511] Processing CreateTablet for tablet 909d267da7b1425ca840557610effccf (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5b23ef99ce5840d6a9915924f516930f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:32.593999   818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909d267da7b1425ca840557610effccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.590615   817 tablet_service.cc:1511] Processing CreateTablet for tablet 555fe9146bc04fd1b989bd4103b0e919 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5b23ef99ce5840d6a9915924f516930f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:32.595391   817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 555fe9146bc04fd1b989bd4103b0e919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.613957   868 tablet_bootstrap.cc:492] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403: Bootstrap starting.
I20260709 13:11:32.619663   869 tablet_bootstrap.cc:492] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410: Bootstrap starting.
I20260709 13:11:32.623996   868 tablet_bootstrap.cc:654] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.627678   870 tablet_bootstrap.cc:492] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5: Bootstrap starting.
I20260709 13:11:32.628881   869 tablet_bootstrap.cc:654] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.632120   868 tablet_bootstrap.cc:492] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403: No bootstrap required, opened a new log
I20260709 13:11:32.632579   868 ts_tablet_manager.cc:1403] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.005s
I20260709 13:11:32.633443   869 tablet_bootstrap.cc:492] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410: No bootstrap required, opened a new log
I20260709 13:11:32.633841   869 ts_tablet_manager.cc:1403] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:11:32.635006   870 tablet_bootstrap.cc:654] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.635507   868 raft_consensus.cc:359] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.635901   869 raft_consensus.cc:359] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.636332   868 raft_consensus.cc:385] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.636569   869 raft_consensus.cc:385] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.636708   868 raft_consensus.cc:740] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 701ce531e2504f12943e0efcb869e403, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.636979   869 raft_consensus.cc:740] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44476e19129f41b0bb33a0d9982c7410, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.637607   868 consensus_queue.cc:260] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.637786   869 consensus_queue.cc:260] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.640462   870 tablet_bootstrap.cc:492] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5: No bootstrap required, opened a new log
I20260709 13:11:32.641144   870 ts_tablet_manager.cc:1403] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.005s
I20260709 13:11:32.645109   869 ts_tablet_manager.cc:1434] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.004s
I20260709 13:11:32.644591   779 heartbeater.cc:499] Master 127.21.31.126:41005 was elected leader, sending a full tablet report...
I20260709 13:11:32.644474   853 heartbeater.cc:499] Master 127.21.31.126:41005 was elected leader, sending a full tablet report...
I20260709 13:11:32.644716   870 raft_consensus.cc:359] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.644119   868 ts_tablet_manager.cc:1434] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.004s
I20260709 13:11:32.651661   870 raft_consensus.cc:385] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.652289   870 raft_consensus.cc:740] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe54e21512d4420b6c069a7908c5bc5, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.652755   868 tablet_bootstrap.cc:492] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403: Bootstrap starting.
I20260709 13:11:32.653020   870 consensus_queue.cc:260] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.651644   869 tablet_bootstrap.cc:492] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410: Bootstrap starting.
I20260709 13:11:32.660650   869 tablet_bootstrap.cc:654] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.661116   868 tablet_bootstrap.cc:654] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.675077   875 raft_consensus.cc:493] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.675813   875 raft_consensus.cc:515] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.679884   704 heartbeater.cc:499] Master 127.21.31.126:41005 was elected leader, sending a full tablet report...
I20260709 13:11:32.680056   870 ts_tablet_manager.cc:1434] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5: Time spent starting tablet: real 0.038s	user 0.013s	sys 0.015s
I20260709 13:11:32.681171   870 tablet_bootstrap.cc:492] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5: Bootstrap starting.
I20260709 13:11:32.681865   868 tablet_bootstrap.cc:492] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403: No bootstrap required, opened a new log
I20260709 13:11:32.682269   869 tablet_bootstrap.cc:492] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410: No bootstrap required, opened a new log
I20260709 13:11:32.682379   868 ts_tablet_manager.cc:1403] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.018s
I20260709 13:11:32.682909   869 ts_tablet_manager.cc:1403] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.020s
I20260709 13:11:32.687583   870 tablet_bootstrap.cc:654] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.689284   869 raft_consensus.cc:359] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.685642   868 raft_consensus.cc:359] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.690608   868 raft_consensus.cc:385] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.691000   868 raft_consensus.cc:740] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 701ce531e2504f12943e0efcb869e403, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.692077   868 consensus_queue.cc:260] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.694895   868 ts_tablet_manager.cc:1434] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:11:32.690080   875 leader_election.cc:290] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65:40743), 701ce531e2504f12943e0efcb869e403 (127.21.31.66:41507)
I20260709 13:11:32.695758   869 raft_consensus.cc:385] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.696182   869 raft_consensus.cc:740] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44476e19129f41b0bb33a0d9982c7410, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.697180   869 consensus_queue.cc:260] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.702162   869 ts_tablet_manager.cc:1434] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:11:32.703626   679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555fe9146bc04fd1b989bd4103b0e919" candidate_uuid: "44476e19129f41b0bb33a0d9982c7410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5" is_pre_election: true
I20260709 13:11:32.704571   679 raft_consensus.cc:2468] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44476e19129f41b0bb33a0d9982c7410 in term 0.
I20260709 13:11:32.710659   793 leader_election.cc:304] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44476e19129f41b0bb33a0d9982c7410, bbe54e21512d4420b6c069a7908c5bc5; no voters: 
I20260709 13:11:32.711833   870 tablet_bootstrap.cc:492] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5: No bootstrap required, opened a new log
I20260709 13:11:32.712423   870 ts_tablet_manager.cc:1403] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.013s
I20260709 13:11:32.712204   754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555fe9146bc04fd1b989bd4103b0e919" candidate_uuid: "44476e19129f41b0bb33a0d9982c7410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "701ce531e2504f12943e0efcb869e403" is_pre_election: true
I20260709 13:11:32.713133   875 raft_consensus.cc:493] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.713855   875 raft_consensus.cc:515] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.715940   875 leader_election.cc:290] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65:40743), 701ce531e2504f12943e0efcb869e403 (127.21.31.66:41507)
I20260709 13:11:32.715713   870 raft_consensus.cc:359] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.716465   875 raft_consensus.cc:2804] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:32.716835   875 raft_consensus.cc:493] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:32.717190   875 raft_consensus.cc:3060] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:32.717921   753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909d267da7b1425ca840557610effccf" candidate_uuid: "44476e19129f41b0bb33a0d9982c7410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "701ce531e2504f12943e0efcb869e403" is_pre_election: true
I20260709 13:11:32.718590   753 raft_consensus.cc:2468] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44476e19129f41b0bb33a0d9982c7410 in term 0.
I20260709 13:11:32.720059   792 leader_election.cc:304] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44476e19129f41b0bb33a0d9982c7410, 701ce531e2504f12943e0efcb869e403; no voters: 
I20260709 13:11:32.713115   754 raft_consensus.cc:2468] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44476e19129f41b0bb33a0d9982c7410 in term 0.
I20260709 13:11:32.716472   870 raft_consensus.cc:385] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.722458   870 raft_consensus.cc:740] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe54e21512d4420b6c069a7908c5bc5, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.722800   881 raft_consensus.cc:2804] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:32.723721   679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909d267da7b1425ca840557610effccf" candidate_uuid: "44476e19129f41b0bb33a0d9982c7410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5" is_pre_election: true
I20260709 13:11:32.723434   870 consensus_queue.cc:260] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.723227   881 raft_consensus.cc:493] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:32.724491   881 raft_consensus.cc:3060] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:32.724589   679 raft_consensus.cc:2468] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44476e19129f41b0bb33a0d9982c7410 in term 0.
I20260709 13:11:32.725492   875 raft_consensus.cc:515] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.727428   870 ts_tablet_manager.cc:1434] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:11:32.728971   679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555fe9146bc04fd1b989bd4103b0e919" candidate_uuid: "44476e19129f41b0bb33a0d9982c7410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5"
I20260709 13:11:32.729526   679 raft_consensus.cc:3060] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:32.729909   875 leader_election.cc:290] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [CANDIDATE]: Term 1 election: Requested vote from peers bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65:40743), 701ce531e2504f12943e0efcb869e403 (127.21.31.66:41507)
I20260709 13:11:32.729456   754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555fe9146bc04fd1b989bd4103b0e919" candidate_uuid: "44476e19129f41b0bb33a0d9982c7410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "701ce531e2504f12943e0efcb869e403"
I20260709 13:11:32.731109   754 raft_consensus.cc:3060] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:32.731760   881 raft_consensus.cc:515] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.733928   881 leader_election.cc:290] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [CANDIDATE]: Term 1 election: Requested vote from peers bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65:40743), 701ce531e2504f12943e0efcb869e403 (127.21.31.66:41507)
I20260709 13:11:32.735560   678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909d267da7b1425ca840557610effccf" candidate_uuid: "44476e19129f41b0bb33a0d9982c7410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5"
I20260709 13:11:32.735944   753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909d267da7b1425ca840557610effccf" candidate_uuid: "44476e19129f41b0bb33a0d9982c7410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "701ce531e2504f12943e0efcb869e403"
I20260709 13:11:32.736184   678 raft_consensus.cc:3060] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:32.736562   753 raft_consensus.cc:3060] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:32.738806   679 raft_consensus.cc:2468] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44476e19129f41b0bb33a0d9982c7410 in term 1.
I20260709 13:11:32.740005   793 leader_election.cc:304] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44476e19129f41b0bb33a0d9982c7410, bbe54e21512d4420b6c069a7908c5bc5; no voters: 
I20260709 13:11:32.740980   881 raft_consensus.cc:2804] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:32.741513   881 raft_consensus.cc:697] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 1 LEADER]: Becoming Leader. State: Replica: 44476e19129f41b0bb33a0d9982c7410, State: Running, Role: LEADER
I20260709 13:11:32.742308   881 consensus_queue.cc:237] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.744450   753 raft_consensus.cc:2468] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44476e19129f41b0bb33a0d9982c7410 in term 1.
I20260709 13:11:32.744663   678 raft_consensus.cc:2468] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44476e19129f41b0bb33a0d9982c7410 in term 1.
I20260709 13:11:32.745512   792 leader_election.cc:304] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44476e19129f41b0bb33a0d9982c7410, 701ce531e2504f12943e0efcb869e403; no voters: 
I20260709 13:11:32.746942   875 raft_consensus.cc:2804] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:32.748135   875 raft_consensus.cc:697] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 1 LEADER]: Becoming Leader. State: Replica: 44476e19129f41b0bb33a0d9982c7410, State: Running, Role: LEADER
I20260709 13:11:32.749605   875 consensus_queue.cc:237] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.747038   754 raft_consensus.cc:2468] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44476e19129f41b0bb33a0d9982c7410 in term 1.
I20260709 13:11:32.759744   576 catalog_manager.cc:5697] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "44476e19129f41b0bb33a0d9982c7410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:32.771107   576 catalog_manager.cc:5697] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "44476e19129f41b0bb33a0d9982c7410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:32.826354   576 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35854:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:32.828256   576 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:32.853217   668 tablet_service.cc:1511] Processing CreateTablet for tablet ef81fe3fd2c14322ac8262cd19b1adb6 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6182969c86f64ceeba5dfdf2ecb28e68]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:32.854146   743 tablet_service.cc:1511] Processing CreateTablet for tablet ef81fe3fd2c14322ac8262cd19b1adb6 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6182969c86f64ceeba5dfdf2ecb28e68]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:32.854611   668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef81fe3fd2c14322ac8262cd19b1adb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.855435   743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef81fe3fd2c14322ac8262cd19b1adb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.856424   818 tablet_service.cc:1511] Processing CreateTablet for tablet ef81fe3fd2c14322ac8262cd19b1adb6 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6182969c86f64ceeba5dfdf2ecb28e68]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:11:32.857695   818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef81fe3fd2c14322ac8262cd19b1adb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.862038   744 tablet_service.cc:1511] Processing CreateTablet for tablet 52e16a7ee62d438aad3216e720358154 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6182969c86f64ceeba5dfdf2ecb28e68]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:32.862962   669 tablet_service.cc:1511] Processing CreateTablet for tablet 52e16a7ee62d438aad3216e720358154 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6182969c86f64ceeba5dfdf2ecb28e68]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:32.863538   744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52e16a7ee62d438aad3216e720358154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.864193   669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52e16a7ee62d438aad3216e720358154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.864128   817 tablet_service.cc:1511] Processing CreateTablet for tablet 52e16a7ee62d438aad3216e720358154 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=6182969c86f64ceeba5dfdf2ecb28e68]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:11:32.865551   817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52e16a7ee62d438aad3216e720358154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:32.875274   869 tablet_bootstrap.cc:492] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410: Bootstrap starting.
I20260709 13:11:32.881002   869 tablet_bootstrap.cc:654] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.883432   870 tablet_bootstrap.cc:492] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5: Bootstrap starting.
I20260709 13:11:32.886658   869 tablet_bootstrap.cc:492] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410: No bootstrap required, opened a new log
I20260709 13:11:32.887174   869 ts_tablet_manager.cc:1403] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:11:32.890074   869 raft_consensus.cc:359] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.890465   870 tablet_bootstrap.cc:654] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.890766   869 raft_consensus.cc:385] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.891137   869 raft_consensus.cc:740] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44476e19129f41b0bb33a0d9982c7410, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.892385   868 tablet_bootstrap.cc:492] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403: Bootstrap starting.
I20260709 13:11:32.896201   869 consensus_queue.cc:260] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.898880   869 ts_tablet_manager.cc:1434] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:11:32.899394   870 tablet_bootstrap.cc:492] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5: No bootstrap required, opened a new log
I20260709 13:11:32.899868   868 tablet_bootstrap.cc:654] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.899914   869 tablet_bootstrap.cc:492] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410: Bootstrap starting.
I20260709 13:11:32.899890   870 ts_tablet_manager.cc:1403] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.005s
I20260709 13:11:32.906016   868 tablet_bootstrap.cc:492] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403: No bootstrap required, opened a new log
I20260709 13:11:32.906648   868 ts_tablet_manager.cc:1403] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.003s
I20260709 13:11:32.907305   869 tablet_bootstrap.cc:654] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.907063   870 raft_consensus.cc:359] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.907846   870 raft_consensus.cc:385] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.908145   870 raft_consensus.cc:740] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe54e21512d4420b6c069a7908c5bc5, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.908906   870 consensus_queue.cc:260] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.911098   870 ts_tablet_manager.cc:1434] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:11:32.912642   870 tablet_bootstrap.cc:492] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5: Bootstrap starting.
I20260709 13:11:32.912884   868 raft_consensus.cc:359] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.913669   868 raft_consensus.cc:385] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.913815   869 tablet_bootstrap.cc:492] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410: No bootstrap required, opened a new log
I20260709 13:11:32.914001   868 raft_consensus.cc:740] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 701ce531e2504f12943e0efcb869e403, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.914346   869 ts_tablet_manager.cc:1403] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.002s
I20260709 13:11:32.915050   868 consensus_queue.cc:260] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.917339   869 raft_consensus.cc:359] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.918066   869 raft_consensus.cc:385] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.918344   869 raft_consensus.cc:740] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44476e19129f41b0bb33a0d9982c7410, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.919159   869 consensus_queue.cc:260] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.919915   868 ts_tablet_manager.cc:1434] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:11:32.921073   868 tablet_bootstrap.cc:492] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403: Bootstrap starting.
I20260709 13:11:32.921344   870 tablet_bootstrap.cc:654] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.921687   869 ts_tablet_manager.cc:1434] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:32.928906   868 tablet_bootstrap.cc:654] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:32.936520   874 raft_consensus.cc:493] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.937013   874 raft_consensus.cc:515] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.937997   868 tablet_bootstrap.cc:492] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403: No bootstrap required, opened a new log
I20260709 13:11:32.938576   868 ts_tablet_manager.cc:1403] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.005s
I20260709 13:11:32.939013   870 tablet_bootstrap.cc:492] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5: No bootstrap required, opened a new log
I20260709 13:11:32.939496   870 ts_tablet_manager.cc:1403] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.003s
I20260709 13:11:32.941666   868 raft_consensus.cc:359] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.942585   868 raft_consensus.cc:385] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.943048   868 raft_consensus.cc:740] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 701ce531e2504f12943e0efcb869e403, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.942947   870 raft_consensus.cc:359] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.943923   870 raft_consensus.cc:385] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:32.944275   870 raft_consensus.cc:740] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe54e21512d4420b6c069a7908c5bc5, State: Initialized, Role: FOLLOWER
I20260709 13:11:32.944360   868 consensus_queue.cc:260] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.945559   870 consensus_queue.cc:260] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.947959   870 ts_tablet_manager.cc:1434] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:11:32.948755   874 leader_election.cc:290] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65:40743), 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67:37447)
I20260709 13:11:32.969902   868 ts_tablet_manager.cc:1434] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403: Time spent starting tablet: real 0.031s	user 0.016s	sys 0.013s
I20260709 13:11:32.984359   678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6" candidate_uuid: "701ce531e2504f12943e0efcb869e403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5" is_pre_election: true
I20260709 13:11:32.985028   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6" candidate_uuid: "701ce531e2504f12943e0efcb869e403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44476e19129f41b0bb33a0d9982c7410" is_pre_election: true
I20260709 13:11:32.985284   678 raft_consensus.cc:2468] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 701ce531e2504f12943e0efcb869e403 in term 0.
I20260709 13:11:32.985931   828 raft_consensus.cc:2468] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 701ce531e2504f12943e0efcb869e403 in term 0.
I20260709 13:11:32.986874   719 leader_election.cc:304] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 701ce531e2504f12943e0efcb869e403, bbe54e21512d4420b6c069a7908c5bc5; no voters: 
I20260709 13:11:32.987937   897 raft_consensus.cc:2804] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:32.988497   897 raft_consensus.cc:493] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:32.988883   897 raft_consensus.cc:3060] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:32.996966   897 raft_consensus.cc:515] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:32.998766   897 leader_election.cc:290] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [CANDIDATE]: Term 1 election: Requested vote from peers bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65:40743), 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67:37447)
I20260709 13:11:32.999610   678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6" candidate_uuid: "701ce531e2504f12943e0efcb869e403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5"
I20260709 13:11:32.999773   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6" candidate_uuid: "701ce531e2504f12943e0efcb869e403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44476e19129f41b0bb33a0d9982c7410"
I20260709 13:11:33.000188   678 raft_consensus.cc:3060] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:33.000370   828 raft_consensus.cc:3060] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:33.005218   678 raft_consensus.cc:2468] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 701ce531e2504f12943e0efcb869e403 in term 1.
I20260709 13:11:33.006244   719 leader_election.cc:304] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 701ce531e2504f12943e0efcb869e403, bbe54e21512d4420b6c069a7908c5bc5; no voters: 
I20260709 13:11:33.006726   828 raft_consensus.cc:2468] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 701ce531e2504f12943e0efcb869e403 in term 1.
I20260709 13:11:33.006891   897 raft_consensus.cc:2804] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:33.007537   897 raft_consensus.cc:697] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 1 LEADER]: Becoming Leader. State: Replica: 701ce531e2504f12943e0efcb869e403, State: Running, Role: LEADER
I20260709 13:11:33.008737   897 consensus_queue.cc:237] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.016717   576 catalog_manager.cc:5697] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 reported cstate change: term changed from 0 to 1, leader changed from <none> to 701ce531e2504f12943e0efcb869e403 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "701ce531e2504f12943e0efcb869e403" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:33.120929   625 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:33.140368   876 raft_consensus.cc:493] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.140931   876 raft_consensus.cc:515] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.142895   876 leader_election.cc:290] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 701ce531e2504f12943e0efcb869e403 (127.21.31.66:41507), 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67:37447)
I20260709 13:11:33.154644   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e16a7ee62d438aad3216e720358154" candidate_uuid: "bbe54e21512d4420b6c069a7908c5bc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44476e19129f41b0bb33a0d9982c7410" is_pre_election: true
I20260709 13:11:33.154844   754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e16a7ee62d438aad3216e720358154" candidate_uuid: "bbe54e21512d4420b6c069a7908c5bc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "701ce531e2504f12943e0efcb869e403" is_pre_election: true
I20260709 13:11:33.155236   828 raft_consensus.cc:2468] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbe54e21512d4420b6c069a7908c5bc5 in term 0.
I20260709 13:11:33.155715   754 raft_consensus.cc:2468] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbe54e21512d4420b6c069a7908c5bc5 in term 0.
I20260709 13:11:33.156430   644 leader_election.cc:304] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44476e19129f41b0bb33a0d9982c7410, bbe54e21512d4420b6c069a7908c5bc5; no voters: 
I20260709 13:11:33.157191   876 raft_consensus.cc:2804] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:33.157584   876 raft_consensus.cc:493] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:33.157919   876 raft_consensus.cc:3060] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:33.162808   876 raft_consensus.cc:515] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.164327   876 leader_election.cc:290] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [CANDIDATE]: Term 1 election: Requested vote from peers 701ce531e2504f12943e0efcb869e403 (127.21.31.66:41507), 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67:37447)
I20260709 13:11:33.165382   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e16a7ee62d438aad3216e720358154" candidate_uuid: "bbe54e21512d4420b6c069a7908c5bc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44476e19129f41b0bb33a0d9982c7410"
I20260709 13:11:33.165382   754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e16a7ee62d438aad3216e720358154" candidate_uuid: "bbe54e21512d4420b6c069a7908c5bc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "701ce531e2504f12943e0efcb869e403"
I20260709 13:11:33.166234   828 raft_consensus.cc:3060] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:33.166294   754 raft_consensus.cc:3060] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:33.167732   875 consensus_queue.cc:1048] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:33.172267   828 raft_consensus.cc:2468] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe54e21512d4420b6c069a7908c5bc5 in term 1.
I20260709 13:11:33.173532   644 leader_election.cc:304] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44476e19129f41b0bb33a0d9982c7410, bbe54e21512d4420b6c069a7908c5bc5; no voters: 
I20260709 13:11:33.174415   876 raft_consensus.cc:2804] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:33.174248   754 raft_consensus.cc:2468] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe54e21512d4420b6c069a7908c5bc5 in term 1.
I20260709 13:11:33.176012   876 raft_consensus.cc:697] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 LEADER]: Becoming Leader. State: Replica: bbe54e21512d4420b6c069a7908c5bc5, State: Running, Role: LEADER
I20260709 13:11:33.177870   876 consensus_queue.cc:237] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.185118   875 consensus_queue.cc:1048] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:33.187525   577 catalog_manager.cc:5697] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "bbe54e21512d4420b6c069a7908c5bc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:33.192116   875 consensus_queue.cc:1048] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:33.204496   875 consensus_queue.cc:1048] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:33.228715   828 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "909d267da7b1425ca840557610effccf"
dest_uuid: "44476e19129f41b0bb33a0d9982c7410"
mode: GRACEFUL
new_leader_uuid: "bbe54e21512d4420b6c069a7908c5bc5"
 from {username='slave'} at 127.0.0.1:52748
I20260709 13:11:33.229226   828 raft_consensus.cc:606] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 1 LEADER]: Received request to transfer leadership to TS bbe54e21512d4420b6c069a7908c5bc5
I20260709 13:11:33.245715   753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6"
dest_uuid: "701ce531e2504f12943e0efcb869e403"
mode: GRACEFUL
new_leader_uuid: "bbe54e21512d4420b6c069a7908c5bc5"
 from {username='slave'} at 127.0.0.1:48922
I20260709 13:11:33.246140   753 raft_consensus.cc:606] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 1 LEADER]: Received request to transfer leadership to TS bbe54e21512d4420b6c069a7908c5bc5
I20260709 13:11:33.468971   897 consensus_queue.cc:1048] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:33.481101   897 consensus_queue.cc:1048] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:33.488669   913 raft_consensus.cc:993] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403: : Instructing follower bbe54e21512d4420b6c069a7908c5bc5 to start an election
I20260709 13:11:33.489357   897 raft_consensus.cc:1081] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 1 LEADER]: Signalling peer bbe54e21512d4420b6c069a7908c5bc5 to start an election
I20260709 13:11:33.491287   678 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6"
dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5"
 from {username='slave'} at 127.0.0.1:47386
I20260709 13:11:33.492081   678 raft_consensus.cc:493] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:33.492583   678 raft_consensus.cc:3060] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:33.498548   678 raft_consensus.cc:515] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.500011   678 leader_election.cc:290] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [CANDIDATE]: Term 2 election: Requested vote from peers 701ce531e2504f12943e0efcb869e403 (127.21.31.66:41507), 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67:37447)
I20260709 13:11:33.500928   753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6" candidate_uuid: "bbe54e21512d4420b6c069a7908c5bc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "701ce531e2504f12943e0efcb869e403"
I20260709 13:11:33.501062   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6" candidate_uuid: "bbe54e21512d4420b6c069a7908c5bc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "44476e19129f41b0bb33a0d9982c7410"
I20260709 13:11:33.501678   753 raft_consensus.cc:3055] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:33.501753   828 raft_consensus.cc:3060] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:33.502213   753 raft_consensus.cc:740] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 701ce531e2504f12943e0efcb869e403, State: Running, Role: LEADER
I20260709 13:11:33.503000   753 consensus_queue.cc:260] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.504019   753 raft_consensus.cc:3060] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:33.508002   828 raft_consensus.cc:2468] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe54e21512d4420b6c069a7908c5bc5 in term 2.
I20260709 13:11:33.509083   644 leader_election.cc:304] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44476e19129f41b0bb33a0d9982c7410, bbe54e21512d4420b6c069a7908c5bc5; no voters: 
I20260709 13:11:33.509653   753 raft_consensus.cc:2468] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe54e21512d4420b6c069a7908c5bc5 in term 2.
I20260709 13:11:33.509958   876 raft_consensus.cc:2804] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:33.510491   876 raft_consensus.cc:697] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 2 LEADER]: Becoming Leader. State: Replica: bbe54e21512d4420b6c069a7908c5bc5, State: Running, Role: LEADER
I20260709 13:11:33.511278   876 consensus_queue.cc:237] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.517282   577 catalog_manager.cc:5697] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 reported cstate change: term changed from 1 to 2, leader changed from 701ce531e2504f12943e0efcb869e403 (127.21.31.66) to bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "bbe54e21512d4420b6c069a7908c5bc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:33.566992   876 consensus_queue.cc:1048] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:33.578558   876 consensus_queue.cc:1048] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:33.744335   875 raft_consensus.cc:993] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410: : Instructing follower bbe54e21512d4420b6c069a7908c5bc5 to start an election
I20260709 13:11:33.744817   881 raft_consensus.cc:1081] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 1 LEADER]: Signalling peer bbe54e21512d4420b6c069a7908c5bc5 to start an election
I20260709 13:11:33.746492   678 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "909d267da7b1425ca840557610effccf"
dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5"
 from {username='slave'} at 127.0.0.1:47374
I20260709 13:11:33.746965   678 raft_consensus.cc:493] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:33.747249   678 raft_consensus.cc:3060] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:33.752125   678 raft_consensus.cc:515] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.754078   678 leader_election.cc:290] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [CANDIDATE]: Term 2 election: Requested vote from peers 701ce531e2504f12943e0efcb869e403 (127.21.31.66:41507), 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67:37447)
I20260709 13:11:33.754948   753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909d267da7b1425ca840557610effccf" candidate_uuid: "bbe54e21512d4420b6c069a7908c5bc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "701ce531e2504f12943e0efcb869e403"
I20260709 13:11:33.755368   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909d267da7b1425ca840557610effccf" candidate_uuid: "bbe54e21512d4420b6c069a7908c5bc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "44476e19129f41b0bb33a0d9982c7410"
I20260709 13:11:33.755626   753 raft_consensus.cc:3060] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:33.755986   828 raft_consensus.cc:3055] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:33.756309   828 raft_consensus.cc:740] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 44476e19129f41b0bb33a0d9982c7410, State: Running, Role: LEADER
I20260709 13:11:33.757115   828 consensus_queue.cc:260] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.758098   828 raft_consensus.cc:3060] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:33.760339   753 raft_consensus.cc:2468] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe54e21512d4420b6c069a7908c5bc5 in term 2.
I20260709 13:11:33.761401   643 leader_election.cc:304] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 701ce531e2504f12943e0efcb869e403, bbe54e21512d4420b6c069a7908c5bc5; no voters: 
I20260709 13:11:33.762065   876 raft_consensus.cc:2804] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:33.762478   876 raft_consensus.cc:697] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 2 LEADER]: Becoming Leader. State: Replica: bbe54e21512d4420b6c069a7908c5bc5, State: Running, Role: LEADER
I20260709 13:11:33.762909   828 raft_consensus.cc:2468] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe54e21512d4420b6c069a7908c5bc5 in term 2.
I20260709 13:11:33.763167   876 consensus_queue.cc:237] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:33.769423   574 catalog_manager.cc:5697] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 reported cstate change: term changed from 1 to 2, leader changed from 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67) to bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "bbe54e21512d4420b6c069a7908c5bc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:33.914500   753 raft_consensus.cc:1275] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 2 FOLLOWER]: Refusing update from remote peer bbe54e21512d4420b6c069a7908c5bc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:33.915933   876 consensus_queue.cc:1048] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:33.924273   828 raft_consensus.cc:1275] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 2 FOLLOWER]: Refusing update from remote peer bbe54e21512d4420b6c069a7908c5bc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:33.925653   876 consensus_queue.cc:1048] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:33.963220 21629 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:11:33.965195 21629 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:11:33.965698 21629 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:11:33.977550   678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6"
dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5"
mode: GRACEFUL
new_leader_uuid: "701ce531e2504f12943e0efcb869e403"
 from {username='slave'} at 127.0.0.1:47390
I20260709 13:11:33.978040   678 raft_consensus.cc:606] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 2 LEADER]: Received request to transfer leadership to TS 701ce531e2504f12943e0efcb869e403
I20260709 13:11:33.979074 21629 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:11:34.173786   828 raft_consensus.cc:1275] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 2 FOLLOWER]: Refusing update from remote peer bbe54e21512d4420b6c069a7908c5bc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:34.175339   876 consensus_queue.cc:1048] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:34.182866   753 raft_consensus.cc:1275] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 2 FOLLOWER]: Refusing update from remote peer bbe54e21512d4420b6c069a7908c5bc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:34.184476   876 consensus_queue.cc:1048] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:34.524947   876 raft_consensus.cc:993] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5: : Instructing follower 701ce531e2504f12943e0efcb869e403 to start an election
I20260709 13:11:34.525444   916 raft_consensus.cc:1081] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 2 LEADER]: Signalling peer 701ce531e2504f12943e0efcb869e403 to start an election
I20260709 13:11:34.527107   753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6"
dest_uuid: "701ce531e2504f12943e0efcb869e403"
 from {username='slave'} at 127.0.0.1:48918
I20260709 13:11:34.527570   753 raft_consensus.cc:493] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:34.527854   753 raft_consensus.cc:3060] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:34.532440   753 raft_consensus.cc:515] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:34.534097   753 leader_election.cc:290] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [CANDIDATE]: Term 3 election: Requested vote from peers bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65:40743), 44476e19129f41b0bb33a0d9982c7410 (127.21.31.67:37447)
I20260709 13:11:34.534751   678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6" candidate_uuid: "701ce531e2504f12943e0efcb869e403" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bbe54e21512d4420b6c069a7908c5bc5"
I20260709 13:11:34.535164   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef81fe3fd2c14322ac8262cd19b1adb6" candidate_uuid: "701ce531e2504f12943e0efcb869e403" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "44476e19129f41b0bb33a0d9982c7410"
I20260709 13:11:34.535380   678 raft_consensus.cc:3055] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:34.535703   678 raft_consensus.cc:740] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bbe54e21512d4420b6c069a7908c5bc5, State: Running, Role: LEADER
I20260709 13:11:34.535759   828 raft_consensus.cc:3060] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:34.536617   678 consensus_queue.cc:260] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:34.537865   678 raft_consensus.cc:3060] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:34.542539   828 raft_consensus.cc:2468] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 701ce531e2504f12943e0efcb869e403 in term 3.
I20260709 13:11:34.543744   719 leader_election.cc:304] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44476e19129f41b0bb33a0d9982c7410, 701ce531e2504f12943e0efcb869e403; no voters: 
I20260709 13:11:34.544312   678 raft_consensus.cc:2468] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 701ce531e2504f12943e0efcb869e403 in term 3.
I20260709 13:11:34.544425   897 raft_consensus.cc:2804] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:34.545553   897 raft_consensus.cc:697] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 3 LEADER]: Becoming Leader. State: Replica: 701ce531e2504f12943e0efcb869e403, State: Running, Role: LEADER
I20260709 13:11:34.546212   897 consensus_queue.cc:237] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } }
I20260709 13:11:34.551855   574 catalog_manager.cc:5697] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 reported cstate change: term changed from 2 to 3, leader changed from bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65) to 701ce531e2504f12943e0efcb869e403 (127.21.31.66). New cstate: current_term: 3 leader_uuid: "701ce531e2504f12943e0efcb869e403" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbe54e21512d4420b6c069a7908c5bc5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 40743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "701ce531e2504f12943e0efcb869e403" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 41507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44476e19129f41b0bb33a0d9982c7410" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 37447 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:34.594820 21629 tablet_server.cc:179] TabletServer@127.21.31.65:0 shutting down...
I20260709 13:11:34.612479 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:34.613144 21629 tablet_replica.cc:333] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5: stopping tablet replica
I20260709 13:11:34.613746 21629 raft_consensus.cc:2243] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:34.614264 21629 raft_consensus.cc:2272] T ef81fe3fd2c14322ac8262cd19b1adb6 P bbe54e21512d4420b6c069a7908c5bc5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.616971 21629 tablet_replica.cc:333] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5: stopping tablet replica
I20260709 13:11:34.617666 21629 raft_consensus.cc:2243] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:34.618337 21629 raft_consensus.cc:2272] T 52e16a7ee62d438aad3216e720358154 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.620039 21629 tablet_replica.cc:333] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5: stopping tablet replica
I20260709 13:11:34.620450 21629 raft_consensus.cc:2243] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:34.621361 21629 raft_consensus.cc:2272] T 909d267da7b1425ca840557610effccf P bbe54e21512d4420b6c069a7908c5bc5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.623546 21629 tablet_replica.cc:333] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5: stopping tablet replica
I20260709 13:11:34.624092 21629 raft_consensus.cc:2243] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:34.624497 21629 raft_consensus.cc:2272] T 555fe9146bc04fd1b989bd4103b0e919 P bbe54e21512d4420b6c069a7908c5bc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.648005 21629 tablet_server.cc:196] TabletServer@127.21.31.65:0 shutdown complete.
I20260709 13:11:34.661275 21629 tablet_server.cc:179] TabletServer@127.21.31.66:0 shutting down...
I20260709 13:11:34.680033 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:34.680853 21629 tablet_replica.cc:333] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403: stopping tablet replica
I20260709 13:11:34.681516 21629 raft_consensus.cc:2243] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:34.681962 21629 raft_consensus.cc:2272] T 52e16a7ee62d438aad3216e720358154 P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.683797 21629 tablet_replica.cc:333] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403: stopping tablet replica
I20260709 13:11:34.684298 21629 raft_consensus.cc:2243] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:34.684870 21629 pending_rounds.cc:70] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403: Trying to abort 1 pending ops.
I20260709 13:11:34.685055 21629 pending_rounds.cc:77] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305636636868550656 op_type: NO_OP noop_request { }
I20260709 13:11:34.685374 21629 raft_consensus.cc:2889] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:11:34.685642 21629 raft_consensus.cc:2272] T ef81fe3fd2c14322ac8262cd19b1adb6 P 701ce531e2504f12943e0efcb869e403 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.687863 21629 tablet_replica.cc:333] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403: stopping tablet replica
I20260709 13:11:34.688292 21629 raft_consensus.cc:2243] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:34.688711 21629 raft_consensus.cc:2272] T 555fe9146bc04fd1b989bd4103b0e919 P 701ce531e2504f12943e0efcb869e403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.690395 21629 tablet_replica.cc:333] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403: stopping tablet replica
I20260709 13:11:34.690855 21629 raft_consensus.cc:2243] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:34.691248 21629 raft_consensus.cc:2272] T 909d267da7b1425ca840557610effccf P 701ce531e2504f12943e0efcb869e403 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:34.709833   793 consensus_peers.cc:597] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 -> Peer bbe54e21512d4420b6c069a7908c5bc5 (127.21.31.65:40743): Couldn't send request to peer bbe54e21512d4420b6c069a7908c5bc5. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:40743: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:34.712545 21629 tablet_server.cc:196] TabletServer@127.21.31.66:0 shutdown complete.
I20260709 13:11:34.724499 21629 tablet_server.cc:179] TabletServer@127.21.31.67:0 shutting down...
I20260709 13:11:34.746505 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:34.747088 21629 tablet_replica.cc:333] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410: stopping tablet replica
I20260709 13:11:34.747618 21629 raft_consensus.cc:2243] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:34.748039 21629 raft_consensus.cc:2272] T 52e16a7ee62d438aad3216e720358154 P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.749837 21629 tablet_replica.cc:333] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410: stopping tablet replica
I20260709 13:11:34.750334 21629 raft_consensus.cc:2243] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:34.750734 21629 raft_consensus.cc:2272] T ef81fe3fd2c14322ac8262cd19b1adb6 P 44476e19129f41b0bb33a0d9982c7410 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.752712 21629 tablet_replica.cc:333] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410: stopping tablet replica
I20260709 13:11:34.753197 21629 raft_consensus.cc:2243] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:34.753677 21629 raft_consensus.cc:2272] T 909d267da7b1425ca840557610effccf P 44476e19129f41b0bb33a0d9982c7410 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.756006 21629 tablet_replica.cc:333] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410: stopping tablet replica
I20260709 13:11:34.756595 21629 raft_consensus.cc:2243] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:34.757299 21629 raft_consensus.cc:2272] T 555fe9146bc04fd1b989bd4103b0e919 P 44476e19129f41b0bb33a0d9982c7410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.779278 21629 tablet_server.cc:196] TabletServer@127.21.31.67:0 shutdown complete.
I20260709 13:11:34.791067 21629 master.cc:562] Master@127.21.31.126:41005 shutting down...
I20260709 13:11:34.806857 21629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:34.807636 21629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:34.808007 21629 tablet_replica.cc:333] T 00000000000000000000000000000000 P bdc9fa93f3e84facb13d88e2f0ccf0f9: stopping tablet replica
I20260709 13:11:34.827172 21629 master.cc:584] Master@127.21.31.126:41005 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2871 ms)
[----------] 6 tests from LeaderRebalancerTest (335546 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:11:34.853989 21629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.31.126:42843
I20260709 13:11:34.855023 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:34.859431   926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:34.860283   927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:34.861171   929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:34.861624 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:34.862383 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:34.862591 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:34.862758 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602694862741 us; error 0 us; skew 500 ppm
I20260709 13:11:34.863276 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:34.865409 21629 webserver.cc:533] Webserver started at http://127.21.31.126:43299/ using document root <none> and password file <none>
I20260709 13:11:34.865871 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:34.866075 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:34.866343 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:34.867337 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/master-0-root/instance:
uuid: "2dbfe7a9e8d749af8c623477f7091b13"
format_stamp: "Formatted at 2026-07-09 13:11:34 on dist-test-slave-9s6w"
I20260709 13:11:34.871483 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:11:34.874985   934 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:34.875823 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:11:34.876055 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/master-0-root
uuid: "2dbfe7a9e8d749af8c623477f7091b13"
format_stamp: "Formatted at 2026-07-09 13:11:34 on dist-test-slave-9s6w"
I20260709 13:11:34.876293 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:34.903486 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:34.904402 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:34.941377 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.126:42843
I20260709 13:11:34.941551   985 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.126:42843 every 8 connection(s)
I20260709 13:11:34.945376   986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:34.955170   986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13: Bootstrap starting.
I20260709 13:11:34.959290   986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:34.964519   986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13: No bootstrap required, opened a new log
I20260709 13:11:34.966503   986 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbfe7a9e8d749af8c623477f7091b13" member_type: VOTER }
I20260709 13:11:34.966845   986 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:34.967252   986 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dbfe7a9e8d749af8c623477f7091b13, State: Initialized, Role: FOLLOWER
I20260709 13:11:34.967897   986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbfe7a9e8d749af8c623477f7091b13" member_type: VOTER }
I20260709 13:11:34.968340   986 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:11:34.968564   986 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:11:34.968858   986 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:34.973030   986 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbfe7a9e8d749af8c623477f7091b13" member_type: VOTER }
I20260709 13:11:34.973729   986 leader_election.cc:304] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dbfe7a9e8d749af8c623477f7091b13; no voters: 
I20260709 13:11:34.974843   986 leader_election.cc:290] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:11:34.975154   989 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:34.976423   989 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 1 LEADER]: Becoming Leader. State: Replica: 2dbfe7a9e8d749af8c623477f7091b13, State: Running, Role: LEADER
I20260709 13:11:34.977061   989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbfe7a9e8d749af8c623477f7091b13" member_type: VOTER }
I20260709 13:11:34.978271   986 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:11:34.980070   991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dbfe7a9e8d749af8c623477f7091b13. Latest consensus state: current_term: 1 leader_uuid: "2dbfe7a9e8d749af8c623477f7091b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbfe7a9e8d749af8c623477f7091b13" member_type: VOTER } }
I20260709 13:11:34.980070   990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dbfe7a9e8d749af8c623477f7091b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dbfe7a9e8d749af8c623477f7091b13" member_type: VOTER } }
I20260709 13:11:34.980819   991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:34.980908   990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:34.986369   993 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:11:34.990958   993 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:11:34.993475 21629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:11:34.999104   993 catalog_manager.cc:1383] Generated new cluster ID: 6ba4c0304cde44448b673a1a439e096a
I20260709 13:11:34.999351   993 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:11:35.024444   993 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:11:35.025655   993 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:11:35.038245   993 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13: Generated new TSK 0
I20260709 13:11:35.038786   993 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:11:35.060027 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:35.065949  1007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:35.066376  1008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:35.068786 21629 server_base.cc:1061] running on GCE node
W20260709 13:11:35.069662  1010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:35.070641 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:35.070933 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:35.071146 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602695071102 us; error 0 us; skew 500 ppm
I20260709 13:11:35.071718 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:35.074391 21629 webserver.cc:533] Webserver started at http://127.21.31.65:44119/ using document root <none> and password file <none>
I20260709 13:11:35.074947 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:35.075165 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:35.075445 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:35.076684 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-0-root/instance:
uuid: "49712d6ed31f4f4bacede77e3047f770"
format_stamp: "Formatted at 2026-07-09 13:11:35 on dist-test-slave-9s6w"
I20260709 13:11:35.082934 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:35.086867  1015 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:35.087762 21629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:11:35.088073 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "49712d6ed31f4f4bacede77e3047f770"
format_stamp: "Formatted at 2026-07-09 13:11:35 on dist-test-slave-9s6w"
I20260709 13:11:35.088312 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:35.111799 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:35.113358 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:35.114928 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:35.117478 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:35.117669 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:35.117990 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:35.118137 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:35.163381 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:46467
I20260709 13:11:35.163502  1077 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:46467 every 8 connection(s)
I20260709 13:11:35.168308 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:35.175172  1082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:35.177054  1083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:35.179828  1085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:35.181432 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:35.181820  1078 heartbeater.cc:344] Connected to a master server at 127.21.31.126:42843
I20260709 13:11:35.182250  1078 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:35.182359 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:35.182683 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:35.182871 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602695182835 us; error 0 us; skew 500 ppm
I20260709 13:11:35.182996  1078 heartbeater.cc:507] Master 127.21.31.126:42843 requested a full tablet report, sending...
I20260709 13:11:35.183518 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:35.185297   951 ts_manager.cc:194] Registered new tserver with Master: 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467)
I20260709 13:11:35.186281 21629 webserver.cc:533] Webserver started at http://127.21.31.66:40949/ using document root <none> and password file <none>
I20260709 13:11:35.186954 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:35.187184 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:35.187469 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:35.187665   951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39182
I20260709 13:11:35.188938 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-1-root/instance:
uuid: "847da4898a4d460ba5696b432c782518"
format_stamp: "Formatted at 2026-07-09 13:11:35 on dist-test-slave-9s6w"
I20260709 13:11:35.193558 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:11:35.196700  1090 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:35.197535 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:11:35.197881 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-1-root
uuid: "847da4898a4d460ba5696b432c782518"
format_stamp: "Formatted at 2026-07-09 13:11:35 on dist-test-slave-9s6w"
I20260709 13:11:35.198115 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:35.216396 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:35.217417 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:35.218752 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:35.220824 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:35.220986 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:35.221259 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:35.221459 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:11:35.262341 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.66:42379
I20260709 13:11:35.262439  1152 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.66:42379 every 8 connection(s)
I20260709 13:11:35.267093 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:35.274283  1156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:35.275043  1157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:35.279115 21629 server_base.cc:1061] running on GCE node
W20260709 13:11:35.280071  1159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:35.281044 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:11:35.281184  1153 heartbeater.cc:344] Connected to a master server at 127.21.31.126:42843
W20260709 13:11:35.281284 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:35.281744 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602695281726 us; error 0 us; skew 500 ppm
I20260709 13:11:35.281780  1153 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:35.282353 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:35.282697  1153 heartbeater.cc:507] Master 127.21.31.126:42843 requested a full tablet report, sending...
I20260709 13:11:35.284863   951 ts_manager.cc:194] Registered new tserver with Master: 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:35.285347 21629 webserver.cc:533] Webserver started at http://127.21.31.67:32865/ using document root <none> and password file <none>
I20260709 13:11:35.285882 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:35.286099 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:35.286363 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:35.286375   951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39186
I20260709 13:11:35.287616 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-2-root/instance:
uuid: "5fd41f935e3446228d999b210c2a4e2b"
format_stamp: "Formatted at 2026-07-09 13:11:35 on dist-test-slave-9s6w"
I20260709 13:11:35.292205 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:11:35.295603  1164 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:35.296326 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:11:35.296624 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-2-root
uuid: "5fd41f935e3446228d999b210c2a4e2b"
format_stamp: "Formatted at 2026-07-09 13:11:35 on dist-test-slave-9s6w"
I20260709 13:11:35.296912 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602359137460-21629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:35.315984 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:35.316907 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:35.318245 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:35.320271 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:35.320489 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:35.320719 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:35.320886 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:35.359856 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.67:40165
I20260709 13:11:35.359979  1226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.67:40165 every 8 connection(s)
I20260709 13:11:35.374082  1227 heartbeater.cc:344] Connected to a master server at 127.21.31.126:42843
I20260709 13:11:35.374675  1227 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:35.375459  1227 heartbeater.cc:507] Master 127.21.31.126:42843 requested a full tablet report, sending...
I20260709 13:11:35.377702   951 ts_manager.cc:194] Registered new tserver with Master: 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:35.377820 21629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014991199s
I20260709 13:11:35.379235   951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39196
I20260709 13:11:35.403088   951 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39198:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:35.405556   951 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:35.472301  1192 tablet_service.cc:1511] Processing CreateTablet for tablet 423005aa03b442348db45ed6e4b4be19 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:35.473097  1191 tablet_service.cc:1511] Processing CreateTablet for tablet 568365befae0431a846444a9414c0e69 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:35.473706  1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 423005aa03b442348db45ed6e4b4be19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.474486  1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 568365befae0431a846444a9414c0e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.478333  1042 tablet_service.cc:1511] Processing CreateTablet for tablet 568365befae0431a846444a9414c0e69 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:35.479668  1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 568365befae0431a846444a9414c0e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.480438  1040 tablet_service.cc:1511] Processing CreateTablet for tablet 8f1b8fab7cd84b96bbb298fc00003204 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:35.481763  1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1b8fab7cd84b96bbb298fc00003204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.481415  1190 tablet_service.cc:1511] Processing CreateTablet for tablet b271932cab664b5196fb024cd0b0ee4a (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:35.477048  1041 tablet_service.cc:1511] Processing CreateTablet for tablet b271932cab664b5196fb024cd0b0ee4a (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:35.483093  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b271932cab664b5196fb024cd0b0ee4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.477015  1043 tablet_service.cc:1511] Processing CreateTablet for tablet 423005aa03b442348db45ed6e4b4be19 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:35.486125  1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 423005aa03b442348db45ed6e4b4be19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.488808  1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b271932cab664b5196fb024cd0b0ee4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.502948  1118 tablet_service.cc:1511] Processing CreateTablet for tablet 423005aa03b442348db45ed6e4b4be19 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:35.504271  1118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 423005aa03b442348db45ed6e4b4be19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.505718  1116 tablet_service.cc:1511] Processing CreateTablet for tablet b271932cab664b5196fb024cd0b0ee4a (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:35.507011  1116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b271932cab664b5196fb024cd0b0ee4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.508653  1117 tablet_service.cc:1511] Processing CreateTablet for tablet 568365befae0431a846444a9414c0e69 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:35.510115  1117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 568365befae0431a846444a9414c0e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.513793  1039 tablet_service.cc:1511] Processing CreateTablet for tablet c0b3f2617ffe4e269fccaf988eb27805 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:35.514008  1115 tablet_service.cc:1511] Processing CreateTablet for tablet 8f1b8fab7cd84b96bbb298fc00003204 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:35.515352  1115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1b8fab7cd84b96bbb298fc00003204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.516039  1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b3f2617ffe4e269fccaf988eb27805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.518049  1242 tablet_bootstrap.cc:492] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:35.532030  1114 tablet_service.cc:1511] Processing CreateTablet for tablet c0b3f2617ffe4e269fccaf988eb27805 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:35.534242  1114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b3f2617ffe4e269fccaf988eb27805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.536177  1191 tablet_service.cc:1511] Processing CreateTablet for tablet c0b3f2617ffe4e269fccaf988eb27805 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:35.538007  1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b3f2617ffe4e269fccaf988eb27805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.537990  1038 tablet_service.cc:1511] Processing CreateTablet for tablet a551497d48ca4ed58899428085203dc8 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:35.539501  1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a551497d48ca4ed58899428085203dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.541042  1190 tablet_service.cc:1511] Processing CreateTablet for tablet 8f1b8fab7cd84b96bbb298fc00003204 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:35.546365  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f1b8fab7cd84b96bbb298fc00003204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.559851  1243 tablet_bootstrap.cc:492] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:35.561964  1113 tablet_service.cc:1511] Processing CreateTablet for tablet a551497d48ca4ed58899428085203dc8 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:35.567816  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a551497d48ca4ed58899428085203dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.567915  1242 tablet_bootstrap.cc:654] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.569415  1189 tablet_service.cc:1511] Processing CreateTablet for tablet a551497d48ca4ed58899428085203dc8 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:35.570712  1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a551497d48ca4ed58899428085203dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.567979  1244 tablet_bootstrap.cc:492] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:35.584642  1244 tablet_bootstrap.cc:654] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.590158  1242 tablet_bootstrap.cc:492] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:35.584551  1041 tablet_service.cc:1511] Processing CreateTablet for tablet 7882c3267d5e42df857e4760dec02397 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:35.590775  1242 ts_tablet_manager.cc:1403] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.073s	user 0.013s	sys 0.000s
I20260709 13:11:35.591434  1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7882c3267d5e42df857e4760dec02397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.585558  1243 tablet_bootstrap.cc:654] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.593817  1242 raft_consensus.cc:359] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.594621  1242 raft_consensus.cc:385] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.594965  1242 raft_consensus.cc:740] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.595908  1242 consensus_queue.cc:260] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.604243  1114 tablet_service.cc:1511] Processing CreateTablet for tablet 7882c3267d5e42df857e4760dec02397 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:35.606127  1242 ts_tablet_manager.cc:1434] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:11:35.607113  1227 heartbeater.cc:499] Master 127.21.31.126:42843 was elected leader, sending a full tablet report...
I20260709 13:11:35.610167  1242 tablet_bootstrap.cc:492] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:35.615406  1114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7882c3267d5e42df857e4760dec02397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.617545  1244 tablet_bootstrap.cc:492] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:35.618148  1244 ts_tablet_manager.cc:1403] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.000s
I20260709 13:11:35.616644  1190 tablet_service.cc:1511] Processing CreateTablet for tablet 7882c3267d5e42df857e4760dec02397 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:35.619140  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7882c3267d5e42df857e4760dec02397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.621066  1244 raft_consensus.cc:359] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.621791  1244 raft_consensus.cc:385] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.622089  1244 raft_consensus.cc:740] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.623026  1244 consensus_queue.cc:260] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.624634  1242 tablet_bootstrap.cc:654] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.627967  1243 tablet_bootstrap.cc:492] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:35.628449  1243 ts_tablet_manager.cc:1403] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.069s	user 0.006s	sys 0.028s
I20260709 13:11:35.631212  1243 raft_consensus.cc:359] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.631917  1243 raft_consensus.cc:385] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.632244  1243 raft_consensus.cc:740] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.632951  1243 consensus_queue.cc:260] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.634595  1153 heartbeater.cc:499] Master 127.21.31.126:42843 was elected leader, sending a full tablet report...
I20260709 13:11:35.635603  1244 ts_tablet_manager.cc:1434] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:11:35.636610  1244 tablet_bootstrap.cc:492] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:35.641587  1078 heartbeater.cc:499] Master 127.21.31.126:42843 was elected leader, sending a full tablet report...
I20260709 13:11:35.642308  1243 ts_tablet_manager.cc:1434] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.003s
I20260709 13:11:35.643843  1243 tablet_bootstrap.cc:492] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:35.642436  1244 tablet_bootstrap.cc:654] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.654635  1243 tablet_bootstrap.cc:654] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.657971  1242 tablet_bootstrap.cc:492] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:35.663745  1242 ts_tablet_manager.cc:1403] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.054s	user 0.008s	sys 0.020s
I20260709 13:11:35.666561  1242 raft_consensus.cc:359] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.667320  1242 raft_consensus.cc:385] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.667650  1242 raft_consensus.cc:740] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.668444  1242 consensus_queue.cc:260] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.670822  1242 ts_tablet_manager.cc:1434] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:35.671861  1242 tablet_bootstrap.cc:492] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:35.677232  1038 tablet_service.cc:1511] Processing CreateTablet for tablet a069639d84a14cc8b8478a31a7bfac22 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:35.678546  1242 tablet_bootstrap.cc:654] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.679015  1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a069639d84a14cc8b8478a31a7bfac22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.690614  1242 tablet_bootstrap.cc:492] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:35.691195  1242 ts_tablet_manager.cc:1403] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 13:11:35.694024  1242 raft_consensus.cc:359] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.694841  1242 raft_consensus.cc:385] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.695188  1242 raft_consensus.cc:740] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.696913  1242 consensus_queue.cc:260] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.698335  1247 raft_consensus.cc:493] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.698822  1247 raft_consensus.cc:515] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.701774  1247 leader_election.cc:290] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:35.706847  1114 tablet_service.cc:1511] Processing CreateTablet for tablet a069639d84a14cc8b8478a31a7bfac22 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:35.712517  1039 tablet_service.cc:1511] Processing CreateTablet for tablet a64795fb830e437093b08e6d69e366eb (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:35.719787  1114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a069639d84a14cc8b8478a31a7bfac22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.720088  1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a64795fb830e437093b08e6d69e366eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.714301  1113 tablet_service.cc:1511] Processing CreateTablet for tablet a64795fb830e437093b08e6d69e366eb (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:35.722980  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a64795fb830e437093b08e6d69e366eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.724371  1244 tablet_bootstrap.cc:492] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:35.724886  1244 ts_tablet_manager.cc:1403] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.088s	user 0.033s	sys 0.038s
I20260709 13:11:35.715394  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b271932cab664b5196fb024cd0b0ee4a" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
W20260709 13:11:35.726573  1167 leader_election.cc:343] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:35.727692  1244 raft_consensus.cc:359] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:35.728425  1244 raft_consensus.cc:385] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.728716  1244 raft_consensus.cc:740] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.735776  1244 consensus_queue.cc:260] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:35.738003  1244 ts_tablet_manager.cc:1434] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:11:35.738379  1243 tablet_bootstrap.cc:492] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:35.739303  1243 ts_tablet_manager.cc:1403] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.096s	user 0.025s	sys 0.047s
I20260709 13:11:35.742153  1244 tablet_bootstrap.cc:492] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:35.748334  1242 ts_tablet_manager.cc:1434] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 13:11:35.749467  1242 tablet_bootstrap.cc:492] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:35.750523  1244 tablet_bootstrap.cc:654] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.751745  1190 tablet_service.cc:1511] Processing CreateTablet for tablet a069639d84a14cc8b8478a31a7bfac22 (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:35.752092  1189 tablet_service.cc:1511] Processing CreateTablet for tablet a64795fb830e437093b08e6d69e366eb (DEFAULT_TABLE table=test-workload [id=32c4b673e71941008a63d6056a0cc114]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:35.742202  1243 raft_consensus.cc:359] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:35.752900  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b271932cab664b5196fb024cd0b0ee4a" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:35.752980  1243 raft_consensus.cc:385] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.753667  1243 raft_consensus.cc:740] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.754433  1243 consensus_queue.cc:260] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:35.756619  1243 ts_tablet_manager.cc:1434] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.002s
I20260709 13:11:35.753041  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a069639d84a14cc8b8478a31a7bfac22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.757616  1243 tablet_bootstrap.cc:492] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:35.757799  1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a64795fb830e437093b08e6d69e366eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:35.759774  1242 tablet_bootstrap.cc:654] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.765033  1243 tablet_bootstrap.cc:654] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.769580  1242 tablet_bootstrap.cc:492] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:35.770076  1242 ts_tablet_manager.cc:1403] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.005s
W20260709 13:11:35.770171  1168 leader_election.cc:343] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:35.770627  1168 leader_election.cc:304] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b; no voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518
I20260709 13:11:35.771396  1247 raft_consensus.cc:2749] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:35.773890  1242 raft_consensus.cc:359] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:35.774714  1242 raft_consensus.cc:385] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.775053  1242 raft_consensus.cc:740] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.775774  1242 consensus_queue.cc:260] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:35.778714  1251 raft_consensus.cc:493] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.779187  1251 raft_consensus.cc:515] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.785388  1251 leader_election.cc:290] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:35.792255  1242 ts_tablet_manager.cc:1434] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:11:35.787886  1244 tablet_bootstrap.cc:492] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:35.795693  1242 tablet_bootstrap.cc:492] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:35.796119  1244 ts_tablet_manager.cc:1403] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.000s
I20260709 13:11:35.799714  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b8fab7cd84b96bbb298fc00003204" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:35.800021  1244 raft_consensus.cc:359] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
W20260709 13:11:35.801574  1017 leader_election.cc:343] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:35.802989  1242 tablet_bootstrap.cc:654] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.809612  1244 raft_consensus.cc:385] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.810134  1244 raft_consensus.cc:740] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.811131  1244 consensus_queue.cc:260] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.812351  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b8fab7cd84b96bbb298fc00003204" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:35.813016  1128 raft_consensus.cc:2468] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
I20260709 13:11:35.814528  1019 leader_election.cc:304] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:35.815047  1242 tablet_bootstrap.cc:492] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:35.815582  1242 ts_tablet_manager.cc:1403] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.001s
I20260709 13:11:35.816686  1251 raft_consensus.cc:2804] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:35.817209  1244 ts_tablet_manager.cc:1434] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 13:11:35.818766  1244 tablet_bootstrap.cc:492] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:35.818938  1242 raft_consensus.cc:359] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.817378  1251 raft_consensus.cc:493] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.820969  1242 raft_consensus.cc:385] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.821171  1251 raft_consensus.cc:3060] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:35.821331  1242 raft_consensus.cc:740] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.821744  1243 tablet_bootstrap.cc:492] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:35.822302  1243 ts_tablet_manager.cc:1403] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.065s	user 0.026s	sys 0.035s
I20260709 13:11:35.822152  1242 consensus_queue.cc:260] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.824723  1242 ts_tablet_manager.cc:1434] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:11:35.825243  1243 raft_consensus.cc:359] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.826040  1243 raft_consensus.cc:385] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.826407  1243 raft_consensus.cc:740] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.827168  1243 consensus_queue.cc:260] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.825718  1242 tablet_bootstrap.cc:492] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:35.828754  1251 raft_consensus.cc:515] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.831532  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b8fab7cd84b96bbb298fc00003204" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:35.832100  1128 raft_consensus.cc:3060] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:35.834771  1243 ts_tablet_manager.cc:1434] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:11:35.835194  1242 tablet_bootstrap.cc:654] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.835731  1243 tablet_bootstrap.cc:492] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:35.838981  1128 raft_consensus.cc:2468] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:35.840266  1019 leader_election.cc:304] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:35.841193  1262 raft_consensus.cc:2804] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:35.842629  1243 tablet_bootstrap.cc:654] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.844030  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f1b8fab7cd84b96bbb298fc00003204" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:35.844635  1202 raft_consensus.cc:3060] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:35.848912  1247 raft_consensus.cc:493] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.849582  1247 raft_consensus.cc:515] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:35.852272  1247 leader_election.cc:290] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:35.854656  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:35.854755  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:35.858018  1053 raft_consensus.cc:2468] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
I20260709 13:11:35.859664  1167 leader_election.cc:304] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:35.860862  1247 raft_consensus.cc:2804] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:35.861382  1247 raft_consensus.cc:493] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.861847  1247 raft_consensus.cc:3060] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:35.869678  1128 raft_consensus.cc:2468] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
I20260709 13:11:35.855449  1249 raft_consensus.cc:493] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.871266  1249 raft_consensus.cc:515] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.856894  1202 raft_consensus.cc:2468] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:35.855386  1251 leader_election.cc:290] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:35.878640  1244 tablet_bootstrap.cc:654] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:35.890630  1266 raft_consensus.cc:493] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.888111  1249 leader_election.cc:290] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:35.891390  1266 raft_consensus.cc:515] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:35.911151  1262 raft_consensus.cc:697] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:35.915910  1247 raft_consensus.cc:515] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:35.919008  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:35.919337  1247 leader_election.cc:290] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:35.919595  1128 raft_consensus.cc:3060] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:35.919929  1247 raft_consensus.cc:493] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.920379  1247 raft_consensus.cc:515] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.923521  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:35.924165  1127 raft_consensus.cc:2468] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
I20260709 13:11:35.925470  1168 leader_election.cc:304] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:35.925626  1247 leader_election.cc:290] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:35.926213  1247 raft_consensus.cc:493] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.926834  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:35.927423  1202 raft_consensus.cc:2468] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:35.928278  1128 raft_consensus.cc:2468] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 1.
I20260709 13:11:35.928099  1247 raft_consensus.cc:515] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.928607  1092 leader_election.cc:304] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:35.934283  1249 raft_consensus.cc:2804] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:35.934625  1249 raft_consensus.cc:493] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.934971  1249 raft_consensus.cc:3060] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:35.934911  1168 leader_election.cc:304] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:35.937819  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:35.938247  1247 leader_election.cc:290] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:35.938485  1052 raft_consensus.cc:2468] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:35.938771  1247 raft_consensus.cc:2804] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:35.938575  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:35.930734  1266 leader_election.cc:290] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165), 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467)
I20260709 13:11:35.939165  1247 raft_consensus.cc:493] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.940647  1247 raft_consensus.cc:3060] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:35.914726  1262 consensus_queue.cc:237] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.933073  1244 tablet_bootstrap.cc:492] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:35.941603  1244 ts_tablet_manager.cc:1403] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.123s	user 0.010s	sys 0.011s
I20260709 13:11:35.933166  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:35.939188  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
W20260709 13:11:35.940202  1168 leader_election.cc:343] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:35.931969  1243 tablet_bootstrap.cc:492] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:35.930935  1242 tablet_bootstrap.cc:492] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:35.945766  1053 raft_consensus.cc:2468] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:35.948678  1243 ts_tablet_manager.cc:1403] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.113s	user 0.033s	sys 0.043s
I20260709 13:11:35.949764  1202 raft_consensus.cc:2393] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 847da4898a4d460ba5696b432c782518 in current term 1: Already voted for candidate 5fd41f935e3446228d999b210c2a4e2b in this term.
I20260709 13:11:35.946241  1244 raft_consensus.cc:359] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.945278  1249 raft_consensus.cc:515] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.952595  1243 raft_consensus.cc:359] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.954854  1244 raft_consensus.cc:385] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.958397  1244 raft_consensus.cc:740] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.959338  1244 consensus_queue.cc:260] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.957429  1243 raft_consensus.cc:385] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:35.963083  1243 raft_consensus.cc:740] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:35.963969  1243 consensus_queue.cc:260] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.966378  1251 raft_consensus.cc:493] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.966832  1251 raft_consensus.cc:515] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.968829  1251 leader_election.cc:290] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:35.969916  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:35.969584  1247 raft_consensus.cc:515] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:35.970865  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b271932cab664b5196fb024cd0b0ee4a" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:35.971179  1202 raft_consensus.cc:2393] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 847da4898a4d460ba5696b432c782518 in current term 1: Already voted for candidate 5fd41f935e3446228d999b210c2a4e2b in this term.
I20260709 13:11:35.971509  1201 raft_consensus.cc:2468] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
I20260709 13:11:35.956154  1093 leader_election.cc:304] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:35.972589  1266 raft_consensus.cc:2764] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:11:35.973001  1017 leader_election.cc:304] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:35.974290  1244 ts_tablet_manager.cc:1434] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.003s
I20260709 13:11:35.974469  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:35.974931  1243 ts_tablet_manager.cc:1434] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.002s
I20260709 13:11:35.975036  1053 raft_consensus.cc:3060] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:35.976878  1243 tablet_bootstrap.cc:492] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:35.955972  1242 ts_tablet_manager.cc:1403] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.130s	user 0.040s	sys 0.043s
I20260709 13:11:35.978749  1251 raft_consensus.cc:2804] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:35.966958  1249 leader_election.cc:290] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:35.978847  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b271932cab664b5196fb024cd0b0ee4a" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:35.980015  1251 raft_consensus.cc:493] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.986279  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:35.987105  1127 raft_consensus.cc:2393] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5fd41f935e3446228d999b210c2a4e2b in current term 1: Already voted for candidate 847da4898a4d460ba5696b432c782518 in this term.
I20260709 13:11:35.989096  1247 leader_election.cc:290] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:35.981050  1128 raft_consensus.cc:2468] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
W20260709 13:11:35.989702   999 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:35.982702  1251 raft_consensus.cc:3060] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:35.991235  1247 raft_consensus.cc:2804] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:35.991756  1247 raft_consensus.cc:697] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:35.992583  1247 consensus_queue.cc:237] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:36.000133  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:36.000716  1052 raft_consensus.cc:3060] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.004598  1242 raft_consensus.cc:359] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.005714  1242 raft_consensus.cc:385] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.006174  1242 raft_consensus.cc:740] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.006421   947 catalog_manager.cc:5697] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.007293  1242 consensus_queue.cc:260] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.009406  1244 tablet_bootstrap.cc:492] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:36.012272  1242 ts_tablet_manager.cc:1434] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 13:11:36.014971  1053 raft_consensus.cc:2468] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:36.016340  1242 tablet_bootstrap.cc:492] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:36.016453  1093 leader_election.cc:304] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:36.017887  1249 raft_consensus.cc:2804] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:36.018350  1249 raft_consensus.cc:697] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:36.018639  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:36.019173  1249 consensus_queue.cc:237] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.020016  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:36.020972  1050 raft_consensus.cc:2393] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5fd41f935e3446228d999b210c2a4e2b in current term 1: Already voted for candidate 847da4898a4d460ba5696b432c782518 in this term.
I20260709 13:11:36.023054  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:36.023903  1053 raft_consensus.cc:2393] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5fd41f935e3446228d999b210c2a4e2b in current term 1: Already voted for candidate 847da4898a4d460ba5696b432c782518 in this term.
I20260709 13:11:36.027983   949 catalog_manager.cc:5697] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.028925  1052 raft_consensus.cc:2468] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 1.
W20260709 13:11:36.022953  1167 leader_election.cc:343] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:36.025063  1244 tablet_bootstrap.cc:654] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.030364  1167 leader_election.cc:304] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b; no voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518
I20260709 13:11:36.031212  1247 raft_consensus.cc:2749] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:36.031723  1167 leader_election.cc:304] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b; no voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518
I20260709 13:11:36.032544  1247 raft_consensus.cc:2749] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:36.030014  1243 tablet_bootstrap.cc:654] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.033166  1242 tablet_bootstrap.cc:654] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.036293  1251 raft_consensus.cc:515] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.039474  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b271932cab664b5196fb024cd0b0ee4a" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:36.040012  1128 raft_consensus.cc:3060] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.040050  1251 leader_election.cc:290] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.041131  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b271932cab664b5196fb024cd0b0ee4a" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:36.041728  1201 raft_consensus.cc:3060] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.042972   950 catalog_manager.cc:5697] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.047904  1128 raft_consensus.cc:2468] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:36.048933  1242 tablet_bootstrap.cc:492] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:36.049584  1201 raft_consensus.cc:2468] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:36.050010  1019 leader_election.cc:304] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:36.050269  1242 ts_tablet_manager.cc:1403] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.005s
I20260709 13:11:36.050923  1251 raft_consensus.cc:2804] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:36.051421  1251 raft_consensus.cc:697] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:36.048960  1244 tablet_bootstrap.cc:492] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:36.052333  1244 ts_tablet_manager.cc:1403] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.003s
I20260709 13:11:36.052229  1251 consensus_queue.cc:237] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.053767  1242 raft_consensus.cc:359] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.054538  1242 raft_consensus.cc:385] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.054884  1242 raft_consensus.cc:740] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.055670  1242 consensus_queue.cc:260] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.056421  1244 raft_consensus.cc:359] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.057474  1244 raft_consensus.cc:385] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.057859  1244 raft_consensus.cc:740] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.059217  1242 ts_tablet_manager.cc:1434] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 13:11:36.060642  1242 tablet_bootstrap.cc:492] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:36.062590  1243 tablet_bootstrap.cc:492] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:36.063259  1243 ts_tablet_manager.cc:1403] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.000s
I20260709 13:11:36.069195  1242 tablet_bootstrap.cc:654] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.068876  1243 raft_consensus.cc:359] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.070086  1243 raft_consensus.cc:385] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.070513  1243 raft_consensus.cc:740] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.071807  1244 consensus_queue.cc:260] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.073056  1249 raft_consensus.cc:493] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.073650  1249 raft_consensus.cc:515] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.079807  1249 leader_election.cc:290] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.080158  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a551497d48ca4ed58899428085203dc8" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
W20260709 13:11:36.083335  1093 leader_election.cc:343] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:36.084410  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a551497d48ca4ed58899428085203dc8" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:36.085237  1201 raft_consensus.cc:2468] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:36.085625  1244 ts_tablet_manager.cc:1434] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 13:11:36.086967  1244 tablet_bootstrap.cc:492] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:36.086885  1092 leader_election.cc:304] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 49712d6ed31f4f4bacede77e3047f770
I20260709 13:11:36.088019  1249 raft_consensus.cc:2804] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:36.088502  1249 raft_consensus.cc:493] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:36.089071  1249 raft_consensus.cc:3060] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.089996  1243 consensus_queue.cc:260] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.092722  1243 ts_tablet_manager.cc:1434] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 13:11:36.093796  1243 tablet_bootstrap.cc:492] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:36.096913  1244 tablet_bootstrap.cc:654] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.099570  1242 tablet_bootstrap.cc:492] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:36.100085  1242 ts_tablet_manager.cc:1403] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.014s
I20260709 13:11:36.102504  1243 tablet_bootstrap.cc:654] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.103917  1249 raft_consensus.cc:515] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.106192  1242 raft_consensus.cc:359] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.106494  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a551497d48ca4ed58899428085203dc8" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:36.106989  1242 raft_consensus.cc:385] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.107326  1242 raft_consensus.cc:740] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
W20260709 13:11:36.107782  1093 leader_election.cc:343] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:36.108080  1242 consensus_queue.cc:260] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.111537  1249 leader_election.cc:290] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.113472  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a551497d48ca4ed58899428085203dc8" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:36.114027  1201 raft_consensus.cc:3060] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.119496   947 catalog_manager.cc:5697] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.127199  1242 ts_tablet_manager.cc:1434] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:11:36.128237  1242 tablet_bootstrap.cc:492] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:36.130590  1201 raft_consensus.cc:2468] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:36.132503  1243 tablet_bootstrap.cc:492] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:36.133342  1242 tablet_bootstrap.cc:654] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.133801  1243 ts_tablet_manager.cc:1403] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.004s
I20260709 13:11:36.134312  1092 leader_election.cc:304] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 49712d6ed31f4f4bacede77e3047f770
I20260709 13:11:36.135176  1249 raft_consensus.cc:2804] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:36.136607  1249 raft_consensus.cc:697] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:36.136963  1243 raft_consensus.cc:359] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.137856  1243 raft_consensus.cc:385] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.137734  1249 consensus_queue.cc:237] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.138831  1243 raft_consensus.cc:740] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.140179  1243 consensus_queue.cc:260] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.143251  1243 ts_tablet_manager.cc:1434] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.005s
I20260709 13:11:36.144577  1243 tablet_bootstrap.cc:492] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:36.144729  1242 tablet_bootstrap.cc:492] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:36.145455  1242 ts_tablet_manager.cc:1403] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 13:11:36.148942  1242 raft_consensus.cc:359] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.149902  1242 raft_consensus.cc:385] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.150257  1242 raft_consensus.cc:740] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.150873  1244 tablet_bootstrap.cc:492] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:36.151413  1244 ts_tablet_manager.cc:1403] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.065s	user 0.007s	sys 0.004s
I20260709 13:11:36.151475  1243 tablet_bootstrap.cc:654] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.151173  1242 consensus_queue.cc:260] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.157697  1242 ts_tablet_manager.cc:1434] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:11:36.160307  1244 raft_consensus.cc:359] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.161818  1244 raft_consensus.cc:385] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.162096  1244 raft_consensus.cc:740] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.162834  1244 consensus_queue.cc:260] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.164703  1244 ts_tablet_manager.cc:1434] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:11:36.165671  1244 tablet_bootstrap.cc:492] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:36.166484  1243 tablet_bootstrap.cc:492] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:36.167023  1243 ts_tablet_manager.cc:1403] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.004s
I20260709 13:11:36.169993  1243 raft_consensus.cc:359] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.170761  1243 raft_consensus.cc:385] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.171123  1243 raft_consensus.cc:740] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.170444   948 catalog_manager.cc:5697] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.171973  1244 tablet_bootstrap.cc:654] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.172029  1243 consensus_queue.cc:260] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.174836  1243 ts_tablet_manager.cc:1434] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:11:36.175895  1243 tablet_bootstrap.cc:492] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:36.182274  1243 tablet_bootstrap.cc:654] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.185364  1244 tablet_bootstrap.cc:492] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:36.185966  1244 ts_tablet_manager.cc:1403] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.009s
I20260709 13:11:36.190002  1244 raft_consensus.cc:359] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.190903  1244 raft_consensus.cc:385] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.191241  1244 raft_consensus.cc:740] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.192126  1244 consensus_queue.cc:260] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.193554  1243 tablet_bootstrap.cc:492] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:36.194036  1243 ts_tablet_manager.cc:1403] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.003s
I20260709 13:11:36.194298  1244 ts_tablet_manager.cc:1434] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:11:36.195334  1244 tablet_bootstrap.cc:492] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:36.196552  1243 raft_consensus.cc:359] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.197228  1243 raft_consensus.cc:385] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.197525  1243 raft_consensus.cc:740] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.198169  1243 consensus_queue.cc:260] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.200611  1243 ts_tablet_manager.cc:1434] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:36.201542  1243 tablet_bootstrap.cc:492] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:36.202648  1244 tablet_bootstrap.cc:654] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.207716  1243 tablet_bootstrap.cc:654] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.211796  1244 tablet_bootstrap.cc:492] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:36.212185  1244 ts_tablet_manager.cc:1403] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 13:11:36.214300  1244 raft_consensus.cc:359] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.214785  1244 raft_consensus.cc:385] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.214995  1244 raft_consensus.cc:740] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.215395  1243 tablet_bootstrap.cc:492] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:36.215772  1243 ts_tablet_manager.cc:1403] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.003s
I20260709 13:11:36.215538  1244 consensus_queue.cc:260] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.217389  1244 ts_tablet_manager.cc:1434] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:11:36.218042  1243 raft_consensus.cc:359] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.218273  1244 tablet_bootstrap.cc:492] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:36.218590  1243 raft_consensus.cc:385] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.219038  1243 raft_consensus.cc:740] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.219625  1243 consensus_queue.cc:260] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.221729  1243 ts_tablet_manager.cc:1434] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:11:36.224496  1244 tablet_bootstrap.cc:654] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:36.230055  1244 tablet_bootstrap.cc:492] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:36.230437  1244 ts_tablet_manager.cc:1403] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:11:36.231709  1247 raft_consensus.cc:493] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.232133  1247 raft_consensus.cc:515] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.233031  1244 raft_consensus.cc:359] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.233702  1244 raft_consensus.cc:385] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:36.233873  1247 leader_election.cc:290] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:36.233995  1244 raft_consensus.cc:740] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:36.234670  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7882c3267d5e42df857e4760dec02397" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:36.234880  1244 consensus_queue.cc:260] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.235313  1052 raft_consensus.cc:2468] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
I20260709 13:11:36.235409  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7882c3267d5e42df857e4760dec02397" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:36.236120  1128 raft_consensus.cc:2468] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
I20260709 13:11:36.236356  1167 leader_election.cc:304] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:36.236853  1244 ts_tablet_manager.cc:1434] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:11:36.237211  1247 raft_consensus.cc:2804] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:36.237649  1247 raft_consensus.cc:493] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:36.238018  1247 raft_consensus.cc:3060] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.243463  1247 raft_consensus.cc:515] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.245100  1247 leader_election.cc:290] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:36.245743  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7882c3267d5e42df857e4760dec02397" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:36.246273  1052 raft_consensus.cc:3060] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.246167  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7882c3267d5e42df857e4760dec02397" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:36.246858  1128 raft_consensus.cc:3060] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.252060  1052 raft_consensus.cc:2468] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 1.
I20260709 13:11:36.252506  1128 raft_consensus.cc:2468] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 1.
I20260709 13:11:36.253000  1167 leader_election.cc:304] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:36.253886  1247 raft_consensus.cc:2804] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:36.254418  1247 raft_consensus.cc:697] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:36.255219  1247 consensus_queue.cc:237] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.262775   947 catalog_manager.cc:5697] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:36.305109  1249 raft_consensus.cc:493] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.305735  1249 raft_consensus.cc:515] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.307308  1249 leader_election.cc:290] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.308183  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a64795fb830e437093b08e6d69e366eb" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:36.308295  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a64795fb830e437093b08e6d69e366eb" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:36.308982  1052 raft_consensus.cc:2468] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:36.308985  1201 raft_consensus.cc:2468] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:36.310292  1092 leader_election.cc:304] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:36.310921  1249 raft_consensus.cc:2804] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:36.311221  1249 raft_consensus.cc:493] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:36.311498  1249 raft_consensus.cc:3060] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.316630  1249 raft_consensus.cc:515] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.318382  1249 leader_election.cc:290] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.319000  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a64795fb830e437093b08e6d69e366eb" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:36.319339  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a64795fb830e437093b08e6d69e366eb" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:36.319455  1052 raft_consensus.cc:3060] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.319854  1201 raft_consensus.cc:3060] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.324290  1201 raft_consensus.cc:2468] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:36.324378  1052 raft_consensus.cc:2468] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:36.325199  1092 leader_election.cc:304] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:36.325877  1249 raft_consensus.cc:2804] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:36.326355  1249 raft_consensus.cc:697] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:36.327051  1249 consensus_queue.cc:237] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.331971  1251 raft_consensus.cc:493] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.332453  1251 raft_consensus.cc:515] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.334178  1251 leader_election.cc:290] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.334957  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:36.335282  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:36.334775   947 catalog_manager.cc:5697] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.335496  1128 raft_consensus.cc:2468] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
I20260709 13:11:36.335912  1201 raft_consensus.cc:2468] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
I20260709 13:11:36.336630  1019 leader_election.cc:304] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:36.337394  1251 raft_consensus.cc:2804] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:36.337754  1251 raft_consensus.cc:493] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:36.338014  1251 raft_consensus.cc:3060] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.342895  1251 raft_consensus.cc:515] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.344522  1251 leader_election.cc:290] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.345126  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:36.345635  1128 raft_consensus.cc:3060] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.345547  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:36.346210  1201 raft_consensus.cc:3060] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.350037  1128 raft_consensus.cc:2468] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:36.350492  1201 raft_consensus.cc:2468] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:36.350992  1019 leader_election.cc:304] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:36.351660  1251 raft_consensus.cc:2804] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:36.352108  1251 raft_consensus.cc:697] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:36.352846  1251 consensus_queue.cc:237] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.359175   947 catalog_manager.cc:5697] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.413080  1251 consensus_queue.cc:1048] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.425782  1264 consensus_queue.cc:1048] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:36.479007  1264 raft_consensus.cc:493] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.479569  1264 raft_consensus.cc:515] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.481950  1264 leader_election.cc:290] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.482797  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a551497d48ca4ed58899428085203dc8" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:36.482991  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a551497d48ca4ed58899428085203dc8" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:36.483915  1201 raft_consensus.cc:2393] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49712d6ed31f4f4bacede77e3047f770 in current term 1: Already voted for candidate 847da4898a4d460ba5696b432c782518 in this term.
I20260709 13:11:36.484838  1017 leader_election.cc:304] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770; no voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518
I20260709 13:11:36.485560  1264 raft_consensus.cc:3060] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.491366  1264 raft_consensus.cc:2749] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:36.491669  1249 consensus_queue.cc:1048] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.503657  1304 consensus_queue.cc:1048] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.549511  1247 raft_consensus.cc:493] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.549938  1247 raft_consensus.cc:515] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.551527  1247 leader_election.cc:290] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:36.552557  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a069639d84a14cc8b8478a31a7bfac22" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:36.552611  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a069639d84a14cc8b8478a31a7bfac22" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:36.553164  1128 raft_consensus.cc:2468] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
I20260709 13:11:36.553454  1052 raft_consensus.cc:2468] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
I20260709 13:11:36.554189  1168 leader_election.cc:304] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:36.554834  1247 raft_consensus.cc:2804] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:36.555176  1247 raft_consensus.cc:493] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:36.555404  1247 raft_consensus.cc:3060] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.560580  1247 raft_consensus.cc:515] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.561208  1264 consensus_queue.cc:1048] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.563149  1247 leader_election.cc:290] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:36.563431  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a069639d84a14cc8b8478a31a7bfac22" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:36.563730  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a069639d84a14cc8b8478a31a7bfac22" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:36.564070  1052 raft_consensus.cc:3060] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.564347  1128 raft_consensus.cc:3060] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:36.571477  1052 raft_consensus.cc:2468] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 1.
I20260709 13:11:36.572870  1167 leader_election.cc:304] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:36.573403  1128 raft_consensus.cc:2468] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 1.
I20260709 13:11:36.573776  1271 raft_consensus.cc:2804] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:36.574333  1271 raft_consensus.cc:697] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:36.575284  1271 consensus_queue.cc:237] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.580144  1247 consensus_queue.cc:1048] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.580600  1262 consensus_queue.cc:1048] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:36.591657   949 catalog_manager.cc:5697] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:36.596396  1271 consensus_queue.cc:1048] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:36.598711  1304 consensus_queue.cc:1048] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.616590  1249 consensus_queue.cc:1048] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.661288  1201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
mode: GRACEFUL
new_leader_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:58238
I20260709 13:11:36.662128  1201 raft_consensus.cc:606] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Received request to transfer leadership to TS 49712d6ed31f4f4bacede77e3047f770
I20260709 13:11:36.675434  1128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "568365befae0431a846444a9414c0e69"
dest_uuid: "847da4898a4d460ba5696b432c782518"
mode: GRACEFUL
new_leader_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:43306
I20260709 13:11:36.675948  1128 raft_consensus.cc:606] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Received request to transfer leadership to TS 49712d6ed31f4f4bacede77e3047f770
I20260709 13:11:36.695906  1052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
mode: GRACEFUL
new_leader_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:48028
I20260709 13:11:36.696528  1052 raft_consensus.cc:606] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Received request to transfer leadership to TS 847da4898a4d460ba5696b432c782518
I20260709 13:11:36.705641  1201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7882c3267d5e42df857e4760dec02397"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
mode: GRACEFUL
new_leader_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:58238
I20260709 13:11:36.706255  1201 raft_consensus.cc:606] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Received request to transfer leadership to TS 847da4898a4d460ba5696b432c782518
I20260709 13:11:36.712633  1201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a069639d84a14cc8b8478a31a7bfac22"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
mode: GRACEFUL
new_leader_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:58238
I20260709 13:11:36.713191  1201 raft_consensus.cc:606] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Received request to transfer leadership to TS 847da4898a4d460ba5696b432c782518
I20260709 13:11:36.718617  1271 consensus_queue.cc:1048] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.719427  1128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a64795fb830e437093b08e6d69e366eb"
dest_uuid: "847da4898a4d460ba5696b432c782518"
mode: GRACEFUL
new_leader_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:43306
I20260709 13:11:36.720185  1128 raft_consensus.cc:606] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Received request to transfer leadership to TS 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:36.729921  1271 consensus_queue.cc:1048] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.737759  1271 raft_consensus.cc:993] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b: : Instructing follower 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:36.738314  1307 raft_consensus.cc:1081] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Signalling peer 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:36.740267  1128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7882c3267d5e42df857e4760dec02397"
dest_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:43288
I20260709 13:11:36.740780  1128 raft_consensus.cc:493] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:36.741149  1128 raft_consensus.cc:3060] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.746009  1128 raft_consensus.cc:515] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.748234  1128 leader_election.cc:290] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.748908  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7882c3267d5e42df857e4760dec02397" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:36.748970  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7882c3267d5e42df857e4760dec02397" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:36.749737  1201 raft_consensus.cc:3055] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:36.750116  1052 raft_consensus.cc:3060] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.750135  1201 raft_consensus.cc:740] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:36.751282  1201 consensus_queue.cc:260] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.752506  1201 raft_consensus.cc:3060] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.755576  1052 raft_consensus.cc:2468] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:36.756675  1093 leader_election.cc:304] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:36.757555  1249 raft_consensus.cc:2804] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:36.758024  1249 raft_consensus.cc:697] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:36.758559  1201 raft_consensus.cc:2468] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:36.758997  1249 consensus_queue.cc:237] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.765048   948 catalog_manager.cc:5697] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 1 to 2, leader changed from 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67) to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.788800  1249 consensus_queue.cc:1048] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.795581  1266 raft_consensus.cc:993] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518: : Instructing follower 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:36.796100  1304 raft_consensus.cc:1081] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Signalling peer 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:36.798763  1202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a64795fb830e437093b08e6d69e366eb"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:58226
I20260709 13:11:36.799523  1202 raft_consensus.cc:493] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:36.799942  1202 raft_consensus.cc:3060] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.801895  1304 consensus_queue.cc:1048] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.808197  1202 raft_consensus.cc:515] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.810295  1202 leader_election.cc:290] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 2 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:36.811251  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a64795fb830e437093b08e6d69e366eb" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:36.811892  1128 raft_consensus.cc:3055] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:36.812256  1128 raft_consensus.cc:740] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:36.813045  1128 consensus_queue.cc:260] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.814097  1128 raft_consensus.cc:3060] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.814090  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a64795fb830e437093b08e6d69e366eb" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:36.814973  1262 consensus_queue.cc:1048] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.822414  1128 raft_consensus.cc:2468] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 2.
I20260709 13:11:36.823567  1168 leader_election.cc:304] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 49712d6ed31f4f4bacede77e3047f770
I20260709 13:11:36.824684  1307 raft_consensus.cc:2804] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:36.825067  1262 raft_consensus.cc:993] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770: : Instructing follower 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:36.825810  1264 raft_consensus.cc:1081] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Signalling peer 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:36.825135  1307 raft_consensus.cc:697] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 2 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:36.827683  1307 consensus_queue.cc:237] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.829427  1251 consensus_queue.cc:1048] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:36.834267  1128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805"
dest_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:43302
I20260709 13:11:36.834846  1128 raft_consensus.cc:493] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:36.835210  1128 raft_consensus.cc:3060] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.840523   948 catalog_manager.cc:5697] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 1 to 2, leader changed from 847da4898a4d460ba5696b432c782518 (127.21.31.66) to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 2 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:36.841753  1128 raft_consensus.cc:515] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.844467  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:36.845033  1052 raft_consensus.cc:3055] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:36.845361  1052 raft_consensus.cc:740] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:36.846159  1052 consensus_queue.cc:260] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.847100  1052 raft_consensus.cc:3060] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.854925  1052 raft_consensus.cc:2468] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:36.855729  1128 leader_election.cc:290] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:36.856082  1093 leader_election.cc:304] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:36.856868  1304 raft_consensus.cc:2804] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:36.857297  1304 raft_consensus.cc:697] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:36.857507  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b3f2617ffe4e269fccaf988eb27805" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:36.858290  1202 raft_consensus.cc:3060] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.858325  1304 consensus_queue.cc:237] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:36.863413  1202 raft_consensus.cc:2468] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:36.864239   948 catalog_manager.cc:5697] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 1 to 2, leader changed from 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65) to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:36.996905  1271 raft_consensus.cc:993] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b: : Instructing follower 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:36.997526  1307 raft_consensus.cc:1081] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Signalling peer 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:36.999140  1052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:48002
I20260709 13:11:36.999713  1052 raft_consensus.cc:493] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:37.000025  1052 raft_consensus.cc:3060] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.004938  1052 raft_consensus.cc:515] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:37.006626  1052 leader_election.cc:290] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Requested vote from peers 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:37.007628  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:37.007786  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:37.008338  1202 raft_consensus.cc:3055] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.008415  1128 raft_consensus.cc:3060] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.008745  1202 raft_consensus.cc:740] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:37.009630  1202 consensus_queue.cc:260] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:37.010715  1202 raft_consensus.cc:3060] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.013525  1128 raft_consensus.cc:2468] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:37.014765  1019 leader_election.cc:304] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:37.015576  1251 raft_consensus.cc:2804] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:37.016198  1251 raft_consensus.cc:697] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:37.016956  1251 consensus_queue.cc:237] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:37.018194  1202 raft_consensus.cc:2468] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:37.024631   948 catalog_manager.cc:5697] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 1 to 2, leader changed from 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67) to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:37.109474  1266 raft_consensus.cc:993] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518: : Instructing follower 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:37.109974  1304 raft_consensus.cc:1081] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Signalling peer 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:37.111425  1052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "568365befae0431a846444a9414c0e69"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:48018
I20260709 13:11:37.111882  1052 raft_consensus.cc:493] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:37.112166  1052 raft_consensus.cc:3060] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.117053  1052 raft_consensus.cc:515] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:37.118678  1052 leader_election.cc:290] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:37.119323  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:37.119668  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "568365befae0431a846444a9414c0e69" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:37.119968  1128 raft_consensus.cc:3055] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.120236  1202 raft_consensus.cc:3060] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.120340  1128 raft_consensus.cc:740] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:37.121222  1128 consensus_queue.cc:260] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:37.123330  1128 raft_consensus.cc:3060] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.124141  1307 consensus_queue.cc:1048] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:37.128273  1202 raft_consensus.cc:2468] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:37.133296  1017 leader_election.cc:304] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:37.134349  1251 raft_consensus.cc:2804] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:37.134915  1251 raft_consensus.cc:697] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:37.135833  1251 consensus_queue.cc:237] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:37.144349  1307 consensus_queue.cc:1048] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:37.144519  1128 raft_consensus.cc:2468] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:37.145285  1202 raft_consensus.cc:1275] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.147254  1304 consensus_queue.cc:1048] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.155980  1307 raft_consensus.cc:993] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b: : Instructing follower 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:37.156442  1307 raft_consensus.cc:1081] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Signalling peer 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:37.156334   948 catalog_manager.cc:5697] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 1 to 2, leader changed from 847da4898a4d460ba5696b432c782518 (127.21.31.66) to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:37.160351  1050 raft_consensus.cc:1275] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.160591  1128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a069639d84a14cc8b8478a31a7bfac22"
dest_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:43288
I20260709 13:11:37.161229  1128 raft_consensus.cc:493] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:37.161628  1128 raft_consensus.cc:3060] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.163590  1249 consensus_queue.cc:1048] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:37.169083  1128 raft_consensus.cc:515] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:37.172425  1128 leader_election.cc:290] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:37.173223  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a069639d84a14cc8b8478a31a7bfac22" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:37.173897  1050 raft_consensus.cc:3060] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.174235  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a069639d84a14cc8b8478a31a7bfac22" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:37.175417  1202 raft_consensus.cc:3055] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.175750  1202 raft_consensus.cc:740] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:37.176527  1202 consensus_queue.cc:260] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:37.177690  1202 raft_consensus.cc:3060] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.183791  1050 raft_consensus.cc:2468] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:37.184029  1202 raft_consensus.cc:2468] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:37.185227  1092 leader_election.cc:304] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:37.186000  1249 raft_consensus.cc:2804] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:37.186412  1249 raft_consensus.cc:697] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:37.187140  1249 consensus_queue.cc:237] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:37.193733   949 catalog_manager.cc:5697] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 1 to 2, leader changed from 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67) to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:37.236505  1050 raft_consensus.cc:3060] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.241305  1050 raft_consensus.cc:1275] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.242383  1307 consensus_queue.cc:1048] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.250386  1128 raft_consensus.cc:1275] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.251999  1271 consensus_queue.cc:1048] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.282516  1050 raft_consensus.cc:1275] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.283867  1249 consensus_queue.cc:1048] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.292696  1202 raft_consensus.cc:1275] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.294319  1326 consensus_queue.cc:1048] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.422080  1202 raft_consensus.cc:1275] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.423779  1251 consensus_queue.cc:1048] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.431525  1128 raft_consensus.cc:1275] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.433035  1262 consensus_queue.cc:1048] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.530079  1202 raft_consensus.cc:1275] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.531247  1262 consensus_queue.cc:1048] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.539608  1128 raft_consensus.cc:1275] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.540942  1262 consensus_queue.cc:1048] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.596940  1050 raft_consensus.cc:1275] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.598347  1326 consensus_queue.cc:1048] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.606583  1202 raft_consensus.cc:1275] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:37.608323  1326 consensus_queue.cc:1048] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:39.748194   948 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58442:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:39.750360   948 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:39.796929  1039 tablet_service.cc:1511] Processing CreateTablet for tablet c9961aebffb649fba246b4fde922b974 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:39.798190  1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9961aebffb649fba246b4fde922b974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.797892  1114 tablet_service.cc:1511] Processing CreateTablet for tablet c9961aebffb649fba246b4fde922b974 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:39.799160  1114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9961aebffb649fba246b4fde922b974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.799314  1189 tablet_service.cc:1511] Processing CreateTablet for tablet c9961aebffb649fba246b4fde922b974 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:39.800526  1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9961aebffb649fba246b4fde922b974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.802342  1038 tablet_service.cc:1511] Processing CreateTablet for tablet 9f469f24d76a42719b3c7eafec6aec8d (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:39.803514  1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f469f24d76a42719b3c7eafec6aec8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.803586  1041 tablet_service.cc:1511] Processing CreateTablet for tablet 5c9a41e7f37a4d98b189476d05d822d9 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:39.804790  1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c9a41e7f37a4d98b189476d05d822d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.807508  1113 tablet_service.cc:1511] Processing CreateTablet for tablet 9f469f24d76a42719b3c7eafec6aec8d (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:39.807703  1116 tablet_service.cc:1511] Processing CreateTablet for tablet 5c9a41e7f37a4d98b189476d05d822d9 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:39.808471  1190 tablet_service.cc:1511] Processing CreateTablet for tablet 9f469f24d76a42719b3c7eafec6aec8d (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:39.809787  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f469f24d76a42719b3c7eafec6aec8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.812438  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f469f24d76a42719b3c7eafec6aec8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.816649  1118 tablet_service.cc:1511] Processing CreateTablet for tablet 4a64ea3118f743858b116fa4c3d463a2 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:39.817947  1118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a64ea3118f743858b116fa4c3d463a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.810982  1042 tablet_service.cc:1511] Processing CreateTablet for tablet 9023ef2e63914bc39db6608de9152c5e (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:39.819311  1042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9023ef2e63914bc39db6608de9152c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.821621  1116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c9a41e7f37a4d98b189476d05d822d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.822779  1192 tablet_service.cc:1511] Processing CreateTablet for tablet 9023ef2e63914bc39db6608de9152c5e (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:39.823091  1188 tablet_service.cc:1511] Processing CreateTablet for tablet 4a64ea3118f743858b116fa4c3d463a2 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:39.824072  1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9023ef2e63914bc39db6608de9152c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.811411  1191 tablet_service.cc:1511] Processing CreateTablet for tablet 5c9a41e7f37a4d98b189476d05d822d9 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:39.809774  1115 tablet_service.cc:1511] Processing CreateTablet for tablet 9023ef2e63914bc39db6608de9152c5e (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:39.826357  1115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9023ef2e63914bc39db6608de9152c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.826272  1043 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7ede558daa44d0bef89e46af8fb781 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:39.827574  1043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7ede558daa44d0bef89e46af8fb781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.827324  1187 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7ede558daa44d0bef89e46af8fb781 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:39.830317  1186 tablet_service.cc:1511] Processing CreateTablet for tablet f7e53a48fe1d466bb54526b0a428ff31 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:39.831350  1117 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7ede558daa44d0bef89e46af8fb781 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:39.832182  1112 tablet_service.cc:1511] Processing CreateTablet for tablet f7e53a48fe1d466bb54526b0a428ff31 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:39.833221  1036 tablet_service.cc:1511] Processing CreateTablet for tablet cd22a8edd9b143608ee50df5b9d447f3 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:39.834425  1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd22a8edd9b143608ee50df5b9d447f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.836513  1111 tablet_service.cc:1511] Processing CreateTablet for tablet cd22a8edd9b143608ee50df5b9d447f3 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:39.838471  1035 tablet_service.cc:1511] Processing CreateTablet for tablet f18afa9b1a0c4db8a959a2ecac5ce504 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:39.832219  1037 tablet_service.cc:1511] Processing CreateTablet for tablet f7e53a48fe1d466bb54526b0a428ff31 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:39.823845  1040 tablet_service.cc:1511] Processing CreateTablet for tablet 4a64ea3118f743858b116fa4c3d463a2 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:39.839775  1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f18afa9b1a0c4db8a959a2ecac5ce504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.840514  1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a64ea3118f743858b116fa4c3d463a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.843154  1110 tablet_service.cc:1511] Processing CreateTablet for tablet f18afa9b1a0c4db8a959a2ecac5ce504 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:39.843722  1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e53a48fe1d466bb54526b0a428ff31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.832581  1117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7ede558daa44d0bef89e46af8fb781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.845642  1112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e53a48fe1d466bb54526b0a428ff31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.847754  1110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f18afa9b1a0c4db8a959a2ecac5ce504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.849630  1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c9a41e7f37a4d98b189476d05d822d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.851112  1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a64ea3118f743858b116fa4c3d463a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.853690  1111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd22a8edd9b143608ee50df5b9d447f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.855000  1185 tablet_service.cc:1511] Processing CreateTablet for tablet cd22a8edd9b143608ee50df5b9d447f3 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:39.859321  1184 tablet_service.cc:1511] Processing CreateTablet for tablet f18afa9b1a0c4db8a959a2ecac5ce504 (DEFAULT_TABLE table=soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:39.867476  1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7ede558daa44d0bef89e46af8fb781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.869099  1186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e53a48fe1d466bb54526b0a428ff31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.873643  1184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f18afa9b1a0c4db8a959a2ecac5ce504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.877686  1185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd22a8edd9b143608ee50df5b9d447f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:39.885309  1341 tablet_bootstrap.cc:492] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:39.896461  1341 tablet_bootstrap.cc:654] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:40.031313   999 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:40.038530  1340 tablet_bootstrap.cc:492] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:40.039881  1342 tablet_bootstrap.cc:492] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:40.049067  1340 tablet_bootstrap.cc:654] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.050848  1341 tablet_bootstrap.cc:492] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:40.058030  1341 ts_tablet_manager.cc:1403] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.173s	user 0.013s	sys 0.004s
I20260709 13:11:40.061046  1341 raft_consensus.cc:359] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.064392  1341 raft_consensus.cc:385] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.064774  1341 raft_consensus.cc:740] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.065056  1340 tablet_bootstrap.cc:492] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:40.055783  1342 tablet_bootstrap.cc:654] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.065657  1340 ts_tablet_manager.cc:1403] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.002s
I20260709 13:11:40.068590  1340 raft_consensus.cc:359] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.068588  1341 consensus_queue.cc:260] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.069837  1340 raft_consensus.cc:385] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.070184  1340 raft_consensus.cc:740] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.071308  1342 tablet_bootstrap.cc:492] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:40.070885  1340 consensus_queue.cc:260] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.071635  1341 ts_tablet_manager.cc:1434] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:11:40.072649  1341 tablet_bootstrap.cc:492] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:40.073071  1340 ts_tablet_manager.cc:1434] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:11:40.074074  1340 tablet_bootstrap.cc:492] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:40.074745  1342 ts_tablet_manager.cc:1403] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.006s
I20260709 13:11:40.077898  1342 raft_consensus.cc:359] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.078579  1342 raft_consensus.cc:385] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.078855  1342 raft_consensus.cc:740] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.079648  1342 consensus_queue.cc:260] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.080247  1340 tablet_bootstrap.cc:654] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.081890  1342 ts_tablet_manager.cc:1434] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:40.085569  1342 tablet_bootstrap.cc:492] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:40.092446  1340 tablet_bootstrap.cc:492] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:40.092756  1341 tablet_bootstrap.cc:654] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.092967  1340 ts_tablet_manager.cc:1403] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 13:11:40.097440  1340 raft_consensus.cc:359] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.098345  1340 raft_consensus.cc:385] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.098670  1340 raft_consensus.cc:740] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.098863  1342 tablet_bootstrap.cc:654] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.099541  1340 consensus_queue.cc:260] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.102304  1340 ts_tablet_manager.cc:1434] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:11:40.103994  1340 tablet_bootstrap.cc:492] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:40.112071  1340 tablet_bootstrap.cc:654] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.116276  1342 tablet_bootstrap.cc:492] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:40.116876  1342 ts_tablet_manager.cc:1403] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.006s
I20260709 13:11:40.117162  1341 tablet_bootstrap.cc:492] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:40.117691  1341 ts_tablet_manager.cc:1403] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.022s
I20260709 13:11:40.120093  1342 raft_consensus.cc:359] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.120447  1341 raft_consensus.cc:359] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.120921  1342 raft_consensus.cc:385] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.121158  1341 raft_consensus.cc:385] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.121362  1342 raft_consensus.cc:740] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.121556  1341 raft_consensus.cc:740] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.122387  1342 consensus_queue.cc:260] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.123071  1341 consensus_queue.cc:260] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.125001  1342 ts_tablet_manager.cc:1434] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:11:40.125463  1341 ts_tablet_manager.cc:1434] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:40.126075  1342 tablet_bootstrap.cc:492] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:40.126397  1341 tablet_bootstrap.cc:492] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:40.127823  1340 tablet_bootstrap.cc:492] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:40.128328  1340 ts_tablet_manager.cc:1403] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.010s
I20260709 13:11:40.131318  1340 raft_consensus.cc:359] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.132009  1340 raft_consensus.cc:385] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.132298  1340 raft_consensus.cc:740] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.132848  1342 tablet_bootstrap.cc:654] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.133147  1341 tablet_bootstrap.cc:654] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.133005  1340 consensus_queue.cc:260] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.135444  1340 ts_tablet_manager.cc:1434] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:40.136574  1340 tablet_bootstrap.cc:492] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:40.144330  1340 tablet_bootstrap.cc:654] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.144619  1341 tablet_bootstrap.cc:492] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:40.145022  1342 tablet_bootstrap.cc:492] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:40.145114  1341 ts_tablet_manager.cc:1403] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:11:40.145818  1342 ts_tablet_manager.cc:1403] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 13:11:40.148240  1341 raft_consensus.cc:359] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.148959  1341 raft_consensus.cc:385] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.149255  1341 raft_consensus.cc:740] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.149825  1342 raft_consensus.cc:359] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.150928  1342 raft_consensus.cc:385] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.151389  1342 raft_consensus.cc:740] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.150092  1341 consensus_queue.cc:260] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.152407  1342 consensus_queue.cc:260] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.154670  1341 ts_tablet_manager.cc:1434] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:11:40.155001  1342 ts_tablet_manager.cc:1434] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:11:40.155647  1341 tablet_bootstrap.cc:492] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:40.156137  1342 tablet_bootstrap.cc:492] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:40.156772  1271 raft_consensus.cc:493] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.157266  1271 raft_consensus.cc:515] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.160645  1271 leader_election.cc:290] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:40.162276  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c9a41e7f37a4d98b189476d05d822d9" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:40.162971  1127 raft_consensus.cc:2468] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
I20260709 13:11:40.163545  1342 tablet_bootstrap.cc:654] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.164330  1168 leader_election.cc:304] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:40.167603  1341 tablet_bootstrap.cc:654] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.170360  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c9a41e7f37a4d98b189476d05d822d9" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:40.171089  1050 raft_consensus.cc:2468] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
I20260709 13:11:40.171423  1340 tablet_bootstrap.cc:492] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:40.172156  1340 ts_tablet_manager.cc:1403] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.010s
I20260709 13:11:40.176144  1340 raft_consensus.cc:359] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.177047  1340 raft_consensus.cc:385] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.177424  1340 raft_consensus.cc:740] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.177734  1271 raft_consensus.cc:2804] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:40.178520  1271 raft_consensus.cc:493] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.178438  1340 consensus_queue.cc:260] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.181723  1340 ts_tablet_manager.cc:1434] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:11:40.183015  1340 tablet_bootstrap.cc:492] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:40.179271  1271 raft_consensus.cc:3060] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.191349  1340 tablet_bootstrap.cc:654] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.312156  1328 raft_consensus.cc:493] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.312741  1328 raft_consensus.cc:515] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.316895  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e53a48fe1d466bb54526b0a428ff31" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
W20260709 13:11:40.318748  1017 leader_election.cc:343] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.319588  1328 leader_election.cc:290] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.321535  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e53a48fe1d466bb54526b0a428ff31" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
W20260709 13:11:40.323083  1019 leader_election.cc:343] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.323683  1019 leader_election.cc:304] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770; no voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518
I20260709 13:11:40.325101  1328 raft_consensus.cc:2749] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:40.326823  1093 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:46467, user_credentials={real_user=slave}} blocked reactor thread for 111919us
W20260709 13:11:40.337692  1167 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.65:46467, user_credentials={real_user=slave}} blocked reactor thread for 44277.7us
I20260709 13:11:40.345099  1342 tablet_bootstrap.cc:492] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:40.345721  1342 ts_tablet_manager.cc:1403] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.190s	user 0.092s	sys 0.088s
I20260709 13:11:40.284952  1271 raft_consensus.cc:515] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.346381  1341 tablet_bootstrap.cc:492] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:40.349267  1341 ts_tablet_manager.cc:1403] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.194s	user 0.089s	sys 0.070s
I20260709 13:11:40.348672  1342 raft_consensus.cc:359] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.349668  1342 raft_consensus.cc:385] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.349975  1342 raft_consensus.cc:740] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.350675  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c9a41e7f37a4d98b189476d05d822d9" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:40.350903  1342 consensus_queue.cc:260] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.351300  1127 raft_consensus.cc:3060] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.351347  1340 tablet_bootstrap.cc:492] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:40.352319  1340 ts_tablet_manager.cc:1403] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.170s	user 0.040s	sys 0.114s
I20260709 13:11:40.352131  1341 raft_consensus.cc:359] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.352967  1341 raft_consensus.cc:385] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.353282  1341 raft_consensus.cc:740] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.353868  1342 ts_tablet_manager.cc:1434] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:11:40.348696  1271 leader_election.cc:290] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:40.355031  1342 tablet_bootstrap.cc:492] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:40.354127  1341 consensus_queue.cc:260] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.357306  1249 raft_consensus.cc:493] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.355985  1340 raft_consensus.cc:359] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.358016  1340 raft_consensus.cc:385] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.357903  1249 raft_consensus.cc:515] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.358422  1340 raft_consensus.cc:740] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.359225  1340 consensus_queue.cc:260] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.361558  1340 ts_tablet_manager.cc:1434] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:11:40.361399  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9961aebffb649fba246b4fde922b974" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:40.362197  1202 raft_consensus.cc:2468] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:40.362407  1342 tablet_bootstrap.cc:654] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.362588  1340 tablet_bootstrap.cc:492] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:40.363159  1127 raft_consensus.cc:2468] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 1.
I20260709 13:11:40.363565  1092 leader_election.cc:304] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:40.364375  1249 leader_election.cc:290] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.364935  1249 raft_consensus.cc:2804] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:40.365255  1249 raft_consensus.cc:493] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.365648  1249 raft_consensus.cc:3060] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.357651  1341 ts_tablet_manager.cc:1434] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:11:40.370672  1341 tablet_bootstrap.cc:492] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:40.410071  1358 raft_consensus.cc:493] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.412611  1358 raft_consensus.cc:515] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.408953  1342 tablet_bootstrap.cc:492] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:40.416425  1342 ts_tablet_manager.cc:1403] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.006s
I20260709 13:11:40.421070  1328 raft_consensus.cc:493] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.422958  1168 leader_election.cc:304] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:40.432000  1271 raft_consensus.cc:2804] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:40.432650  1271 raft_consensus.cc:697] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:40.434505  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:40.434252  1271 consensus_queue.cc:237] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.434993  1341 tablet_bootstrap.cc:654] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:40.437070  1168 leader_election.cc:343] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.426107  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c9a41e7f37a4d98b189476d05d822d9" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:40.425966  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9961aebffb649fba246b4fde922b974" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:40.426698  1340 tablet_bootstrap.cc:654] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.439236  1050 raft_consensus.cc:3060] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.439406  1052 raft_consensus.cc:2468] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:40.427314  1358 leader_election.cc:290] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:40.427590  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:40.423753  1342 raft_consensus.cc:359] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.444049  1342 raft_consensus.cc:385] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.444437  1342 raft_consensus.cc:740] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.445302  1342 consensus_queue.cc:260] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
W20260709 13:11:40.446091  1167 leader_election.cc:343] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.447084  1167 leader_election.cc:304] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b; no voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518
I20260709 13:11:40.447978  1342 ts_tablet_manager.cc:1434] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.004s
I20260709 13:11:40.437719  1328 raft_consensus.cc:515] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.450265  1358 raft_consensus.cc:2749] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:40.459617  1342 tablet_bootstrap.cc:492] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:40.460877  1328 leader_election.cc:290] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.469099  1341 tablet_bootstrap.cc:492] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:40.470166  1341 ts_tablet_manager.cc:1403] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.100s	user 0.017s	sys 0.004s
I20260709 13:11:40.472648  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7ede558daa44d0bef89e46af8fb781" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:40.473138  1357 raft_consensus.cc:493] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.475116  1341 raft_consensus.cc:359] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.478261  1271 raft_consensus.cc:493] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.481158  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7ede558daa44d0bef89e46af8fb781" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:40.481397  1249 raft_consensus.cc:515] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.482564  1356 raft_consensus.cc:493] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.483387  1356 raft_consensus.cc:515] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.486054  1356 leader_election.cc:290] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.480397  1202 raft_consensus.cc:2468] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
I20260709 13:11:40.486559  1271 raft_consensus.cc:515] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.488432  1017 leader_election.cc:304] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:40.489356  1271 leader_election.cc:290] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
W20260709 13:11:40.489444  1019 leader_election.cc:343] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.490271  1363 raft_consensus.cc:2804] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:40.491456  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:40.489894  1357 raft_consensus.cc:515] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.493423  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9961aebffb649fba246b4fde922b974" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:40.494110  1202 raft_consensus.cc:3060] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.493628   949 catalog_manager.cc:5697] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:40.494911  1357 leader_election.cc:290] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.495461  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f469f24d76a42719b3c7eafec6aec8d" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:40.496104  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9961aebffb649fba246b4fde922b974" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:40.496793  1053 raft_consensus.cc:3060] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.483929  1249 leader_election.cc:290] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
W20260709 13:11:40.499884  1167 leader_election.cc:343] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.500191  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
W20260709 13:11:40.500491  1093 leader_election.cc:343] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.480032  1341 raft_consensus.cc:385] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.501509  1341 raft_consensus.cc:740] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.501705  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f469f24d76a42719b3c7eafec6aec8d" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:40.490674  1363 raft_consensus.cc:493] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.502054  1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:40.505905  1363 raft_consensus.cc:3060] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.506088  1052 raft_consensus.cc:2468] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:40.507612  1093 leader_election.cc:304] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:40.506006  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:40.508152  1341 consensus_queue.cc:260] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
W20260709 13:11:40.505193  1168 leader_election.cc:343] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.508584  1249 raft_consensus.cc:2804] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:40.509626  1168 leader_election.cc:304] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b; no voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518
I20260709 13:11:40.509821  1249 raft_consensus.cc:493] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.510437  1249 raft_consensus.cc:3060] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.511312  1271 raft_consensus.cc:2749] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:40.512094  1341 ts_tablet_manager.cc:1434] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.002s
W20260709 13:11:40.504668  1092 leader_election.cc:343] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.513024  1092 leader_election.cc:304] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 847da4898a4d460ba5696b432c782518; no voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:40.513677  1341 tablet_bootstrap.cc:492] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
W20260709 13:11:40.514889  1092 leader_election.cc:343] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.515964  1357 raft_consensus.cc:2749] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:40.521723  1202 raft_consensus.cc:2468] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:40.522531  1341 tablet_bootstrap.cc:654] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.522614  1342 tablet_bootstrap.cc:654] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.527030  1340 tablet_bootstrap.cc:492] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:40.522847  1092 leader_election.cc:304] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:40.530829  1340 ts_tablet_manager.cc:1403] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.168s	user 0.020s	sys 0.017s
I20260709 13:11:40.531241  1357 raft_consensus.cc:2804] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:40.531944  1357 raft_consensus.cc:697] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:40.533594  1357 consensus_queue.cc:237] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.524247  1050 raft_consensus.cc:2468] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 1.
I20260709 13:11:40.537222  1340 raft_consensus.cc:359] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.538831  1340 raft_consensus.cc:385] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.539374  1249 raft_consensus.cc:515] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.538847  1053 raft_consensus.cc:2468] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:40.541739  1249 leader_election.cc:290] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.548836  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:40.539690  1340 raft_consensus.cc:740] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.552349  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:40.555456  1342 tablet_bootstrap.cc:492] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:40.554764  1340 consensus_queue.cc:260] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.556082  1342 ts_tablet_manager.cc:1403] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.097s	user 0.018s	sys 0.005s
I20260709 13:11:40.559756  1342 raft_consensus.cc:359] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.560631  1342 raft_consensus.cc:385] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.561002  1342 raft_consensus.cc:740] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.562973  1050 raft_consensus.cc:3060] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:40.564152  1092 leader_election.cc:343] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.565028  1340 ts_tablet_manager.cc:1434] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.004s
I20260709 13:11:40.561751  1342 consensus_queue.cc:260] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.570444  1340 tablet_bootstrap.cc:492] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:40.572180  1342 ts_tablet_manager.cc:1434] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260709 13:11:40.573457  1342 tablet_bootstrap.cc:492] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:40.574994  1050 raft_consensus.cc:2468] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:40.576387  1093 leader_election.cc:304] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:40.577109  1249 raft_consensus.cc:2804] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:40.577613  1249 raft_consensus.cc:697] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:40.579581  1249 consensus_queue.cc:237] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.580700  1340 tablet_bootstrap.cc:654] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.583659  1341 tablet_bootstrap.cc:492] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:40.584208  1341 ts_tablet_manager.cc:1403] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.023s
I20260709 13:11:40.588579  1341 raft_consensus.cc:359] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.595115  1341 raft_consensus.cc:385] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.595431  1341 raft_consensus.cc:740] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.665130  1342 tablet_bootstrap.cc:654] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.664883  1363 raft_consensus.cc:515] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.664605  1341 consensus_queue.cc:260] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.668826  1363 leader_election.cc:290] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.670764  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7ede558daa44d0bef89e46af8fb781" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:40.671528  1202 raft_consensus.cc:3060] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.672132  1340 tablet_bootstrap.cc:492] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:40.672709  1340 ts_tablet_manager.cc:1403] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.103s	user 0.035s	sys 0.050s
I20260709 13:11:40.672770  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7ede558daa44d0bef89e46af8fb781" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:40.674602  1127 raft_consensus.cc:3060] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.678174  1340 raft_consensus.cc:359] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.678988  1340 raft_consensus.cc:385] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.679301  1340 raft_consensus.cc:740] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.680183  1340 consensus_queue.cc:260] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.686007  1340 ts_tablet_manager.cc:1434] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.005s
I20260709 13:11:40.687253  1340 tablet_bootstrap.cc:492] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:40.695525  1202 raft_consensus.cc:2468] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:40.696919  1017 leader_election.cc:304] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:40.698050  1363 raft_consensus.cc:2804] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:40.698345  1341 ts_tablet_manager.cc:1434] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.114s	user 0.019s	sys 0.002s
I20260709 13:11:40.698503  1340 tablet_bootstrap.cc:654] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.698645  1127 raft_consensus.cc:2468] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:40.701359  1341 tablet_bootstrap.cc:492] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:40.705771  1363 raft_consensus.cc:697] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:40.706908  1363 consensus_queue.cc:237] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.711010  1341 tablet_bootstrap.cc:654] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.722502  1341 tablet_bootstrap.cc:492] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:40.722997  1341 ts_tablet_manager.cc:1403] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 13:11:40.726176  1341 raft_consensus.cc:359] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.726970  1341 raft_consensus.cc:385] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.727259  1341 raft_consensus.cc:740] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.728073  1341 consensus_queue.cc:260] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.727756   948 catalog_manager.cc:5697] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:40.729735   948 catalog_manager.cc:5697] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:40.730456  1341 ts_tablet_manager.cc:1434] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:40.731807  1341 tablet_bootstrap.cc:492] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:40.733271  1342 tablet_bootstrap.cc:492] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:40.734272  1342 ts_tablet_manager.cc:1403] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.161s	user 0.022s	sys 0.048s
I20260709 13:11:40.738457  1342 raft_consensus.cc:359] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.739243  1342 raft_consensus.cc:385] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.739604  1342 raft_consensus.cc:740] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.739647   949 catalog_manager.cc:5697] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:40.740305  1340 tablet_bootstrap.cc:492] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:40.740355  1341 tablet_bootstrap.cc:654] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.741011  1340 ts_tablet_manager.cc:1403] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.054s	user 0.028s	sys 0.013s
I20260709 13:11:40.740463  1342 consensus_queue.cc:260] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.743305  1342 ts_tablet_manager.cc:1434] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:11:40.743780  1340 raft_consensus.cc:359] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.744441  1340 raft_consensus.cc:385] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.744722  1340 raft_consensus.cc:740] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.745568  1342 tablet_bootstrap.cc:492] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:40.745520  1340 consensus_queue.cc:260] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.747913  1340 ts_tablet_manager.cc:1434] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:40.749073  1340 tablet_bootstrap.cc:492] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770: Bootstrap starting.
I20260709 13:11:40.753242  1342 tablet_bootstrap.cc:654] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.754864  1340 tablet_bootstrap.cc:654] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.764986  1342 tablet_bootstrap.cc:492] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:40.765578  1342 ts_tablet_manager.cc:1403] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.007s
I20260709 13:11:40.767320  1340 tablet_bootstrap.cc:492] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770: No bootstrap required, opened a new log
I20260709 13:11:40.767822  1340 ts_tablet_manager.cc:1403] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.007s
I20260709 13:11:40.768898  1342 raft_consensus.cc:359] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.769620  1342 raft_consensus.cc:385] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.770556  1340 raft_consensus.cc:359] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.771255  1340 raft_consensus.cc:385] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.771552  1340 raft_consensus.cc:740] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.772337  1340 consensus_queue.cc:260] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.769870  1342 raft_consensus.cc:740] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.774709  1342 consensus_queue.cc:260] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.776698  1342 ts_tablet_manager.cc:1434] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:11:40.776855  1340 ts_tablet_manager.cc:1434] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:11:40.778378  1342 tablet_bootstrap.cc:492] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b: Bootstrap starting.
I20260709 13:11:40.783367  1342 tablet_bootstrap.cc:654] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.788312  1342 tablet_bootstrap.cc:492] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b: No bootstrap required, opened a new log
I20260709 13:11:40.788921  1342 ts_tablet_manager.cc:1403] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b: Time spent bootstrapping tablet: real 0.011s	user 0.002s	sys 0.007s
I20260709 13:11:40.791025  1363 raft_consensus.cc:493] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.791733  1341 tablet_bootstrap.cc:492] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:40.791525  1363 raft_consensus.cc:515] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.792291  1341 ts_tablet_manager.cc:1403] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.061s	user 0.029s	sys 0.019s
I20260709 13:11:40.792294  1342 raft_consensus.cc:359] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.793107  1342 raft_consensus.cc:385] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.793665  1363 leader_election.cc:290] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.793756  1342 raft_consensus.cc:740] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.794996  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f469f24d76a42719b3c7eafec6aec8d" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:40.795136  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f469f24d76a42719b3c7eafec6aec8d" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:40.795527  1202 raft_consensus.cc:2468] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
I20260709 13:11:40.795447  1341 raft_consensus.cc:359] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.795780  1127 raft_consensus.cc:2468] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
I20260709 13:11:40.796247  1341 raft_consensus.cc:385] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.796530  1341 raft_consensus.cc:740] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.796909  1017 leader_election.cc:304] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:40.794661  1342 consensus_queue.cc:260] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.797194  1341 consensus_queue.cc:260] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.799492  1342 ts_tablet_manager.cc:1434] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:11:40.797050  1271 raft_consensus.cc:493] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.799585  1341 ts_tablet_manager.cc:1434] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:40.799969  1271 raft_consensus.cc:515] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.800624  1341 tablet_bootstrap.cc:492] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518: Bootstrap starting.
I20260709 13:11:40.802248  1271 leader_election.cc:290] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:40.803102  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:40.803290  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:40.803790  1050 raft_consensus.cc:2468] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 0.
W20260709 13:11:40.804806  1168 leader_election.cc:343] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.805613  1167 leader_election.cc:304] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 847da4898a4d460ba5696b432c782518
I20260709 13:11:40.805922  1363 raft_consensus.cc:2804] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:40.806347  1363 raft_consensus.cc:493] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.806972  1341 tablet_bootstrap.cc:654] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:40.806695  1363 raft_consensus.cc:3060] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.810704  1271 raft_consensus.cc:2804] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:40.811226  1271 raft_consensus.cc:493] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.811681  1271 raft_consensus.cc:3060] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.816838  1363 raft_consensus.cc:515] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.819334  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f469f24d76a42719b3c7eafec6aec8d" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:40.819887  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f469f24d76a42719b3c7eafec6aec8d" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:40.820219  1127 raft_consensus.cc:3060] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.820477  1202 raft_consensus.cc:3060] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.826818  1202 raft_consensus.cc:2468] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:40.827801  1017 leader_election.cc:304] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:40.828670  1362 raft_consensus.cc:2804] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:40.829099  1362 raft_consensus.cc:697] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:40.829901  1362 consensus_queue.cc:237] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.835364  1363 leader_election.cc:290] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.838599  1127 raft_consensus.cc:2468] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:40.844579  1271 raft_consensus.cc:515] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.846868  1341 tablet_bootstrap.cc:492] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518: No bootstrap required, opened a new log
I20260709 13:11:40.847347  1341 ts_tablet_manager.cc:1403] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.031s
I20260709 13:11:40.846743   947 catalog_manager.cc:5697] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:40.847407  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:40.848055  1050 raft_consensus.cc:3060] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.848176  1271 leader_election.cc:290] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:40.849537  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
W20260709 13:11:40.850829  1168 leader_election.cc:343] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:40.850827  1341 raft_consensus.cc:359] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.851744  1341 raft_consensus.cc:385] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:40.851996  1341 raft_consensus.cc:740] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Initialized, Role: FOLLOWER
I20260709 13:11:40.852653  1341 consensus_queue.cc:260] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.854501  1341 ts_tablet_manager.cc:1434] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:11:40.857379  1050 raft_consensus.cc:2468] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 1.
I20260709 13:11:40.858668  1167 leader_election.cc:304] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 847da4898a4d460ba5696b432c782518
I20260709 13:11:40.859429  1271 raft_consensus.cc:2804] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:40.859877  1362 raft_consensus.cc:493] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.860348  1362 raft_consensus.cc:515] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.861379  1271 raft_consensus.cc:697] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:40.862519  1362 leader_election.cc:290] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.862320  1271 consensus_queue.cc:237] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.864388  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:40.864979  1202 raft_consensus.cc:2468] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
I20260709 13:11:40.865386  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:40.866014  1127 raft_consensus.cc:2468] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 0.
I20260709 13:11:40.866101  1017 leader_election.cc:304] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:40.866938  1362 raft_consensus.cc:2804] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:40.867290  1362 raft_consensus.cc:493] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.867657  1362 raft_consensus.cc:3060] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.872646  1362 raft_consensus.cc:515] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.875293  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:40.875857  1127 raft_consensus.cc:3060] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.878829  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:40.879355  1202 raft_consensus.cc:3060] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.879448   947 catalog_manager.cc:5697] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 1 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:40.881537  1362 leader_election.cc:290] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.882766  1127 raft_consensus.cc:2468] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:40.883718  1019 leader_election.cc:304] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:40.884387  1362 raft_consensus.cc:2804] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:40.884933  1362 raft_consensus.cc:697] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:40.885571  1202 raft_consensus.cc:2468] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 1.
I20260709 13:11:40.885774  1362 consensus_queue.cc:237] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.892194  1249 raft_consensus.cc:493] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.892875  1249 raft_consensus.cc:515] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.894784  1249 leader_election.cc:290] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.894495   947 catalog_manager.cc:5697] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:40.896158  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:40.896538  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:40.896919  1053 raft_consensus.cc:2468] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:40.897188  1202 raft_consensus.cc:2468] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:40.898345  1093 leader_election.cc:304] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:40.899222  1249 raft_consensus.cc:2804] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:40.899605  1249 raft_consensus.cc:493] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.899884  1249 raft_consensus.cc:3060] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.905488  1249 raft_consensus.cc:515] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.906726  1249 leader_election.cc:290] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:40.907678  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:40.907725  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:40.908347  1050 raft_consensus.cc:3060] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.908407  1202 raft_consensus.cc:3060] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.912915  1202 raft_consensus.cc:2468] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:40.913920  1092 leader_election.cc:304] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:40.914651  1050 raft_consensus.cc:2468] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:40.917819  1249 raft_consensus.cc:2804] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:40.918375  1249 raft_consensus.cc:697] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:40.919108  1249 consensus_queue.cc:237] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.926535   947 catalog_manager.cc:5697] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:40.960402  1271 raft_consensus.cc:493] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.960924  1271 raft_consensus.cc:515] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:40.963266  1271 leader_election.cc:290] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:40.963548  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:40.964054  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847da4898a4d460ba5696b432c782518" is_pre_election: true
I20260709 13:11:40.964437  1053 raft_consensus.cc:2393] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5fd41f935e3446228d999b210c2a4e2b in current term 1: Already voted for candidate 847da4898a4d460ba5696b432c782518 in this term.
I20260709 13:11:40.965598  1249 consensus_queue.cc:1048] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:40.965713  1168 leader_election.cc:304] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b; no voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518
I20260709 13:11:40.966521  1271 raft_consensus.cc:3060] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:40.973417  1249 consensus_queue.cc:1048] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:40.978554  1271 raft_consensus.cc:2749] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:40.984655  1358 consensus_queue.cc:1048] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:41.003067  1357 raft_consensus.cc:493] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.003538  1357 raft_consensus.cc:515] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.005594  1357 leader_election.cc:290] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:41.007738  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e53a48fe1d466bb54526b0a428ff31" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:41.008478  1050 raft_consensus.cc:2468] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:41.008885  1358 consensus_queue.cc:1048] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.009660  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e53a48fe1d466bb54526b0a428ff31" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:41.010278  1201 raft_consensus.cc:2468] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 0.
I20260709 13:11:41.009660  1093 leader_election.cc:304] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:41.011766  1356 raft_consensus.cc:2804] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:41.012108  1356 raft_consensus.cc:493] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.012450  1356 raft_consensus.cc:3060] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:41.023206  1356 raft_consensus.cc:515] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.024974  1356 leader_election.cc:290] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:41.025700  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e53a48fe1d466bb54526b0a428ff31" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:41.025960  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e53a48fe1d466bb54526b0a428ff31" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:41.026235  1050 raft_consensus.cc:3060] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:41.026562  1202 raft_consensus.cc:3060] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:41.031925  1050 raft_consensus.cc:2468] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:41.033041  1202 raft_consensus.cc:2468] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 1.
I20260709 13:11:41.033273  1093 leader_election.cc:304] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:41.034400  1357 consensus_queue.cc:1048] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.035687  1357 raft_consensus.cc:2804] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:41.036161  1357 raft_consensus.cc:697] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:41.036983  1357 consensus_queue.cc:237] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
W20260709 13:11:41.039918   999 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:41.046823   948 catalog_manager.cc:5697] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 0 to 1, leader changed from <none> to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 1 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:41.048570  1356 consensus_queue.cc:1048] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.141158  1362 consensus_queue.cc:1048] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.151165  1402 consensus_queue.cc:1048] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.205538  1127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9961aebffb649fba246b4fde922b974"
dest_uuid: "847da4898a4d460ba5696b432c782518"
mode: GRACEFUL
new_leader_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:43306
I20260709 13:11:41.206319  1127 raft_consensus.cc:606] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Received request to transfer leadership to TS 49712d6ed31f4f4bacede77e3047f770
I20260709 13:11:41.214820  1202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c9a41e7f37a4d98b189476d05d822d9"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
mode: GRACEFUL
new_leader_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:58238
I20260709 13:11:41.215415  1202 raft_consensus.cc:606] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Received request to transfer leadership to TS 49712d6ed31f4f4bacede77e3047f770
I20260709 13:11:41.220364  1127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e"
dest_uuid: "847da4898a4d460ba5696b432c782518"
mode: GRACEFUL
new_leader_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:43306
I20260709 13:11:41.220870  1127 raft_consensus.cc:606] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Received request to transfer leadership to TS 49712d6ed31f4f4bacede77e3047f770
I20260709 13:11:41.226531  1202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
mode: GRACEFUL
new_leader_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:58238
I20260709 13:11:41.226974  1202 raft_consensus.cc:606] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Received request to transfer leadership to TS 847da4898a4d460ba5696b432c782518
I20260709 13:11:41.231941  1050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f7ede558daa44d0bef89e46af8fb781"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
mode: GRACEFUL
new_leader_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:48028
I20260709 13:11:41.232420  1050 raft_consensus.cc:606] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Received request to transfer leadership to TS 847da4898a4d460ba5696b432c782518
I20260709 13:11:41.240401  1050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
mode: GRACEFUL
new_leader_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:48028
I20260709 13:11:41.240846  1050 raft_consensus.cc:606] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Received request to transfer leadership to TS 847da4898a4d460ba5696b432c782518
I20260709 13:11:41.245258  1127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504"
dest_uuid: "847da4898a4d460ba5696b432c782518"
mode: GRACEFUL
new_leader_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:43306
I20260709 13:11:41.245779  1127 raft_consensus.cc:606] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Received request to transfer leadership to TS 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:41.250224  1402 consensus_queue.cc:1048] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.261399  1363 consensus_queue.cc:1048] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.322400  1363 consensus_queue.cc:1048] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.329396  1402 raft_consensus.cc:993] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770: : Instructing follower 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:41.329885  1362 raft_consensus.cc:1081] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Signalling peer 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:41.331164  1356 raft_consensus.cc:493] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.331686  1356 raft_consensus.cc:515] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.333032  1128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3"
dest_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:43302
I20260709 13:11:41.333671  1128 raft_consensus.cc:493] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:41.333985  1128 raft_consensus.cc:3060] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.334633  1356 leader_election.cc:290] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:41.335317  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49712d6ed31f4f4bacede77e3047f770" is_pre_election: true
I20260709 13:11:41.336109  1050 raft_consensus.cc:2393] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 847da4898a4d460ba5696b432c782518 in current term 1: Already voted for candidate 5fd41f935e3446228d999b210c2a4e2b in this term.
I20260709 13:11:41.337265  1363 consensus_queue.cc:1048] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.342659  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd41f935e3446228d999b210c2a4e2b" is_pre_election: true
I20260709 13:11:41.344267  1092 leader_election.cc:304] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 847da4898a4d460ba5696b432c782518; no voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:41.345326  1356 raft_consensus.cc:3060] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:41.346122  1128 raft_consensus.cc:515] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.353484  1128 leader_election.cc:290] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:41.354406  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:41.355500  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:41.355748  1356 raft_consensus.cc:2749] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:41.356087  1050 raft_consensus.cc:3055] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.356398  1050 raft_consensus.cc:740] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:41.357227  1050 consensus_queue.cc:260] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.358132  1050 raft_consensus.cc:3060] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:41.358068  1363 consensus_queue.cc:1175] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5fd41f935e3446228d999b210c2a4e2b" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:11:41.363252  1050 raft_consensus.cc:2468] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:41.364276  1093 leader_election.cc:304] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:41.364980  1356 raft_consensus.cc:2804] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.365491  1356 raft_consensus.cc:697] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:41.366173  1356 consensus_queue.cc:237] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.372146   947 catalog_manager.cc:5697] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 1 to 2, leader changed from 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65) to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:41.407083  1249 raft_consensus.cc:993] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518: : Instructing follower 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:41.407559  1356 raft_consensus.cc:1081] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Signalling peer 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:41.409025  1358 consensus_queue.cc:1048] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.409287  1050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9961aebffb649fba246b4fde922b974"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:48018
I20260709 13:11:41.409955  1050 raft_consensus.cc:493] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:41.410367  1050 raft_consensus.cc:3060] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.420403  1050 raft_consensus.cc:515] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.422914  1050 leader_election.cc:290] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:41.422766  1396 consensus_queue.cc:1048] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.424633  1202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9961aebffb649fba246b4fde922b974" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:41.424751  1128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9961aebffb649fba246b4fde922b974" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:41.425349  1202 raft_consensus.cc:3060] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.425499  1128 raft_consensus.cc:3055] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.425940  1128 raft_consensus.cc:740] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:41.426695  1128 consensus_queue.cc:260] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.427975  1128 raft_consensus.cc:3060] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.434593  1396 raft_consensus.cc:993] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b: : Instructing follower 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:41.435132  1409 raft_consensus.cc:1081] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Signalling peer 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:41.435979  1202 raft_consensus.cc:2468] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:41.437350  1017 leader_election.cc:304] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:41.438202  1362 raft_consensus.cc:2804] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.438323  1127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2"
dest_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:43288
I20260709 13:11:41.438732  1362 raft_consensus.cc:697] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:41.438937  1127 raft_consensus.cc:493] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:41.439343  1127 raft_consensus.cc:3060] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.439594  1362 consensus_queue.cc:237] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.445569  1128 raft_consensus.cc:2468] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:41.447675  1127 raft_consensus.cc:515] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.450356  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:41.450910  1050 raft_consensus.cc:3060] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.451375  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a64ea3118f743858b116fa4c3d463a2" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:41.451812  1201 raft_consensus.cc:3055] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.452025  1201 raft_consensus.cc:740] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:41.452651  1201 consensus_queue.cc:260] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.453459  1201 raft_consensus.cc:3060] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.450439  1127 leader_election.cc:290] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:41.458353  1050 raft_consensus.cc:2468] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:41.459375  1356 raft_consensus.cc:993] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518: : Instructing follower 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:41.459239  1093 leader_election.cc:304] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:41.459827  1356 raft_consensus.cc:1081] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Signalling peer 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:41.460255  1357 raft_consensus.cc:2804] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.460549  1201 raft_consensus.cc:2468] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:41.460700  1357 raft_consensus.cc:697] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:41.461886  1050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:48018
I20260709 13:11:41.461757  1357 consensus_queue.cc:237] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.462373  1050 raft_consensus.cc:493] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:41.460646   948 catalog_manager.cc:5697] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 1 to 2, leader changed from 847da4898a4d460ba5696b432c782518 (127.21.31.66) to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:41.462747  1050 raft_consensus.cc:3060] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.468786  1357 consensus_queue.cc:1048] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.469926  1050 raft_consensus.cc:515] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.472188  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:41.472445  1050 leader_election.cc:290] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:41.472761  1127 raft_consensus.cc:3055] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.473160  1127 raft_consensus.cc:740] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:41.473457  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9023ef2e63914bc39db6608de9152c5e" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:41.474071  1201 raft_consensus.cc:3060] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.474009  1127 consensus_queue.cc:260] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.475549  1127 raft_consensus.cc:3060] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.482565  1127 raft_consensus.cc:2468] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:41.482901  1201 raft_consensus.cc:2468] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:41.484468  1017 leader_election.cc:304] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:41.484129   947 catalog_manager.cc:5697] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 1 to 2, leader changed from 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67) to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:41.485405  1363 raft_consensus.cc:2804] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.485888  1363 raft_consensus.cc:697] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:41.486801  1363 consensus_queue.cc:237] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.489859  1249 consensus_queue.cc:1048] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.495299  1249 consensus_queue.cc:1048] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.508567  1414 consensus_queue.cc:1048] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:41.512663   948 catalog_manager.cc:5697] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 1 to 2, leader changed from 847da4898a4d460ba5696b432c782518 (127.21.31.66) to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:41.519825  1357 raft_consensus.cc:993] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518: : Instructing follower 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:41.520776  1414 raft_consensus.cc:1081] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Signalling peer 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:41.524200  1201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:58226
I20260709 13:11:41.524855  1201 raft_consensus.cc:493] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:41.525218  1201 raft_consensus.cc:3060] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.532285  1201 raft_consensus.cc:515] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.533985  1201 leader_election.cc:290] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 2 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:41.535012  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:41.535621  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18afa9b1a0c4db8a959a2ecac5ce504" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:41.535658  1127 raft_consensus.cc:3055] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.536228  1127 raft_consensus.cc:740] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:41.537022  1127 consensus_queue.cc:260] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.538311  1127 raft_consensus.cc:3060] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.541652  1053 raft_consensus.cc:3060] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.544552  1127 raft_consensus.cc:2468] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 2.
I20260709 13:11:41.546276  1168 leader_election.cc:304] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:41.547091  1396 raft_consensus.cc:2804] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.547529  1396 raft_consensus.cc:697] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 2 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:41.548622  1053 raft_consensus.cc:2468] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 2.
I20260709 13:11:41.548391  1396 consensus_queue.cc:237] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.555503   948 catalog_manager.cc:5697] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 1 to 2, leader changed from 847da4898a4d460ba5696b432c782518 (127.21.31.66) to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 2 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:41.567884  1358 raft_consensus.cc:993] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b: : Instructing follower 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:41.568329  1396 raft_consensus.cc:1081] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Signalling peer 49712d6ed31f4f4bacede77e3047f770 to start an election
I20260709 13:11:41.569800  1050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c9a41e7f37a4d98b189476d05d822d9"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
 from {username='slave'} at 127.0.0.1:48002
I20260709 13:11:41.570307  1050 raft_consensus.cc:493] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:41.570622  1050 raft_consensus.cc:3060] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.574747  1050 raft_consensus.cc:515] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.576041  1050 leader_election.cc:290] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Requested vote from peers 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:41.577221  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c9a41e7f37a4d98b189476d05d822d9" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:41.577428  1127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c9a41e7f37a4d98b189476d05d822d9" candidate_uuid: "49712d6ed31f4f4bacede77e3047f770" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:41.577853  1201 raft_consensus.cc:3055] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.578233  1201 raft_consensus.cc:740] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:41.578287  1127 raft_consensus.cc:3060] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.578930  1201 consensus_queue.cc:260] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.580044  1201 raft_consensus.cc:3060] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.583494  1127 raft_consensus.cc:2468] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:41.584698  1201 raft_consensus.cc:2468] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49712d6ed31f4f4bacede77e3047f770 in term 2.
I20260709 13:11:41.584698  1019 leader_election.cc:304] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:41.585978  1402 raft_consensus.cc:2804] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.586503  1402 raft_consensus.cc:697] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Becoming Leader. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:41.587469  1402 consensus_queue.cc:237] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.594189   948 catalog_manager.cc:5697] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 reported cstate change: term changed from 1 to 2, leader changed from 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67) to 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "49712d6ed31f4f4bacede77e3047f770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:41.684845  1362 raft_consensus.cc:993] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770: : Instructing follower 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:41.685448  1402 raft_consensus.cc:1081] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Signalling peer 847da4898a4d460ba5696b432c782518 to start an election
I20260709 13:11:41.687081  1127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f7ede558daa44d0bef89e46af8fb781"
dest_uuid: "847da4898a4d460ba5696b432c782518"
 from {username='slave'} at 127.0.0.1:43302
I20260709 13:11:41.687556  1127 raft_consensus.cc:493] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:41.687845  1127 raft_consensus.cc:3060] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.692554  1127 raft_consensus.cc:515] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.694011  1127 leader_election.cc:290] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165)
I20260709 13:11:41.694975  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7ede558daa44d0bef89e46af8fb781" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:41.695295  1201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7ede558daa44d0bef89e46af8fb781" candidate_uuid: "847da4898a4d460ba5696b432c782518" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
I20260709 13:11:41.695771  1050 raft_consensus.cc:3055] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.695843  1201 raft_consensus.cc:3060] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.696426  1050 raft_consensus.cc:740] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:41.697232  1050 consensus_queue.cc:260] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.698393  1050 raft_consensus.cc:3060] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.702791  1201 raft_consensus.cc:2468] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:41.704013  1092 leader_election.cc:304] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:41.704670  1050 raft_consensus.cc:2468] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847da4898a4d460ba5696b432c782518 in term 2.
I20260709 13:11:41.704797  1357 raft_consensus.cc:2804] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.705497  1357 raft_consensus.cc:697] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Becoming Leader. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:41.706233  1357 consensus_queue.cc:237] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:41.712828   948 catalog_manager.cc:5697] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 reported cstate change: term changed from 1 to 2, leader changed from 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65) to 847da4898a4d460ba5696b432c782518 (127.21.31.66). New cstate: current_term: 2 leader_uuid: "847da4898a4d460ba5696b432c782518" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:41.746099  1201 raft_consensus.cc:3060] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.750754  1201 raft_consensus.cc:1275] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.751883  1357 consensus_queue.cc:1048] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:41.762895  1050 raft_consensus.cc:1275] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.764290  1356 consensus_queue.cc:1048] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:41.882303  1127 raft_consensus.cc:1275] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.883471  1363 consensus_queue.cc:1048] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:41.890846  1202 raft_consensus.cc:1275] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.891981  1363 consensus_queue.cc:1048] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:41.894901  1127 raft_consensus.cc:1275] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.896245  1363 consensus_queue.cc:1048] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:41.899545  1202 raft_consensus.cc:1275] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.900897  1363 consensus_queue.cc:1048] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:41.919123  1202 raft_consensus.cc:1275] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.920342  1356 consensus_queue.cc:1048] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:41.937036  1050 raft_consensus.cc:1275] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.939914  1357 consensus_queue.cc:1048] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:42.048532  1127 raft_consensus.cc:1275] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:42.050062  1358 consensus_queue.cc:1048] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:42.058374  1053 raft_consensus.cc:1275] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:42.060189  1396 consensus_queue.cc:1048] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:42.068473  1127 raft_consensus.cc:1275] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:42.069705  1416 consensus_queue.cc:1048] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:42.077941  1202 raft_consensus.cc:1275] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 49712d6ed31f4f4bacede77e3047f770: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:42.079870  1362 consensus_queue.cc:1048] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:42.144289  1053 raft_consensus.cc:1275] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:42.145687  1357 consensus_queue.cc:1048] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:42.154747  1202 raft_consensus.cc:1275] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Refusing update from remote peer 847da4898a4d460ba5696b432c782518: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:42.156733  1249 consensus_queue.cc:1048] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:44.249343   947 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58442:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:11:44.269447   947 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58442:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:11:44.404466 21629 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:11:44.520684  1449 log.cc:927] Time spent T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
W20260709 13:11:44.525776  1450 log.cc:927] Time spent T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:11:44.539448  1462 tablet.cc:1724] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:44.555671  1448 log.cc:927] Time spent T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:11:44.565599  1228 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:9f469f24d76a42719b3c7eafec6aec8d)
W20260709 13:11:44.587039  1228 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:9f469f24d76a42719b3c7eafec6aec8d)
W20260709 13:11:44.589192  1228 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:9f469f24d76a42719b3c7eafec6aec8d)
I20260709 13:11:44.595122  1465 tablet.cc:1724] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.615747  1467 tablet.cc:1724] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:44.645051  1017 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.31.67:40165, user_credentials={real_user=slave}} blocked reactor thread for 80686.3us
I20260709 13:11:44.645928  1478 tablet.cc:1724] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.613867  1461 tablet.cc:1724] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.741217  1484 tablet.cc:1724] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.747347  1486 tablet.cc:1724] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.755421  1053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9961aebffb649fba246b4fde922b974"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
mode: GRACEFUL
new_leader_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:35702
I20260709 13:11:44.756140  1053 raft_consensus.cc:606] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Received request to transfer leadership to TS 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:44.771723  1421 raft_consensus.cc:993] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770: : Instructing follower 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:44.772367  1421 raft_consensus.cc:1081] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Signalling peer 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:44.775766  1201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9961aebffb649fba246b4fde922b974"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:58216
I20260709 13:11:44.776353  1201 raft_consensus.cc:493] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:44.776698  1201 raft_consensus.cc:3060] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.768622  1487 tablet.cc:1724] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.750757  1465 tablet.cc:1724] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.791757  1201 raft_consensus.cc:515] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:44.796671  1201 leader_election.cc:290] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 3 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:44.797851  1490 tablet.cc:1724] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.782572  1467 tablet.cc:1724] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.802762  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9961aebffb649fba246b4fde922b974" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:44.813419  1053 raft_consensus.cc:3055] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:44.814670  1053 raft_consensus.cc:740] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:44.815608  1053 consensus_queue.cc:260] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:44.816740  1053 raft_consensus.cc:3060] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.824833  1488 tablet.cc:1724] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.797304  1462 tablet.cc:1724] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.834786  1461 tablet.cc:1724] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.824710  1478 tablet.cc:1724] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.827555  1489 tablet.cc:1724] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.829627  1486 tablet.cc:1724] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.861181  1487 tablet.cc:1724] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.830678  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9961aebffb649fba246b4fde922b974" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:44.862452  1126 raft_consensus.cc:3060] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.867149  1462 tablet.cc:1724] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.869903  1126 raft_consensus.cc:2468] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 3.
I20260709 13:11:44.871177  1168 leader_election.cc:304] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:44.872040  1477 raft_consensus.cc:2804] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:44.872481  1477 raft_consensus.cc:697] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 3 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:44.872726  1488 tablet.cc:1724] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.873348  1477 consensus_queue.cc:237] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:44.874503  1125 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3"
dest_uuid: "847da4898a4d460ba5696b432c782518"
mode: GRACEFUL
new_leader_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:51796
I20260709 13:11:44.875101  1125 raft_consensus.cc:606] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Received request to transfer leadership to TS 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:44.877223 21629 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:11:44.877868 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:44.850365  1465 tablet.cc:1724] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.855813  1053 raft_consensus.cc:2468] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 3.
I20260709 13:11:44.808893  1492 tablet.cc:1724] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.830194  1484 tablet.cc:1724] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.879307  1490 tablet.cc:1724] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.897660  1461 tablet.cc:1724] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.900023   948 catalog_manager.cc:5697] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 2 to 3, leader changed from 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65) to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 3 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:44.901909   948 catalog_manager.cc:5789] TS 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67:40165) does not have the latest schema for tablet c9961aebffb649fba246b4fde922b974 (table soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15]). Expected version 1 got 0
I20260709 13:11:44.910960  1488 tablet.cc:1724] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.916388  1053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19"
dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
mode: GRACEFUL
new_leader_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:35702
I20260709 13:11:44.917982  1053 raft_consensus.cc:606] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Received request to transfer leadership to TS 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:44.913733  1490 tablet.cc:1724] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:44.932348  1126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a551497d48ca4ed58899428085203dc8"
dest_uuid: "847da4898a4d460ba5696b432c782518"
mode: GRACEFUL
new_leader_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:51796
I20260709 13:11:44.932924  1126 raft_consensus.cc:606] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Received request to transfer leadership to TS 5fd41f935e3446228d999b210c2a4e2b
I20260709 13:11:44.934551 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:11:44.934983 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:44.941583   950 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13: soft_deleted_table [id=cb7a3f701bf54f8b9b7a0953a4f8fb15] alter complete (version 1)
I20260709 13:11:45.198727  1454 raft_consensus.cc:993] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518: : Instructing follower 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:45.199334  1456 raft_consensus.cc:1081] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Signalling peer 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:45.201061  1198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:58226
I20260709 13:11:45.201866  1198 raft_consensus.cc:493] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:45.202157  1198 raft_consensus.cc:3060] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.208407  1198 raft_consensus.cc:515] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:45.210479  1198 leader_election.cc:290] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 3 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:45.211167  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:45.211776  1053 raft_consensus.cc:3060] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.211355  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd22a8edd9b143608ee50df5b9d447f3" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:45.212312  1126 raft_consensus.cc:3055] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.212591  1126 raft_consensus.cc:740] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:45.213480  1126 consensus_queue.cc:260] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:45.214648  1126 raft_consensus.cc:3060] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.217664  1053 raft_consensus.cc:2468] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 3.
I20260709 13:11:45.218685  1167 leader_election.cc:304] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:45.219681  1358 raft_consensus.cc:2804] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:45.220062  1126 raft_consensus.cc:2468] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 3.
I20260709 13:11:45.220258  1358 raft_consensus.cc:697] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 3 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:45.221431  1358 consensus_queue.cc:237] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:45.230712   950 catalog_manager.cc:5697] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 2 to 3, leader changed from 847da4898a4d460ba5696b432c782518 (127.21.31.66) to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 3 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:45.270743  1475 raft_consensus.cc:993] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770: : Instructing follower 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:45.271263  1440 raft_consensus.cc:1081] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Signalling peer 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:45.272737  1198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:58216
I20260709 13:11:45.273180  1198 raft_consensus.cc:493] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:45.273499  1198 raft_consensus.cc:3060] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.278385  1198 raft_consensus.cc:515] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:45.280023  1198 leader_election.cc:290] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 3 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:45.281075  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:45.281092  1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423005aa03b442348db45ed6e4b4be19" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:45.281999  1126 raft_consensus.cc:3060] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.282115  1053 raft_consensus.cc:3055] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.282697  1053 raft_consensus.cc:740] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49712d6ed31f4f4bacede77e3047f770, State: Running, Role: LEADER
I20260709 13:11:45.283540  1053 consensus_queue.cc:260] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:45.284766  1053 raft_consensus.cc:3060] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.289003  1126 raft_consensus.cc:2468] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 3.
I20260709 13:11:45.289994  1168 leader_election.cc:304] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd41f935e3446228d999b210c2a4e2b, 847da4898a4d460ba5696b432c782518; no voters: 
I20260709 13:11:45.290606  1053 raft_consensus.cc:2468] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 3.
I20260709 13:11:45.290784  1358 raft_consensus.cc:2804] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:45.291359  1358 raft_consensus.cc:697] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 3 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:45.292176  1358 consensus_queue.cc:237] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } }
I20260709 13:11:45.293740  1053 raft_consensus.cc:1275] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:45.295195  1477 consensus_queue.cc:1048] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:45.302441   950 catalog_manager.cc:5697] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 2 to 3, leader changed from 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65) to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 3 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:45.321925  1126 raft_consensus.cc:1275] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:45.323607  1502 consensus_queue.cc:1048] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:45.429280  1456 raft_consensus.cc:993] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518: : Instructing follower 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:45.429764  1454 raft_consensus.cc:1081] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Signalling peer 5fd41f935e3446228d999b210c2a4e2b to start an election
I20260709 13:11:45.431414  1198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a551497d48ca4ed58899428085203dc8"
dest_uuid: "5fd41f935e3446228d999b210c2a4e2b"
 from {username='slave'} at 127.0.0.1:58226
I20260709 13:11:45.432008  1198 raft_consensus.cc:493] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:45.432405  1198 raft_consensus.cc:3060] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.438952  1198 raft_consensus.cc:515] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:45.440491  1198 leader_election.cc:290] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 2 election: Requested vote from peers 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467), 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379)
I20260709 13:11:45.441146  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a551497d48ca4ed58899428085203dc8" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49712d6ed31f4f4bacede77e3047f770"
I20260709 13:11:45.441497  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a551497d48ca4ed58899428085203dc8" candidate_uuid: "5fd41f935e3446228d999b210c2a4e2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "847da4898a4d460ba5696b432c782518"
I20260709 13:11:45.441774  1050 raft_consensus.cc:3060] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.442175  1126 raft_consensus.cc:3055] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:45.442487  1126 raft_consensus.cc:740] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 847da4898a4d460ba5696b432c782518, State: Running, Role: LEADER
I20260709 13:11:45.443176  1126 consensus_queue.cc:260] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:45.444113  1126 raft_consensus.cc:3060] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.446465  1050 raft_consensus.cc:2468] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 2.
I20260709 13:11:45.447330  1167 leader_election.cc:304] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49712d6ed31f4f4bacede77e3047f770, 5fd41f935e3446228d999b210c2a4e2b; no voters: 
I20260709 13:11:45.447988  1502 raft_consensus.cc:2804] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:45.448451  1502 raft_consensus.cc:697] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 2 LEADER]: Becoming Leader. State: Replica: 5fd41f935e3446228d999b210c2a4e2b, State: Running, Role: LEADER
I20260709 13:11:45.449296  1126 raft_consensus.cc:2468] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd41f935e3446228d999b210c2a4e2b in term 2.
I20260709 13:11:45.449285  1502 consensus_queue.cc:237] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } }
I20260709 13:11:45.460493   948 catalog_manager.cc:5697] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b reported cstate change: term changed from 1 to 2, leader changed from 847da4898a4d460ba5696b432c782518 (127.21.31.66) to 5fd41f935e3446228d999b210c2a4e2b (127.21.31.67). New cstate: current_term: 2 leader_uuid: "5fd41f935e3446228d999b210c2a4e2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fd41f935e3446228d999b210c2a4e2b" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 40165 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:45.681005  1126 raft_consensus.cc:1275] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:45.682531  1477 consensus_queue.cc:1048] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:45.689766  1050 raft_consensus.cc:1275] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:45.696097  1502 consensus_queue.cc:1048] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:45.725504  1126 raft_consensus.cc:1275] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:45.726668  1502 consensus_queue.cc:1048] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:45.736598  1050 raft_consensus.cc:1275] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:45.737773  1506 consensus_queue.cc:1048] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:45.830919  1126 raft_consensus.cc:1275] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:45.831986  1477 consensus_queue.cc:1048] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "847da4898a4d460ba5696b432c782518" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:45.841792  1050 raft_consensus.cc:1275] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Refusing update from remote peer 5fd41f935e3446228d999b210c2a4e2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:45.843097  1358 consensus_queue.cc:1048] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49712d6ed31f4f4bacede77e3047f770" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:45.996472 21629 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:11:46.002323 21629 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:11:46.002750 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:46.007735 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:11:46.008369 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:47.040866 21629 tablet_server.cc:179] TabletServer@127.21.31.65:0 shutting down...
W20260709 13:11:47.070123  1167 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
I20260709 13:11:47.072117 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:47.072983 21629 tablet_replica.cc:333] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.073683 21629 raft_consensus.cc:2243] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.074361 21629 raft_consensus.cc:2272] T 4a64ea3118f743858b116fa4c3d463a2 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:47.074430  1167 consensus_peers.cc:597] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b -> Peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Couldn't send request to peer 49712d6ed31f4f4bacede77e3047f770. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.077186 21629 tablet_replica.cc:333] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.078125 21629 raft_consensus.cc:2243] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.078761 21629 raft_consensus.cc:2272] T f18afa9b1a0c4db8a959a2ecac5ce504 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.081143 21629 tablet_replica.cc:333] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.081924 21629 raft_consensus.cc:2243] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.082444 21629 raft_consensus.cc:2272] T cd22a8edd9b143608ee50df5b9d447f3 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.084442 21629 tablet_replica.cc:333] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.085057 21629 raft_consensus.cc:2243] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.086040 21629 raft_consensus.cc:2272] T 9f469f24d76a42719b3c7eafec6aec8d P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.088208 21629 tablet_replica.cc:333] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.088775 21629 raft_consensus.cc:2243] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.089522 21629 raft_consensus.cc:2272] T 9023ef2e63914bc39db6608de9152c5e P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.091445 21629 tablet_replica.cc:333] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.091990 21629 raft_consensus.cc:2243] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.092419 21629 raft_consensus.cc:2272] T 0f7ede558daa44d0bef89e46af8fb781 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.094237 21629 tablet_replica.cc:333] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.094734 21629 raft_consensus.cc:2243] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.095165 21629 raft_consensus.cc:2272] T f7e53a48fe1d466bb54526b0a428ff31 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.096944 21629 tablet_replica.cc:333] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.097535 21629 raft_consensus.cc:2243] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.098310 21629 raft_consensus.cc:2272] T b271932cab664b5196fb024cd0b0ee4a P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.100538 21629 tablet_replica.cc:333] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.101095 21629 raft_consensus.cc:2243] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.101832 21629 raft_consensus.cc:2272] T 5c9a41e7f37a4d98b189476d05d822d9 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.103770 21629 tablet_replica.cc:333] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.104437 21629 raft_consensus.cc:2243] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.104876 21629 raft_consensus.cc:2272] T c9961aebffb649fba246b4fde922b974 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.106875 21629 tablet_replica.cc:333] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.107398 21629 raft_consensus.cc:2243] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.107823 21629 raft_consensus.cc:2272] T a64795fb830e437093b08e6d69e366eb P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.109839 21629 tablet_replica.cc:333] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.110353 21629 raft_consensus.cc:2243] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.110803 21629 raft_consensus.cc:2272] T c0b3f2617ffe4e269fccaf988eb27805 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.112777 21629 tablet_replica.cc:333] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.113281 21629 raft_consensus.cc:2243] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.114362 21629 raft_consensus.cc:2272] T 568365befae0431a846444a9414c0e69 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.116827 21629 tablet_replica.cc:333] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.117733 21629 raft_consensus.cc:2243] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.118767 21629 raft_consensus.cc:2272] T 8f1b8fab7cd84b96bbb298fc00003204 P 49712d6ed31f4f4bacede77e3047f770 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:47.119632  1093 consensus_peers.cc:597] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 -> Peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Couldn't send request to peer 49712d6ed31f4f4bacede77e3047f770. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.121608 21629 tablet_replica.cc:333] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.122195 21629 raft_consensus.cc:2243] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.122630 21629 raft_consensus.cc:2272] T a069639d84a14cc8b8478a31a7bfac22 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.124729 21629 tablet_replica.cc:333] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.125207 21629 raft_consensus.cc:2243] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.125662 21629 raft_consensus.cc:2272] T 423005aa03b442348db45ed6e4b4be19 P 49712d6ed31f4f4bacede77e3047f770 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.128028 21629 tablet_replica.cc:333] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.128526 21629 raft_consensus.cc:2243] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.129024 21629 raft_consensus.cc:2272] T a551497d48ca4ed58899428085203dc8 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.130931 21629 tablet_replica.cc:333] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770: stopping tablet replica
I20260709 13:11:47.131410 21629 raft_consensus.cc:2243] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.131827 21629 raft_consensus.cc:2272] T 7882c3267d5e42df857e4760dec02397 P 49712d6ed31f4f4bacede77e3047f770 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.172508 21629 tablet_server.cc:196] TabletServer@127.21.31.65:0 shutdown complete.
W20260709 13:11:47.196971  1093 consensus_peers.cc:597] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 -> Peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Couldn't send request to peer 49712d6ed31f4f4bacede77e3047f770. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:46467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.206992 21629 tablet_server.cc:179] TabletServer@127.21.31.66:0 shutting down...
I20260709 13:11:47.240593 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:47.241358 21629 tablet_replica.cc:333] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.242285 21629 raft_consensus.cc:2243] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.243078 21629 raft_consensus.cc:2272] T 4a64ea3118f743858b116fa4c3d463a2 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.244848 21629 tablet_replica.cc:333] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.245411 21629 raft_consensus.cc:2243] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.245783 21629 raft_consensus.cc:2272] T f18afa9b1a0c4db8a959a2ecac5ce504 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.248152 21629 tablet_replica.cc:333] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.248993 21629 raft_consensus.cc:2243] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.249518 21629 raft_consensus.cc:2272] T cd22a8edd9b143608ee50df5b9d447f3 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:47.249847  1168 consensus_peers.cc:597] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b -> Peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Couldn't send request to peer 847da4898a4d460ba5696b432c782518. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.251540 21629 tablet_replica.cc:333] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.252259 21629 raft_consensus.cc:2243] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.253209 21629 raft_consensus.cc:2272] T 0f7ede558daa44d0bef89e46af8fb781 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.255295 21629 tablet_replica.cc:333] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.255893 21629 raft_consensus.cc:2243] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.256690 21629 raft_consensus.cc:2272] T f7e53a48fe1d466bb54526b0a428ff31 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.258884 21629 tablet_replica.cc:333] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.259529 21629 raft_consensus.cc:2243] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.259948 21629 raft_consensus.cc:2272] T c9961aebffb649fba246b4fde922b974 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.261910 21629 tablet_replica.cc:333] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.262478 21629 raft_consensus.cc:2243] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.262912 21629 raft_consensus.cc:2272] T 9f469f24d76a42719b3c7eafec6aec8d P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.264664 21629 tablet_replica.cc:333] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.265213 21629 raft_consensus.cc:2243] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.265731 21629 raft_consensus.cc:2272] T b271932cab664b5196fb024cd0b0ee4a P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.268354 21629 tablet_replica.cc:333] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.268919 21629 raft_consensus.cc:2243] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:47.269253  1168 consensus_peers.cc:597] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b -> Peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Couldn't send request to peer 847da4898a4d460ba5696b432c782518. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.269390 21629 raft_consensus.cc:2272] T 5c9a41e7f37a4d98b189476d05d822d9 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.271865 21629 tablet_replica.cc:333] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.272442 21629 raft_consensus.cc:2243] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.272893 21629 raft_consensus.cc:2272] T 9023ef2e63914bc39db6608de9152c5e P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.275269 21629 tablet_replica.cc:333] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.275877 21629 raft_consensus.cc:2243] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.276470 21629 raft_consensus.cc:2272] T a64795fb830e437093b08e6d69e366eb P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.279206 21629 tablet_replica.cc:333] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.279896 21629 raft_consensus.cc:2243] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.280766 21629 raft_consensus.cc:2272] T 7882c3267d5e42df857e4760dec02397 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:47.280975  1167 consensus_peers.cc:597] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b -> Peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Couldn't send request to peer 49712d6ed31f4f4bacede77e3047f770. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:46467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:47.281484  1167 consensus_peers.cc:597] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b -> Peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Couldn't send request to peer 49712d6ed31f4f4bacede77e3047f770. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:46467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.282882 21629 tablet_replica.cc:333] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.283437 21629 raft_consensus.cc:2243] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.284181 21629 raft_consensus.cc:2272] T a069639d84a14cc8b8478a31a7bfac22 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.286085 21629 tablet_replica.cc:333] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.286603 21629 raft_consensus.cc:2243] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.287026 21629 raft_consensus.cc:2272] T 8f1b8fab7cd84b96bbb298fc00003204 P 847da4898a4d460ba5696b432c782518 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.290061 21629 tablet_replica.cc:333] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.290699 21629 raft_consensus.cc:2243] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.291143 21629 raft_consensus.cc:2272] T a551497d48ca4ed58899428085203dc8 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.293036 21629 tablet_replica.cc:333] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.293571 21629 raft_consensus.cc:2243] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.294292 21629 raft_consensus.cc:2272] T c0b3f2617ffe4e269fccaf988eb27805 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.296155 21629 tablet_replica.cc:333] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.296636 21629 raft_consensus.cc:2243] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.297031 21629 raft_consensus.cc:2272] T 568365befae0431a846444a9414c0e69 P 847da4898a4d460ba5696b432c782518 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.299201 21629 tablet_replica.cc:333] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518: stopping tablet replica
I20260709 13:11:47.299723 21629 raft_consensus.cc:2243] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.300287 21629 raft_consensus.cc:2272] T 423005aa03b442348db45ed6e4b4be19 P 847da4898a4d460ba5696b432c782518 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.340178 21629 tablet_server.cc:196] TabletServer@127.21.31.66:0 shutdown complete.
W20260709 13:11:47.340955  1168 consensus_peers.cc:597] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b -> Peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Couldn't send request to peer 847da4898a4d460ba5696b432c782518. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:42379: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.370792 21629 tablet_server.cc:179] TabletServer@127.21.31.67:0 shutting down...
W20260709 13:11:47.396780  1167 consensus_peers.cc:597] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b -> Peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Couldn't send request to peer 49712d6ed31f4f4bacede77e3047f770. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:46467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:47.409715  1168 consensus_peers.cc:597] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b -> Peer 847da4898a4d460ba5696b432c782518 (127.21.31.66:42379): Couldn't send request to peer 847da4898a4d460ba5696b432c782518. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.66:42379: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:47.410609  1167 consensus_peers.cc:597] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b -> Peer 49712d6ed31f4f4bacede77e3047f770 (127.21.31.65:46467): Couldn't send request to peer 49712d6ed31f4f4bacede77e3047f770. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:46467: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.413187 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:47.414251 21629 tablet_replica.cc:333] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.415011 21629 raft_consensus.cc:2243] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.415537 21629 raft_consensus.cc:2272] T f7e53a48fe1d466bb54526b0a428ff31 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.417744 21629 tablet_replica.cc:333] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.418354 21629 raft_consensus.cc:2243] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.418835 21629 raft_consensus.cc:2272] T 4a64ea3118f743858b116fa4c3d463a2 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.420819 21629 tablet_replica.cc:333] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.421512 21629 raft_consensus.cc:2243] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.421964 21629 raft_consensus.cc:2272] T 9023ef2e63914bc39db6608de9152c5e P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.423691 21629 tablet_replica.cc:333] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.424338 21629 raft_consensus.cc:2243] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.424773 21629 raft_consensus.cc:2272] T 9f469f24d76a42719b3c7eafec6aec8d P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.426466 21629 tablet_replica.cc:333] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.426996 21629 raft_consensus.cc:2243] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.427400 21629 raft_consensus.cc:2272] T 0f7ede558daa44d0bef89e46af8fb781 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.429288 21629 tablet_replica.cc:333] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.429888 21629 raft_consensus.cc:2243] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.430598 21629 raft_consensus.cc:2272] T c9961aebffb649fba246b4fde922b974 P 5fd41f935e3446228d999b210c2a4e2b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.432416 21629 tablet_replica.cc:333] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.432945 21629 raft_consensus.cc:2243] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.433717 21629 raft_consensus.cc:2272] T f18afa9b1a0c4db8a959a2ecac5ce504 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.435559 21629 tablet_replica.cc:333] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.436116 21629 raft_consensus.cc:2243] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.436833 21629 raft_consensus.cc:2272] T a64795fb830e437093b08e6d69e366eb P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.438740 21629 tablet_replica.cc:333] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.439278 21629 raft_consensus.cc:2243] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.439719 21629 raft_consensus.cc:2272] T 7882c3267d5e42df857e4760dec02397 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.441613 21629 tablet_replica.cc:333] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.442126 21629 raft_consensus.cc:2243] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.442836 21629 raft_consensus.cc:2272] T a551497d48ca4ed58899428085203dc8 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.444727 21629 tablet_replica.cc:333] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.445264 21629 raft_consensus.cc:2243] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.445709 21629 raft_consensus.cc:2272] T 8f1b8fab7cd84b96bbb298fc00003204 P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.447595 21629 tablet_replica.cc:333] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.448117 21629 raft_consensus.cc:2243] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.448559 21629 raft_consensus.cc:2272] T a069639d84a14cc8b8478a31a7bfac22 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.450364 21629 tablet_replica.cc:333] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.450876 21629 raft_consensus.cc:2243] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.451582 21629 raft_consensus.cc:2272] T 423005aa03b442348db45ed6e4b4be19 P 5fd41f935e3446228d999b210c2a4e2b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.453426 21629 tablet_replica.cc:333] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.453923 21629 raft_consensus.cc:2243] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.454622 21629 raft_consensus.cc:2272] T cd22a8edd9b143608ee50df5b9d447f3 P 5fd41f935e3446228d999b210c2a4e2b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.456382 21629 tablet_replica.cc:333] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.456992 21629 raft_consensus.cc:2243] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.457423 21629 raft_consensus.cc:2272] T c0b3f2617ffe4e269fccaf988eb27805 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.459233 21629 tablet_replica.cc:333] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.459700 21629 raft_consensus.cc:2243] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.460107 21629 raft_consensus.cc:2272] T 5c9a41e7f37a4d98b189476d05d822d9 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.461783 21629 tablet_replica.cc:333] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.462249 21629 raft_consensus.cc:2243] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.462654 21629 raft_consensus.cc:2272] T b271932cab664b5196fb024cd0b0ee4a P 5fd41f935e3446228d999b210c2a4e2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.464504 21629 tablet_replica.cc:333] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b: stopping tablet replica
I20260709 13:11:47.464958 21629 raft_consensus.cc:2243] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:47.465399 21629 raft_consensus.cc:2272] T 568365befae0431a846444a9414c0e69 P 5fd41f935e3446228d999b210c2a4e2b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.507068 21629 tablet_server.cc:196] TabletServer@127.21.31.67:0 shutdown complete.
I20260709 13:11:47.540038 21629 master.cc:562] Master@127.21.31.126:42843 shutting down...
I20260709 13:11:47.562783 21629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:47.563272 21629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:47.563548 21629 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2dbfe7a9e8d749af8c623477f7091b13: stopping tablet replica
I20260709 13:11:47.585014 21629 master.cc:584] Master@127.21.31.126:42843 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12784 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:11:47.639427 21629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.31.126:34049
I20260709 13:11:47.640457 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:47.646140  1518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:47.646251  1517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:47.647094  1520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:47.648416 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:47.649236 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:47.649467 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:47.649653 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602707649640 us; error 0 us; skew 500 ppm
I20260709 13:11:47.650193 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:47.652676 21629 webserver.cc:533] Webserver started at http://127.21.31.126:42011/ using document root <none> and password file <none>
I20260709 13:11:47.653146 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:47.653316 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:47.653614 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:47.654613 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/master-0-root/instance:
uuid: "7f318e82dda5499dace937e8c351b9d5"
format_stamp: "Formatted at 2026-07-09 13:11:47 on dist-test-slave-9s6w"
I20260709 13:11:47.658658 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 13:11:47.661845  1525 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:47.662663 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:11:47.662889 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/master-0-root
uuid: "7f318e82dda5499dace937e8c351b9d5"
format_stamp: "Formatted at 2026-07-09 13:11:47 on dist-test-slave-9s6w"
I20260709 13:11:47.663108 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:47.683581 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:47.684609 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:47.723943 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.126:34049
I20260709 13:11:47.724025  1576 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.126:34049 every 8 connection(s)
I20260709 13:11:47.728794  1577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:47.741510  1577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5: Bootstrap starting.
I20260709 13:11:47.746802  1577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.751282  1577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5: No bootstrap required, opened a new log
I20260709 13:11:47.753670  1577 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f318e82dda5499dace937e8c351b9d5" member_type: VOTER }
I20260709 13:11:47.754050  1577 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.754319  1577 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f318e82dda5499dace937e8c351b9d5, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.754971  1577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f318e82dda5499dace937e8c351b9d5" member_type: VOTER }
I20260709 13:11:47.755432  1577 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:11:47.755636  1577 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:11:47.755918  1577 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.761209  1577 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f318e82dda5499dace937e8c351b9d5" member_type: VOTER }
I20260709 13:11:47.761830  1577 leader_election.cc:304] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f318e82dda5499dace937e8c351b9d5; no voters: 
I20260709 13:11:47.762899  1577 leader_election.cc:290] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:11:47.763332  1580 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:47.764634  1580 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 1 LEADER]: Becoming Leader. State: Replica: 7f318e82dda5499dace937e8c351b9d5, State: Running, Role: LEADER
I20260709 13:11:47.765445  1580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f318e82dda5499dace937e8c351b9d5" member_type: VOTER }
I20260709 13:11:47.765906  1577 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:11:47.770262  1581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f318e82dda5499dace937e8c351b9d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f318e82dda5499dace937e8c351b9d5" member_type: VOTER } }
I20260709 13:11:47.770418  1582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f318e82dda5499dace937e8c351b9d5. Latest consensus state: current_term: 1 leader_uuid: "7f318e82dda5499dace937e8c351b9d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f318e82dda5499dace937e8c351b9d5" member_type: VOTER } }
I20260709 13:11:47.771107  1581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:47.771420  1582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:47.773414  1586 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:11:47.778723  1586 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:11:47.783728 21629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:11:47.788434  1586 catalog_manager.cc:1383] Generated new cluster ID: f93bae63e53f465ea087a13c8d60ec96
I20260709 13:11:47.788702  1586 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:11:47.825032  1586 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:11:47.826272  1586 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:11:47.841073  1586 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5: Generated new TSK 0
I20260709 13:11:47.841603  1586 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:11:47.849891 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:47.856231  1598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:47.857334  1599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:47.859179  1601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:47.859609 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:47.860468 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:47.860678 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:47.860849 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602707860834 us; error 0 us; skew 500 ppm
I20260709 13:11:47.861428 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:47.863565 21629 webserver.cc:533] Webserver started at http://127.21.31.65:39537/ using document root <none> and password file <none>
I20260709 13:11:47.864078 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:47.864282 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:47.864562 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:47.865751 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-0-root/instance:
uuid: "75ec73734cb347918a1298a2baa490e5"
format_stamp: "Formatted at 2026-07-09 13:11:47 on dist-test-slave-9s6w"
I20260709 13:11:47.870365 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:11:47.873490  1606 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:47.874282 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:11:47.874575 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-0-root
uuid: "75ec73734cb347918a1298a2baa490e5"
format_stamp: "Formatted at 2026-07-09 13:11:47 on dist-test-slave-9s6w"
I20260709 13:11:47.874845 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:47.892771 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:47.893918 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:47.895395 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:47.897565 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:47.897782 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:47.898008 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:47.898207 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:47.948391 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.65:35271
I20260709 13:11:47.948506  1668 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.65:35271 every 8 connection(s)
I20260709 13:11:47.954530 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:47.962124  1673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:47.963665  1674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:47.965188 21629 server_base.cc:1061] running on GCE node
W20260709 13:11:47.966748  1677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:47.967890 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:47.968078 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:47.968302 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602707968282 us; error 0 us; skew 500 ppm
I20260709 13:11:47.968919 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:47.969373  1669 heartbeater.cc:344] Connected to a master server at 127.21.31.126:34049
I20260709 13:11:47.969918  1669 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:47.970887  1669 heartbeater.cc:507] Master 127.21.31.126:34049 requested a full tablet report, sending...
I20260709 13:11:47.971534 21629 webserver.cc:533] Webserver started at http://127.21.31.66:36005/ using document root <none> and password file <none>
I20260709 13:11:47.972086 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:47.972295 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:47.972553 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:47.973374  1542 ts_manager.cc:194] Registered new tserver with Master: 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271)
I20260709 13:11:47.974043 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-1-root/instance:
uuid: "3a60ed84fa294767b85c19013457b81b"
format_stamp: "Formatted at 2026-07-09 13:11:47 on dist-test-slave-9s6w"
I20260709 13:11:47.975816  1542 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45838
I20260709 13:11:47.979331 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:11:47.982585  1681 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:47.983325 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:11:47.983613 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-1-root
uuid: "3a60ed84fa294767b85c19013457b81b"
format_stamp: "Formatted at 2026-07-09 13:11:47 on dist-test-slave-9s6w"
I20260709 13:11:47.983902 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:48.004216 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:48.005362 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:48.006775 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:48.008862 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:48.009073 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:48.009344 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:48.009518 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:48.050724 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.66:42197
I20260709 13:11:48.050866  1743 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.66:42197 every 8 connection(s)
I20260709 13:11:48.055609 21629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:48.062628  1747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:48.063561  1748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:48.066426  1751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:48.066694 21629 server_base.cc:1061] running on GCE node
I20260709 13:11:48.067664 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:48.067924 21629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:48.068104 21629 hybrid_clock.cc:648] HybridClock initialized: now 1783602708068085 us; error 0 us; skew 500 ppm
I20260709 13:11:48.068650 21629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:48.071470 21629 webserver.cc:533] Webserver started at http://127.21.31.67:44335/ using document root <none> and password file <none>
I20260709 13:11:48.072055 21629 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:48.072261  1744 heartbeater.cc:344] Connected to a master server at 127.21.31.126:34049
I20260709 13:11:48.072288 21629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:48.072738  1744 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:48.072780 21629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:48.073544  1744 heartbeater.cc:507] Master 127.21.31.126:34049 requested a full tablet report, sending...
I20260709 13:11:48.074163 21629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-2-root/instance:
uuid: "d4286d1c49ba4bee8f2f4b7478053898"
format_stamp: "Formatted at 2026-07-09 13:11:48 on dist-test-slave-9s6w"
I20260709 13:11:48.075675  1542 ts_manager.cc:194] Registered new tserver with Master: 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:48.077811  1542 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45848
I20260709 13:11:48.080319 21629 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:48.083636  1755 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:48.084389 21629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:11:48.084637 21629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-2-root
uuid: "d4286d1c49ba4bee8f2f4b7478053898"
format_stamp: "Formatted at 2026-07-09 13:11:48 on dist-test-slave-9s6w"
I20260709 13:11:48.084867 21629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskqlOMWX/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602359137460-21629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:48.102605 21629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:48.103657 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:48.105069 21629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:48.107268 21629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:48.107487 21629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:48.107728 21629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:48.107911 21629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:48.152614 21629 rpc_server.cc:307] RPC server started. Bound to: 127.21.31.67:38515
I20260709 13:11:48.152765  1817 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.31.67:38515 every 8 connection(s)
I20260709 13:11:48.167929  1818 heartbeater.cc:344] Connected to a master server at 127.21.31.126:34049
I20260709 13:11:48.168603  1818 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:48.169595  1818 heartbeater.cc:507] Master 127.21.31.126:34049 requested a full tablet report, sending...
I20260709 13:11:48.171710  1542 ts_manager.cc:194] Registered new tserver with Master: d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:48.172538 21629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016206034s
I20260709 13:11:48.173441  1542 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45854
I20260709 13:11:48.198086  1542 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45862:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:48.200604  1542 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:48.276655  1634 tablet_service.cc:1511] Processing CreateTablet for tablet ba63fe49e7ae44fca02e50d33499027c (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:48.278126  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba63fe49e7ae44fca02e50d33499027c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.279264  1781 tablet_service.cc:1511] Processing CreateTablet for tablet ee5c391761d44bc4b94a7fc2aeb37505 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:48.280160  1783 tablet_service.cc:1511] Processing CreateTablet for tablet ba63fe49e7ae44fca02e50d33499027c (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:48.280674  1781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5c391761d44bc4b94a7fc2aeb37505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.281431  1783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba63fe49e7ae44fca02e50d33499027c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.282269  1704 tablet_service.cc:1511] Processing CreateTablet for tablet 60a5949a56134e88802dd55f641521e8 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:48.283596  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a5949a56134e88802dd55f641521e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.283463  1708 tablet_service.cc:1511] Processing CreateTablet for tablet a5d20f7524334d9ebaee08b4b4a64b49 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:48.285917  1707 tablet_service.cc:1511] Processing CreateTablet for tablet ee5c391761d44bc4b94a7fc2aeb37505 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:48.287242  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5c391761d44bc4b94a7fc2aeb37505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.286942  1780 tablet_service.cc:1511] Processing CreateTablet for tablet d7510352befc4c2f9feceab054b5443c (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:48.288771  1780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7510352befc4c2f9feceab054b5443c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.290552  1706 tablet_service.cc:1511] Processing CreateTablet for tablet d7510352befc4c2f9feceab054b5443c (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:48.290733  1779 tablet_service.cc:1511] Processing CreateTablet for tablet f1cad82a28d94f7aae10cc833bbcb6fe (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:48.291893  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7510352befc4c2f9feceab054b5443c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.292177  1779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1cad82a28d94f7aae10cc833bbcb6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.294520  1705 tablet_service.cc:1511] Processing CreateTablet for tablet f1cad82a28d94f7aae10cc833bbcb6fe (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:48.295243  1782 tablet_service.cc:1511] Processing CreateTablet for tablet a5d20f7524334d9ebaee08b4b4a64b49 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:48.295853  1705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1cad82a28d94f7aae10cc833bbcb6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.298766  1631 tablet_service.cc:1511] Processing CreateTablet for tablet d7510352befc4c2f9feceab054b5443c (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:48.300078  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7510352befc4c2f9feceab054b5443c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.300825  1778 tablet_service.cc:1511] Processing CreateTablet for tablet 60a5949a56134e88802dd55f641521e8 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:48.302178  1778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a5949a56134e88802dd55f641521e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.302552  1632 tablet_service.cc:1511] Processing CreateTablet for tablet ee5c391761d44bc4b94a7fc2aeb37505 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:48.304858  1782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d20f7524334d9ebaee08b4b4a64b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.305011  1633 tablet_service.cc:1511] Processing CreateTablet for tablet a5d20f7524334d9ebaee08b4b4a64b49 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:48.282300  1709 tablet_service.cc:1511] Processing CreateTablet for tablet ba63fe49e7ae44fca02e50d33499027c (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:48.309484  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba63fe49e7ae44fca02e50d33499027c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.318049  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 8a3f75fbfbaf4606a0d646b57979e7d5 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:48.319332  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a3f75fbfbaf4606a0d646b57979e7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.322373  1630 tablet_service.cc:1511] Processing CreateTablet for tablet f1cad82a28d94f7aae10cc833bbcb6fe (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:48.323359  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d20f7524334d9ebaee08b4b4a64b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.324627  1629 tablet_service.cc:1511] Processing CreateTablet for tablet 60a5949a56134e88802dd55f641521e8 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:48.326846  1628 tablet_service.cc:1511] Processing CreateTablet for tablet 8a3f75fbfbaf4606a0d646b57979e7d5 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:48.303843  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5c391761d44bc4b94a7fc2aeb37505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.339443  1629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a5949a56134e88802dd55f641521e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.340220  1630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1cad82a28d94f7aae10cc833bbcb6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.344856  1777 tablet_service.cc:1511] Processing CreateTablet for tablet 8a3f75fbfbaf4606a0d646b57979e7d5 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:48.345702  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d20f7524334d9ebaee08b4b4a64b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.346276  1777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a3f75fbfbaf4606a0d646b57979e7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.349799  1628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a3f75fbfbaf4606a0d646b57979e7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.368055  1627 tablet_service.cc:1511] Processing CreateTablet for tablet 763c80373e664640b2ccb1aba6ecd425 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:48.369565  1627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763c80373e664640b2ccb1aba6ecd425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.387336  1702 tablet_service.cc:1511] Processing CreateTablet for tablet 763c80373e664640b2ccb1aba6ecd425 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:48.388651  1702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763c80373e664640b2ccb1aba6ecd425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.398475  1626 tablet_service.cc:1511] Processing CreateTablet for tablet 66ff68d2564e4b2eab19e1e61bf4c207 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:48.399652  1701 tablet_service.cc:1511] Processing CreateTablet for tablet 66ff68d2564e4b2eab19e1e61bf4c207 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:48.400290  1626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ff68d2564e4b2eab19e1e61bf4c207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.400869  1701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ff68d2564e4b2eab19e1e61bf4c207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.410459  1780 tablet_service.cc:1511] Processing CreateTablet for tablet 763c80373e664640b2ccb1aba6ecd425 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:48.417786  1780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763c80373e664640b2ccb1aba6ecd425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.478008  1783 tablet_service.cc:1511] Processing CreateTablet for tablet 66ff68d2564e4b2eab19e1e61bf4c207 (DEFAULT_TABLE table=test-workload [id=231675020309449789f9914bbd396cbe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:48.480264  1783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ff68d2564e4b2eab19e1e61bf4c207. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:48.499378  1834 tablet_bootstrap.cc:492] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:48.503710  1835 tablet_bootstrap.cc:492] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:48.506716  1833 tablet_bootstrap.cc:492] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:48.505507  1834 tablet_bootstrap.cc:654] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.510964  1835 tablet_bootstrap.cc:654] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.516320  1835 tablet_bootstrap.cc:492] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:48.516770  1835 ts_tablet_manager.cc:1403] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:11:48.517676  1834 tablet_bootstrap.cc:492] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:48.517930  1833 tablet_bootstrap.cc:654] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.518090  1834 ts_tablet_manager.cc:1403] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.007s
I20260709 13:11:48.519680  1835 raft_consensus.cc:359] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.520465  1835 raft_consensus.cc:385] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.520246  1834 raft_consensus.cc:359] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.520907  1835 raft_consensus.cc:740] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.521103  1834 raft_consensus.cc:385] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.521520  1834 raft_consensus.cc:740] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.521860  1835 consensus_queue.cc:260] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.522207  1834 consensus_queue.cc:260] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.522892  1833 tablet_bootstrap.cc:492] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:48.523514  1833 ts_tablet_manager.cc:1403] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 13:11:48.524330  1669 heartbeater.cc:499] Master 127.21.31.126:34049 was elected leader, sending a full tablet report...
I20260709 13:11:48.526422  1833 raft_consensus.cc:359] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.527145  1833 raft_consensus.cc:385] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.527446  1833 raft_consensus.cc:740] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.528134  1835 ts_tablet_manager.cc:1434] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:11:48.528154  1833 consensus_queue.cc:260] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.529861  1835 tablet_bootstrap.cc:492] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:48.537073  1818 heartbeater.cc:499] Master 127.21.31.126:34049 was elected leader, sending a full tablet report...
I20260709 13:11:48.538117  1833 ts_tablet_manager.cc:1434] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.010s
I20260709 13:11:48.538393  1744 heartbeater.cc:499] Master 127.21.31.126:34049 was elected leader, sending a full tablet report...
I20260709 13:11:48.539163  1833 tablet_bootstrap.cc:492] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:48.542259  1835 tablet_bootstrap.cc:654] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.545470  1833 tablet_bootstrap.cc:654] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.550706  1834 ts_tablet_manager.cc:1434] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.032s	user 0.015s	sys 0.002s
I20260709 13:11:48.551827  1834 tablet_bootstrap.cc:492] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:48.552178  1835 tablet_bootstrap.cc:492] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:48.552855  1835 ts_tablet_manager.cc:1403] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.004s
I20260709 13:11:48.555985  1835 raft_consensus.cc:359] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.556676  1835 raft_consensus.cc:385] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.556749  1833 tablet_bootstrap.cc:492] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:48.557031  1835 raft_consensus.cc:740] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.557348  1833 ts_tablet_manager.cc:1403] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.012s
I20260709 13:11:48.557974  1835 consensus_queue.cc:260] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.560842  1834 tablet_bootstrap.cc:654] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.561488  1835 ts_tablet_manager.cc:1434] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.002s
I20260709 13:11:48.561648  1833 raft_consensus.cc:359] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.562362  1833 raft_consensus.cc:385] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.562475  1835 tablet_bootstrap.cc:492] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:48.562687  1833 raft_consensus.cc:740] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.563568  1833 consensus_queue.cc:260] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.566264  1833 ts_tablet_manager.cc:1434] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:11:48.567468  1833 tablet_bootstrap.cc:492] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:48.569077  1835 tablet_bootstrap.cc:654] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.578037  1833 tablet_bootstrap.cc:654] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.579994  1835 tablet_bootstrap.cc:492] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:48.580510  1835 ts_tablet_manager.cc:1403] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.009s
I20260709 13:11:48.582999  1834 tablet_bootstrap.cc:492] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:48.583511  1834 ts_tablet_manager.cc:1403] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.014s
I20260709 13:11:48.583374  1835 raft_consensus.cc:359] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.584038  1833 tablet_bootstrap.cc:492] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:48.584177  1835 raft_consensus.cc:385] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.584707  1835 raft_consensus.cc:740] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.584838  1833 ts_tablet_manager.cc:1403] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 13:11:48.585597  1835 consensus_queue.cc:260] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.586534  1834 raft_consensus.cc:359] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.587263  1834 raft_consensus.cc:385] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.587556  1834 raft_consensus.cc:740] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.588092  1835 ts_tablet_manager.cc:1434] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:48.588287  1834 consensus_queue.cc:260] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.589107  1835 tablet_bootstrap.cc:492] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:48.590417  1834 ts_tablet_manager.cc:1434] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:11:48.591372  1834 tablet_bootstrap.cc:492] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:48.588315  1833 raft_consensus.cc:359] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.594175  1833 raft_consensus.cc:385] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.594470  1833 raft_consensus.cc:740] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.595366  1835 tablet_bootstrap.cc:654] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.595188  1833 consensus_queue.cc:260] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.597589  1833 ts_tablet_manager.cc:1434] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:11:48.598846  1833 tablet_bootstrap.cc:492] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:48.605791  1834 tablet_bootstrap.cc:654] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.605865  1833 tablet_bootstrap.cc:654] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.606329  1835 tablet_bootstrap.cc:492] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:48.606819  1835 ts_tablet_manager.cc:1403] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 13:11:48.610239  1835 raft_consensus.cc:359] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.611174  1835 raft_consensus.cc:385] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.611552  1835 raft_consensus.cc:740] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.612854  1835 consensus_queue.cc:260] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.616685  1835 ts_tablet_manager.cc:1434] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:11:48.617992  1835 tablet_bootstrap.cc:492] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:48.628208  1835 tablet_bootstrap.cc:654] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.628230  1834 tablet_bootstrap.cc:492] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:48.629000  1833 tablet_bootstrap.cc:492] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:48.629079  1834 ts_tablet_manager.cc:1403] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.016s
I20260709 13:11:48.629717  1833 ts_tablet_manager.cc:1403] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.031s	user 0.027s	sys 0.003s
I20260709 13:11:48.632910  1834 raft_consensus.cc:359] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.632916  1833 raft_consensus.cc:359] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.633827  1834 raft_consensus.cc:385] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.634052  1833 raft_consensus.cc:385] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.634241  1834 raft_consensus.cc:740] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.634436  1833 raft_consensus.cc:740] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.635192  1835 tablet_bootstrap.cc:492] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:48.635231  1833 consensus_queue.cc:260] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.635948  1835 ts_tablet_manager.cc:1403] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:11:48.635274  1834 consensus_queue.cc:260] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.637987  1834 ts_tablet_manager.cc:1434] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:11:48.638087  1833 ts_tablet_manager.cc:1434] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:11:48.639128  1833 tablet_bootstrap.cc:492] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:48.639258  1834 tablet_bootstrap.cc:492] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:48.639109  1835 raft_consensus.cc:359] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } }
I20260709 13:11:48.640187  1835 raft_consensus.cc:385] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.640528  1835 raft_consensus.cc:740] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.641422  1835 consensus_queue.cc:260] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } }
I20260709 13:11:48.645725  1834 tablet_bootstrap.cc:654] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.646158  1833 tablet_bootstrap.cc:654] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.646891  1840 raft_consensus.cc:493] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.647480  1840 raft_consensus.cc:515] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.649037  1839 raft_consensus.cc:493] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.649410  1841 raft_consensus.cc:493] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.650096  1839 raft_consensus.cc:515] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.650156  1841 raft_consensus.cc:515] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.653640  1839 leader_election.cc:290] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:48.671877  1840 leader_election.cc:290] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:48.684223  1841 leader_election.cc:290] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:48.694080  1833 tablet_bootstrap.cc:492] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:48.694691  1833 ts_tablet_manager.cc:1403] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.056s	user 0.023s	sys 0.017s
I20260709 13:11:48.703697  1835 ts_tablet_manager.cc:1434] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 13:11:48.704031  1834 tablet_bootstrap.cc:492] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:48.704694  1834 ts_tablet_manager.cc:1403] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.013s
I20260709 13:11:48.701342  1833 raft_consensus.cc:359] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } }
I20260709 13:11:48.706401  1833 raft_consensus.cc:385] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.706700  1833 raft_consensus.cc:740] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.708340  1835 tablet_bootstrap.cc:492] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:48.707368  1833 consensus_queue.cc:260] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } }
I20260709 13:11:48.713647  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:48.714941  1833 ts_tablet_manager.cc:1434] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.003s
W20260709 13:11:48.715530  1609 leader_election.cc:343] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.715975  1833 tablet_bootstrap.cc:492] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:48.718758  1839 raft_consensus.cc:493] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.719368  1839 raft_consensus.cc:515] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } }
I20260709 13:11:48.721856  1839 leader_election.cc:290] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:48.722743  1833 tablet_bootstrap.cc:654] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.722820  1834 raft_consensus.cc:359] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.723423  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba63fe49e7ae44fca02e50d33499027c" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:48.723826  1834 raft_consensus.cc:385] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.724107  1793 raft_consensus.cc:2468] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:48.724315  1834 raft_consensus.cc:740] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.727674  1834 consensus_queue.cc:260] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.730080  1834 ts_tablet_manager.cc:1434] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:11:48.731055  1834 tablet_bootstrap.cc:492] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:48.732070  1835 tablet_bootstrap.cc:654] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.737133  1834 tablet_bootstrap.cc:654] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.725237  1609 leader_election.cc:304] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:48.740717  1839 raft_consensus.cc:2804] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.741114  1839 raft_consensus.cc:493] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.741468  1839 raft_consensus.cc:3060] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.741799  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a5949a56134e88802dd55f641521e8" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:48.742547  1644 raft_consensus.cc:2468] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:48.745122  1683 leader_election.cc:304] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:48.751756  1840 raft_consensus.cc:2804] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.752137  1840 raft_consensus.cc:493] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.752465  1840 raft_consensus.cc:3060] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.753423  1841 raft_consensus.cc:493] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.754400  1841 raft_consensus.cc:515] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.764313  1841 leader_election.cc:290] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:48.765790  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:48.767253  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba63fe49e7ae44fca02e50d33499027c" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:48.767453  1840 raft_consensus.cc:515] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.768919  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7510352befc4c2f9feceab054b5443c" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:48.770531  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a5949a56134e88802dd55f641521e8" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:48.771176  1643 raft_consensus.cc:2468] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 0.
W20260709 13:11:48.772818  1757 leader_election.cc:343] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.773610  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a5949a56134e88802dd55f641521e8" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:48.774186  1644 raft_consensus.cc:3060] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.775591  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a5949a56134e88802dd55f641521e8" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:48.777295  1793 raft_consensus.cc:2468] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
W20260709 13:11:48.781401  1590 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:48.777529  1839 raft_consensus.cc:515] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } }
I20260709 13:11:48.782795  1840 leader_election.cc:290] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:48.784904  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a5949a56134e88802dd55f641521e8" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:48.785596  1793 raft_consensus.cc:3060] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:48.785038  1607 leader_election.cc:343] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.787778  1607 leader_election.cc:304] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5; no voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898
I20260709 13:11:48.786392  1839 leader_election.cc:290] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Requested vote from peers d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:48.786484  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba63fe49e7ae44fca02e50d33499027c" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:48.788940  1792 raft_consensus.cc:3060] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.791204  1835 tablet_bootstrap.cc:492] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:48.791481  1644 raft_consensus.cc:2468] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:48.791788  1835 ts_tablet_manager.cc:1403] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.084s	user 0.014s	sys 0.003s
I20260709 13:11:48.792668  1683 leader_election.cc:304] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:48.793591  1840 raft_consensus.cc:2804] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.795426  1793 raft_consensus.cc:2468] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:48.796118  1835 raft_consensus.cc:359] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.797132  1835 raft_consensus.cc:385] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.797303  1834 tablet_bootstrap.cc:492] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:48.797670  1835 raft_consensus.cc:740] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.797971  1834 ts_tablet_manager.cc:1403] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.012s
I20260709 13:11:48.798861  1835 consensus_queue.cc:260] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.800307  1792 raft_consensus.cc:2468] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:48.801617  1835 ts_tablet_manager.cc:1434] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:11:48.801362  1834 raft_consensus.cc:359] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } }
I20260709 13:11:48.801962  1609 leader_election.cc:304] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:48.805042  1835 tablet_bootstrap.cc:492] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:48.805187  1833 tablet_bootstrap.cc:492] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:48.805859  1833 ts_tablet_manager.cc:1403] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.090s	user 0.031s	sys 0.021s
I20260709 13:11:48.797791  1839 raft_consensus.cc:2749] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:48.809001  1839 raft_consensus.cc:2804] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.809343  1833 raft_consensus.cc:359] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.810268  1833 raft_consensus.cc:385] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.810611  1833 raft_consensus.cc:740] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.802181  1834 raft_consensus.cc:385] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.811161  1834 raft_consensus.cc:740] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.811112  1839 raft_consensus.cc:697] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:48.811493  1833 consensus_queue.cc:260] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.812371  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a5949a56134e88802dd55f641521e8" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:48.811936  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7510352befc4c2f9feceab054b5443c" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:48.812862  1834 consensus_queue.cc:260] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } }
I20260709 13:11:48.814189  1833 ts_tablet_manager.cc:1434] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
W20260709 13:11:48.814874  1756 leader_election.cc:343] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.815199  1833 tablet_bootstrap.cc:492] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:48.815512  1756 leader_election.cc:304] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4286d1c49ba4bee8f2f4b7478053898; no voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5
I20260709 13:11:48.817044  1841 raft_consensus.cc:2749] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:48.822038  1834 ts_tablet_manager.cc:1434] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.024s	user 0.011s	sys 0.004s
I20260709 13:11:48.822530  1835 tablet_bootstrap.cc:654] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.825176  1834 tablet_bootstrap.cc:492] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:48.828004  1833 tablet_bootstrap.cc:654] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.832497  1839 consensus_queue.cc:237] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } }
I20260709 13:11:48.834569  1834 tablet_bootstrap.cc:654] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.835451  1835 tablet_bootstrap.cc:492] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:48.836042  1835 ts_tablet_manager.cc:1403] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.005s
I20260709 13:11:48.841144  1835 raft_consensus.cc:359] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.841879  1833 tablet_bootstrap.cc:492] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:48.842211  1835 raft_consensus.cc:385] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.842418  1833 ts_tablet_manager.cc:1403] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.006s
I20260709 13:11:48.842765  1835 raft_consensus.cc:740] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.843971  1835 consensus_queue.cc:260] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.845505  1833 raft_consensus.cc:359] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.846016  1834 tablet_bootstrap.cc:492] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:48.846268  1833 raft_consensus.cc:385] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.846588  1834 ts_tablet_manager.cc:1403] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.002s
I20260709 13:11:48.846624  1833 raft_consensus.cc:740] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.847087  1835 ts_tablet_manager.cc:1434] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:11:48.848244  1835 tablet_bootstrap.cc:492] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:48.849624  1834 raft_consensus.cc:359] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.850358  1834 raft_consensus.cc:385] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.850699  1834 raft_consensus.cc:740] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.850729  1840 raft_consensus.cc:697] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
W20260709 13:11:48.851303  1607 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:42197, user_credentials={real_user=slave}} blocked reactor thread for 72506.5us
I20260709 13:11:48.851712  1834 consensus_queue.cc:260] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.851763  1840 consensus_queue.cc:237] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.855734  1834 ts_tablet_manager.cc:1434] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 13:11:48.856277  1835 tablet_bootstrap.cc:654] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.856796  1834 tablet_bootstrap.cc:492] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:48.857470  1833 consensus_queue.cc:260] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.861734  1833 ts_tablet_manager.cc:1434] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.004s
I20260709 13:11:48.862707  1833 tablet_bootstrap.cc:492] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:48.865599  1834 tablet_bootstrap.cc:654] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.867759  1835 tablet_bootstrap.cc:492] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:48.868307  1835 ts_tablet_manager.cc:1403] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 13:11:48.871654  1835 raft_consensus.cc:359] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.871598  1542 catalog_manager.cc:5697] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:48.872783  1835 raft_consensus.cc:385] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.874261  1835 raft_consensus.cc:740] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.875047  1835 consensus_queue.cc:260] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
W20260709 13:11:48.852100  1607 leader_election.cc:343] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.877089  1834 tablet_bootstrap.cc:492] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:48.877605  1834 ts_tablet_manager.cc:1403] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.007s
I20260709 13:11:48.877575  1757 leader_election.cc:304] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:48.878930  1841 raft_consensus.cc:2764] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:11:48.880357  1833 tablet_bootstrap.cc:654] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.878556  1539 catalog_manager.cc:5697] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 1 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:48.880641  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba63fe49e7ae44fca02e50d33499027c" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:48.880722  1834 raft_consensus.cc:359] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.881552  1834 raft_consensus.cc:385] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.881590  1719 raft_consensus.cc:3060] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.881839  1835 ts_tablet_manager.cc:1434] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:11:48.881914  1834 raft_consensus.cc:740] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.882663  1840 raft_consensus.cc:493] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.883150  1835 tablet_bootstrap.cc:492] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:48.882921  1834 consensus_queue.cc:260] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.883174  1840 raft_consensus.cc:515] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.885522  1834 ts_tablet_manager.cc:1434] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:48.885629  1840 leader_election.cc:290] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:48.886585  1834 tablet_bootstrap.cc:492] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:48.886833  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ff68d2564e4b2eab19e1e61bf4c207" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:48.887292  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ff68d2564e4b2eab19e1e61bf4c207" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:48.887887  1644 raft_consensus.cc:2468] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
W20260709 13:11:48.888401  1684 leader_election.cc:343] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.889101  1683 leader_election.cc:304] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: d4286d1c49ba4bee8f2f4b7478053898
I20260709 13:11:48.889518  1835 tablet_bootstrap.cc:654] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.890214  1840 raft_consensus.cc:2804] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.890923  1840 raft_consensus.cc:493] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.891381  1840 raft_consensus.cc:3060] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.895090  1834 tablet_bootstrap.cc:654] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.898826  1719 raft_consensus.cc:2468] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:48.900539  1840 raft_consensus.cc:515] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.901340  1835 tablet_bootstrap.cc:492] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:48.901794  1835 ts_tablet_manager.cc:1403] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.008s
I20260709 13:11:48.899369  1839 raft_consensus.cc:493] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.902916  1839 raft_consensus.cc:515] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.903215  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ff68d2564e4b2eab19e1e61bf4c207" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:48.903793  1644 raft_consensus.cc:3060] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.905172  1835 raft_consensus.cc:359] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.905983  1835 raft_consensus.cc:385] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.906270  1835 raft_consensus.cc:740] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.906940  1839 leader_election.cc:290] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:48.907531  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5c391761d44bc4b94a7fc2aeb37505" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:48.908438  1792 raft_consensus.cc:2468] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:48.909612  1609 leader_election.cc:304] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:48.907624  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5c391761d44bc4b94a7fc2aeb37505" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:48.910957  1644 raft_consensus.cc:2468] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
W20260709 13:11:48.911119  1607 leader_election.cc:343] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.911387  1835 consensus_queue.cc:260] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.910560  1839 raft_consensus.cc:2804] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.912748  1839 raft_consensus.cc:493] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.913193  1839 raft_consensus.cc:3060] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.912106  1683 leader_election.cc:304] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:48.913519  1835 ts_tablet_manager.cc:1434] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:11:48.914129  1866 raft_consensus.cc:2804] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.914604  1866 raft_consensus.cc:697] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:48.915409  1866 consensus_queue.cc:237] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.913344  1834 tablet_bootstrap.cc:492] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:48.916286  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ff68d2564e4b2eab19e1e61bf4c207" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:48.916476  1834 ts_tablet_manager.cc:1403] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.000s
I20260709 13:11:48.917191  1840 leader_election.cc:290] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
W20260709 13:11:48.917672  1684 leader_election.cc:343] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.919905  1834 raft_consensus.cc:359] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.925796  1834 raft_consensus.cc:385] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.926139  1834 raft_consensus.cc:740] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.926919  1834 consensus_queue.cc:260] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.929004  1834 ts_tablet_manager.cc:1434] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:11:48.929912  1834 tablet_bootstrap.cc:492] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:48.930068  1833 tablet_bootstrap.cc:492] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:48.924317  1839 raft_consensus.cc:515] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.930560  1833 ts_tablet_manager.cc:1403] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.068s	user 0.008s	sys 0.003s
I20260709 13:11:48.932724  1839 leader_election.cc:290] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:48.933281  1833 raft_consensus.cc:359] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.935137  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5c391761d44bc4b94a7fc2aeb37505" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:48.936930  1541 catalog_manager.cc:5697] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 1 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:48.937738  1792 raft_consensus.cc:3060] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.937839  1834 tablet_bootstrap.cc:654] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.935377  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5c391761d44bc4b94a7fc2aeb37505" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:48.938599  1719 raft_consensus.cc:3060] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.939631  1833 raft_consensus.cc:385] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.939939  1833 raft_consensus.cc:740] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.944180  1841 raft_consensus.cc:493] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.944669  1841 raft_consensus.cc:515] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.946761  1841 leader_election.cc:290] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:48.948030  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1cad82a28d94f7aae10cc833bbcb6fe" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:48.948825  1718 raft_consensus.cc:2468] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 0.
I20260709 13:11:48.948895  1833 consensus_queue.cc:260] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.950058  1756 leader_election.cc:304] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:48.947947  1792 raft_consensus.cc:2468] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:48.950470  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1cad82a28d94f7aae10cc833bbcb6fe" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:48.951359  1644 raft_consensus.cc:2468] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 0.
I20260709 13:11:48.951632  1609 leader_election.cc:304] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:48.952729  1865 raft_consensus.cc:2804] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.953187  1865 raft_consensus.cc:697] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:48.953986  1865 consensus_queue.cc:237] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.954775  1841 raft_consensus.cc:493] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.955410  1841 raft_consensus.cc:515] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.958012  1841 leader_election.cc:290] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:48.960035  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:48.960860  1644 raft_consensus.cc:2468] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 0.
I20260709 13:11:48.964680  1757 leader_election.cc:304] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:48.962978  1841 raft_consensus.cc:2804] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.966497  1841 raft_consensus.cc:493] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.966995  1841 raft_consensus.cc:3060] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.970309  1541 catalog_manager.cc:5697] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:48.973184  1719 raft_consensus.cc:2468] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:48.974287  1865 raft_consensus.cc:493] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.974979  1865 raft_consensus.cc:515] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.978654  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c80373e664640b2ccb1aba6ecd425" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:48.979360  1718 raft_consensus.cc:2468] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:48.980535  1607 leader_election.cc:304] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:48.981153  1834 tablet_bootstrap.cc:492] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:48.982028  1834 ts_tablet_manager.cc:1403] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.052s	user 0.018s	sys 0.029s
I20260709 13:11:48.981472  1867 raft_consensus.cc:2804] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.982477  1867 raft_consensus.cc:493] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.982860  1867 raft_consensus.cc:3060] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.985023  1834 raft_consensus.cc:359] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.985860  1834 raft_consensus.cc:385] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.986171  1834 raft_consensus.cc:740] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.986984  1834 consensus_queue.cc:260] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.988329  1866 raft_consensus.cc:493] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.988767  1866 raft_consensus.cc:515] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.990170  1867 raft_consensus.cc:515] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.990708  1841 raft_consensus.cc:515] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:48.991289  1884 raft_consensus.cc:2804] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.991736  1884 raft_consensus.cc:493] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.991842  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:48.992090  1884 raft_consensus.cc:3060] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.992280  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:48.992517  1644 raft_consensus.cc:2468] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:48.992851  1841 leader_election.cc:290] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:48.993587  1683 leader_election.cc:304] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
W20260709 13:11:48.993700  1756 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.31.66:42197, user_credentials={real_user=slave}} blocked reactor thread for 43748.1us
I20260709 13:11:48.994609  1868 raft_consensus.cc:2804] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.994985  1868 raft_consensus.cc:493] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.995306  1868 raft_consensus.cc:3060] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.990756  1866 leader_election.cc:290] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:48.995999  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1cad82a28d94f7aae10cc833bbcb6fe" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:48.996650  1719 raft_consensus.cc:3060] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.997718  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:48.999245  1884 raft_consensus.cc:515] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.000494  1793 raft_consensus.cc:2393] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a60ed84fa294767b85c19013457b81b in current term 1: Already voted for candidate d4286d1c49ba4bee8f2f4b7478053898 in this term.
I20260709 13:11:49.001292  1884 leader_election.cc:290] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:49.002980  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1cad82a28d94f7aae10cc833bbcb6fe" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:49.003350  1719 raft_consensus.cc:2468] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 1.
I20260709 13:11:49.003726  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:49.003831  1643 raft_consensus.cc:3060] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.005106  1756 leader_election.cc:304] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:49.005577  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c80373e664640b2ccb1aba6ecd425" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:49.006067  1833 ts_tablet_manager.cc:1434] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.075s	user 0.012s	sys 0.000s
I20260709 13:11:48.981209  1865 leader_election.cc:290] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.006222  1792 raft_consensus.cc:2468] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:49.006786  1868 raft_consensus.cc:515] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.008044  1718 raft_consensus.cc:2393] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4286d1c49ba4bee8f2f4b7478053898 in current term 1: Already voted for candidate 3a60ed84fa294767b85c19013457b81b in this term.
I20260709 13:11:49.008792  1868 leader_election.cc:290] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.008971  1833 tablet_bootstrap.cc:492] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:49.009253  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:49.009909  1642 raft_consensus.cc:3060] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.006260  1884 raft_consensus.cc:2804] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.011019  1834 ts_tablet_manager.cc:1434] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:11:49.013432  1867 leader_election.cc:290] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.011217  1884 raft_consensus.cc:697] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Becoming Leader. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:49.015091  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c80373e664640b2ccb1aba6ecd425" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:49.015834  1718 raft_consensus.cc:3060] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.016471  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:49.017373  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:49.016806  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c80373e664640b2ccb1aba6ecd425" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:49.018599  1791 raft_consensus.cc:2393] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a60ed84fa294767b85c19013457b81b in current term 1: Already voted for candidate d4286d1c49ba4bee8f2f4b7478053898 in this term.
I20260709 13:11:49.018957  1793 raft_consensus.cc:3060] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.020105  1684 leader_election.cc:304] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b; no voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898
I20260709 13:11:49.020932  1868 raft_consensus.cc:2749] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:49.025687  1643 raft_consensus.cc:2468] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 1.
I20260709 13:11:49.025688  1718 raft_consensus.cc:2468] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:49.028458  1642 raft_consensus.cc:2468] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 1.
I20260709 13:11:49.029533  1793 raft_consensus.cc:2468] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:49.016070  1884 consensus_queue.cc:237] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.032778  1607 leader_election.cc:304] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:49.034207  1867 raft_consensus.cc:2804] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.034130  1757 leader_election.cc:304] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:49.034912  1867 raft_consensus.cc:697] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:49.035223  1886 raft_consensus.cc:2804] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.035779  1867 consensus_queue.cc:237] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.035847  1886 raft_consensus.cc:697] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Becoming Leader. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:49.037344  1886 consensus_queue.cc:237] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.038364  1833 tablet_bootstrap.cc:654] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.042670  1833 tablet_bootstrap.cc:492] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:49.043151  1833 ts_tablet_manager.cc:1403] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 13:11:49.045717  1833 raft_consensus.cc:359] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.046339  1833 raft_consensus.cc:385] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.046576  1833 raft_consensus.cc:740] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.047258  1833 consensus_queue.cc:260] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.048970  1833 ts_tablet_manager.cc:1434] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:49.051501  1541 catalog_manager.cc:5697] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.056874  1539 catalog_manager.cc:5697] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:49.069799  1539 catalog_manager.cc:5697] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:49.149501  1867 raft_consensus.cc:493] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.150008  1867 raft_consensus.cc:515] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.151577  1867 leader_election.cc:290] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.152560  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7510352befc4c2f9feceab054b5443c" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:49.152685  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7510352befc4c2f9feceab054b5443c" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:49.153175  1718 raft_consensus.cc:2468] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:49.153280  1793 raft_consensus.cc:2468] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:49.154203  1607 leader_election.cc:304] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:49.154837  1867 raft_consensus.cc:2804] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:49.155182  1867 raft_consensus.cc:493] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.155403  1867 raft_consensus.cc:3060] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.160069  1867 raft_consensus.cc:515] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.161561  1867 leader_election.cc:290] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.162284  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7510352befc4c2f9feceab054b5443c" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:49.162760  1718 raft_consensus.cc:3060] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.162761  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7510352befc4c2f9feceab054b5443c" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:49.163511  1793 raft_consensus.cc:3060] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.167551  1718 raft_consensus.cc:2468] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:49.168481  1607 leader_election.cc:304] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:49.169100  1867 raft_consensus.cc:2804] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.169610  1867 raft_consensus.cc:697] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:49.169981  1793 raft_consensus.cc:2468] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:49.170444  1867 consensus_queue.cc:237] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.178670  1539 catalog_manager.cc:5697] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.242172  1868 consensus_queue.cc:1048] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.246254  1868 raft_consensus.cc:493] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.246785  1868 raft_consensus.cc:515] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.248865  1868 leader_election.cc:290] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.249135  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:49.249788  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:49.249814  1643 raft_consensus.cc:2468] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:49.250591  1791 raft_consensus.cc:2468] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:49.251825  1683 leader_election.cc:304] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:49.256131  1868 raft_consensus.cc:2804] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:49.256522  1868 raft_consensus.cc:493] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.256866  1868 raft_consensus.cc:3060] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.263238  1840 consensus_queue.cc:1048] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:49.264035  1868 raft_consensus.cc:515] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.266106  1868 leader_election.cc:290] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.267213  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:49.267817  1791 raft_consensus.cc:3060] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.268070  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:49.268661  1642 raft_consensus.cc:3060] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.275604  1642 raft_consensus.cc:2468] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:49.277102  1683 leader_election.cc:304] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:49.277911  1791 raft_consensus.cc:2468] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:49.278429  1868 raft_consensus.cc:2804] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.279093  1868 raft_consensus.cc:697] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:49.280980  1868 consensus_queue.cc:237] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.288663  1539 catalog_manager.cc:5697] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 1 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.291113  1867 consensus_queue.cc:1048] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.302157  1865 consensus_queue.cc:1048] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:49.405923  1791 raft_consensus.cc:3060] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.411849  1868 consensus_queue.cc:1048] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.424247  1868 consensus_queue.cc:1048] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.443411  1865 consensus_queue.cc:1048] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.452081  1865 consensus_queue.cc:1048] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.453590  1793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49"
dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
mode: GRACEFUL
new_leader_uuid: "75ec73734cb347918a1298a2baa490e5"
 from {username='slave'} at 127.0.0.1:53306
I20260709 13:11:49.454205  1793 raft_consensus.cc:606] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Received request to transfer leadership to TS 75ec73734cb347918a1298a2baa490e5
I20260709 13:11:49.471434  1792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1cad82a28d94f7aae10cc833bbcb6fe"
dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
mode: GRACEFUL
new_leader_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:53306
I20260709 13:11:49.472030  1792 raft_consensus.cc:606] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Received request to transfer leadership to TS 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:49.487715  1865 consensus_queue.cc:1048] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.493183  1643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "763c80373e664640b2ccb1aba6ecd425"
dest_uuid: "75ec73734cb347918a1298a2baa490e5"
mode: GRACEFUL
new_leader_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:33130
I20260709 13:11:49.493741  1643 raft_consensus.cc:606] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Received request to transfer leadership to TS 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:49.501264  1839 consensus_queue.cc:1048] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.515511  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66ff68d2564e4b2eab19e1e61bf4c207"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
mode: GRACEFUL
new_leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
 from {username='slave'} at 127.0.0.1:53950
I20260709 13:11:49.516146  1717 raft_consensus.cc:606] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Received request to transfer leadership to TS d4286d1c49ba4bee8f2f4b7478053898
I20260709 13:11:49.562415  1886 consensus_queue.cc:1048] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.567288  1886 consensus_queue.cc:1048] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.571195  1912 raft_consensus.cc:993] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898: : Instructing follower 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:49.572921  1886 raft_consensus.cc:1081] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Signalling peer 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:49.579547  1717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1cad82a28d94f7aae10cc833bbcb6fe"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:53938
I20260709 13:11:49.579840  1886 consensus_queue.cc:1048] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.580080  1717 raft_consensus.cc:493] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:49.583559  1717 raft_consensus.cc:3060] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.586946  1886 raft_consensus.cc:993] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898: : Instructing follower 75ec73734cb347918a1298a2baa490e5 to start an election
I20260709 13:11:49.587422  1886 raft_consensus.cc:1081] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Signalling peer 75ec73734cb347918a1298a2baa490e5 to start an election
I20260709 13:11:49.589833  1644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49"
dest_uuid: "75ec73734cb347918a1298a2baa490e5"
 from {username='slave'} at 127.0.0.1:33116
I20260709 13:11:49.590523  1644 raft_consensus.cc:493] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:49.591002  1644 raft_consensus.cc:3060] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.592865  1912 consensus_queue.cc:1048] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.598275  1717 raft_consensus.cc:515] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.599869  1644 raft_consensus.cc:515] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.600932  1643 raft_consensus.cc:1240] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Rejecting Update request from peer d4286d1c49ba4bee8f2f4b7478053898 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:49.601048  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1cad82a28d94f7aae10cc833bbcb6fe" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:49.601679  1642 raft_consensus.cc:3060] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.605161  1912 consensus_queue.cc:1059] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:49.605492  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1cad82a28d94f7aae10cc833bbcb6fe" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:49.605643  1644 leader_election.cc:290] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 2 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.606184  1792 raft_consensus.cc:3055] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:49.606676  1792 raft_consensus.cc:740] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:49.607484  1792 consensus_queue.cc:260] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.607693  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:49.606213  1912 raft_consensus.cc:3055] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:49.608378  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d20f7524334d9ebaee08b4b4a64b49" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:49.608497  1912 raft_consensus.cc:740] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:49.608493  1717 leader_election.cc:290] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.613977  1912 consensus_queue.cc:260] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.615087  1912 raft_consensus.cc:3060] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.608419  1792 raft_consensus.cc:3060] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.623237  1642 raft_consensus.cc:2468] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 2.
I20260709 13:11:49.624612  1683 leader_election.cc:304] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:49.625497  1868 raft_consensus.cc:2804] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:49.625978  1868 raft_consensus.cc:697] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:49.626784  1868 consensus_queue.cc:237] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.628513  1865 consensus_queue.cc:1048] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.630353  1791 raft_consensus.cc:2468] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 2.
I20260709 13:11:49.630479  1792 raft_consensus.cc:2468] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 2.
I20260709 13:11:49.632352  1609 leader_election.cc:304] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:49.633299  1865 raft_consensus.cc:2804] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:49.633807  1865 raft_consensus.cc:697] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 2 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:49.634423  1539 catalog_manager.cc:5697] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 1 to 2, leader changed from d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67) to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 2 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.634682  1865 consensus_queue.cc:237] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.643532  1541 catalog_manager.cc:5697] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 1 to 2, leader changed from d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67) to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.645356  1839 consensus_queue.cc:1048] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.726346  1868 consensus_queue.cc:1048] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.736402  1866 consensus_queue.cc:1048] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.988076  1865 raft_consensus.cc:993] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5: : Instructing follower 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:49.988497  1865 raft_consensus.cc:1081] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Signalling peer 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:49.990269  1718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "763c80373e664640b2ccb1aba6ecd425"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:53248
I20260709 13:11:49.990824  1718 raft_consensus.cc:493] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:49.991097  1718 raft_consensus.cc:3060] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.997134  1718 raft_consensus.cc:515] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:49.998744  1718 leader_election.cc:290] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:49.999817  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c80373e664640b2ccb1aba6ecd425" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:49.999857  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763c80373e664640b2ccb1aba6ecd425" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:50.000444  1642 raft_consensus.cc:3055] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:50.000514  1793 raft_consensus.cc:3060] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.000764  1642 raft_consensus.cc:740] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:50.001611  1642 consensus_queue.cc:260] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:50.002550  1642 raft_consensus.cc:3060] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.005718  1793 raft_consensus.cc:2468] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 2.
I20260709 13:11:50.006737  1684 leader_election.cc:304] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:50.007411  1840 raft_consensus.cc:2804] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:50.007792  1840 raft_consensus.cc:697] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:50.008004  1642 raft_consensus.cc:2468] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 2.
I20260709 13:11:50.008520  1840 consensus_queue.cc:237] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:50.016525  1542 catalog_manager.cc:5697] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 1 to 2, leader changed from 75ec73734cb347918a1298a2baa490e5 (127.21.31.65) to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 2 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:50.026024  1866 raft_consensus.cc:993] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b: : Instructing follower d4286d1c49ba4bee8f2f4b7478053898 to start an election
I20260709 13:11:50.026464  1840 raft_consensus.cc:1081] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Signalling peer d4286d1c49ba4bee8f2f4b7478053898 to start an election
I20260709 13:11:50.027815  1793 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66ff68d2564e4b2eab19e1e61bf4c207"
dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
 from {username='slave'} at 127.0.0.1:53296
I20260709 13:11:50.028246  1793 raft_consensus.cc:493] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:50.028489  1793 raft_consensus.cc:3060] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.033010  1793 raft_consensus.cc:515] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:50.034579  1793 leader_election.cc:290] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 2 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:50.035255  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ff68d2564e4b2eab19e1e61bf4c207" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:50.035562  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ff68d2564e4b2eab19e1e61bf4c207" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:50.035892  1642 raft_consensus.cc:3060] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.036196  1718 raft_consensus.cc:3055] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:50.036597  1718 raft_consensus.cc:740] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:50.037426  1718 consensus_queue.cc:260] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:50.038635  1718 raft_consensus.cc:3060] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.042662  1642 raft_consensus.cc:2468] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 2.
I20260709 13:11:50.043807  1757 leader_election.cc:304] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:50.044474  1912 raft_consensus.cc:2804] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:50.044934  1912 raft_consensus.cc:697] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 LEADER]: Becoming Leader. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:50.045264  1718 raft_consensus.cc:2468] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 2.
I20260709 13:11:50.045784  1912 consensus_queue.cc:237] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:50.052532  1542 catalog_manager.cc:5697] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 reported cstate change: term changed from 1 to 2, leader changed from 3a60ed84fa294767b85c19013457b81b (127.21.31.66) to d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:50.058117  1793 raft_consensus.cc:1275] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Refusing update from remote peer 75ec73734cb347918a1298a2baa490e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.059427  1865 consensus_queue.cc:1048] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:50.066838  1718 raft_consensus.cc:3060] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.075584  1718 raft_consensus.cc:1275] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Refusing update from remote peer 75ec73734cb347918a1298a2baa490e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.077250  1867 consensus_queue.cc:1048] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:50.133459  1642 raft_consensus.cc:1275] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.134968  1840 consensus_queue.cc:1048] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:50.142536  1793 raft_consensus.cc:1275] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.144315  1840 consensus_queue.cc:1048] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:50.538362  1718 raft_consensus.cc:1275] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Refusing update from remote peer d4286d1c49ba4bee8f2f4b7478053898: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.540271  1912 consensus_queue.cc:1048] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:50.546689  1793 raft_consensus.cc:1275] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.548844  1840 consensus_queue.cc:1048] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:50.565634  1642 raft_consensus.cc:1275] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.571621  1840 consensus_queue.cc:1048] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:50.572624  1642 raft_consensus.cc:1275] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Refusing update from remote peer d4286d1c49ba4bee8f2f4b7478053898: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.576171  1924 consensus_queue.cc:1048] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:52.555871  1539 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60402:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:11:52.558911  1539 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:11:52.613986  1631 tablet_service.cc:1511] Processing CreateTablet for tablet c125d13005a14b7e922c42382b90747a (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:52.615077  1704 tablet_service.cc:1511] Processing CreateTablet for tablet c125d13005a14b7e922c42382b90747a (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:52.615932  1631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c125d13005a14b7e922c42382b90747a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.616650  1783 tablet_service.cc:1511] Processing CreateTablet for tablet c125d13005a14b7e922c42382b90747a (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:11:52.617921  1633 tablet_service.cc:1511] Processing CreateTablet for tablet 8ba207fc5f4944e4ad08e006f58a49eb (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:52.619299  1633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba207fc5f4944e4ad08e006f58a49eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.621129  1780 tablet_service.cc:1511] Processing CreateTablet for tablet 8ba207fc5f4944e4ad08e006f58a49eb (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:52.623497  1706 tablet_service.cc:1511] Processing CreateTablet for tablet 8ba207fc5f4944e4ad08e006f58a49eb (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:11:52.623507  1627 tablet_service.cc:1511] Processing CreateTablet for tablet bf793f30e6ae497799b0f68f771e840b (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:52.616721  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c125d13005a14b7e922c42382b90747a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.625968  1627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf793f30e6ae497799b0f68f771e840b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.630689  1707 tablet_service.cc:1511] Processing CreateTablet for tablet bf793f30e6ae497799b0f68f771e840b (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:52.629144  1781 tablet_service.cc:1511] Processing CreateTablet for tablet bf793f30e6ae497799b0f68f771e840b (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:11:52.631779  1706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba207fc5f4944e4ad08e006f58a49eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.634541  1777 tablet_service.cc:1511] Processing CreateTablet for tablet f853b6de82ff4282b7626c1009031ed3 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:52.632994  1632 tablet_service.cc:1511] Processing CreateTablet for tablet f853b6de82ff4282b7626c1009031ed3 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:52.635802  1634 tablet_service.cc:1511] Processing CreateTablet for tablet 7d7e38e6d7aa41cab3272ec7143f3a6b (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:52.636718  1632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f853b6de82ff4282b7626c1009031ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.618355  1783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c125d13005a14b7e922c42382b90747a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.637382  1634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d7e38e6d7aa41cab3272ec7143f3a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.639395  1779 tablet_service.cc:1511] Processing CreateTablet for tablet 7d7e38e6d7aa41cab3272ec7143f3a6b (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:52.641906  1630 tablet_service.cc:1511] Processing CreateTablet for tablet 33f8e265000542f3830defbaca47151f (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:52.648514  1630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33f8e265000542f3830defbaca47151f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.642653  1708 tablet_service.cc:1511] Processing CreateTablet for tablet f853b6de82ff4282b7626c1009031ed3 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:11:52.645076  1703 tablet_service.cc:1511] Processing CreateTablet for tablet 7d7e38e6d7aa41cab3272ec7143f3a6b (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:11:52.646170  1782 tablet_service.cc:1511] Processing CreateTablet for tablet 33f8e265000542f3830defbaca47151f (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:52.654368  1628 tablet_service.cc:1511] Processing CreateTablet for tablet 5ad46f2fe7044ff390bfdeb77c22fed5 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:52.655731  1628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad46f2fe7044ff390bfdeb77c22fed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.646612  1709 tablet_service.cc:1511] Processing CreateTablet for tablet 33f8e265000542f3830defbaca47151f (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:11:52.647010  1707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf793f30e6ae497799b0f68f771e840b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.659649  1708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f853b6de82ff4282b7626c1009031ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.661643  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33f8e265000542f3830defbaca47151f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.664743  1778 tablet_service.cc:1511] Processing CreateTablet for tablet 5ad46f2fe7044ff390bfdeb77c22fed5 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:52.641276  1779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d7e38e6d7aa41cab3272ec7143f3a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.666113  1703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d7e38e6d7aa41cab3272ec7143f3a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.666173  1778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad46f2fe7044ff390bfdeb77c22fed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.667624  1782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33f8e265000542f3830defbaca47151f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.668594  1705 tablet_service.cc:1511] Processing CreateTablet for tablet 5ad46f2fe7044ff390bfdeb77c22fed5 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:11:52.670104  1705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad46f2fe7044ff390bfdeb77c22fed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.673671  1780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba207fc5f4944e4ad08e006f58a49eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.677230  1777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f853b6de82ff4282b7626c1009031ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.689639  1781 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf793f30e6ae497799b0f68f771e840b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.742481  1941 tablet_bootstrap.cc:492] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:52.751456  1941 tablet_bootstrap.cc:654] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.757927  1940 tablet_bootstrap.cc:492] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:52.766156  1941 tablet_bootstrap.cc:492] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:52.766793  1941 ts_tablet_manager.cc:1403] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.000s
I20260709 13:11:52.767158  1940 tablet_bootstrap.cc:654] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.770066  1941 raft_consensus.cc:359] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.770917  1941 raft_consensus.cc:385] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.771262  1941 raft_consensus.cc:740] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.772156  1941 consensus_queue.cc:260] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.776388  1630 tablet_service.cc:1511] Processing CreateTablet for tablet 5e5087882aa34d15b0c6c1c81d8e6fc0 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:52.777671  1630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e5087882aa34d15b0c6c1c81d8e6fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.779973  1783 tablet_service.cc:1511] Processing CreateTablet for tablet 5e5087882aa34d15b0c6c1c81d8e6fc0 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:52.782618  1943 tablet_bootstrap.cc:492] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:52.782974  1941 ts_tablet_manager.cc:1434] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:11:52.785580  1701 tablet_service.cc:1511] Processing CreateTablet for tablet 5e5087882aa34d15b0c6c1c81d8e6fc0 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:11:52.786825  1701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e5087882aa34d15b0c6c1c81d8e6fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.786226  1941 tablet_bootstrap.cc:492] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:52.783069  1628 tablet_service.cc:1511] Processing CreateTablet for tablet 45048a76026e49149b143ba4eed71501 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:52.798066  1628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45048a76026e49149b143ba4eed71501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.803800  1943 tablet_bootstrap.cc:654] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.781219  1783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e5087882aa34d15b0c6c1c81d8e6fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.811025  1778 tablet_service.cc:1511] Processing CreateTablet for tablet 45048a76026e49149b143ba4eed71501 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:52.812309  1778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45048a76026e49149b143ba4eed71501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.815438  1941 tablet_bootstrap.cc:654] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:52.818358  1590 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:52.830516  1941 tablet_bootstrap.cc:492] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:52.831352  1941 ts_tablet_manager.cc:1403] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.005s
I20260709 13:11:52.834980  1941 raft_consensus.cc:359] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.835845  1941 raft_consensus.cc:385] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.836192  1941 raft_consensus.cc:740] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.837263  1941 consensus_queue.cc:260] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.838722  1709 tablet_service.cc:1511] Processing CreateTablet for tablet 45048a76026e49149b143ba4eed71501 (DEFAULT_TABLE table=soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:11:52.846259  1941 ts_tablet_manager.cc:1434] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 13:11:52.847505  1940 tablet_bootstrap.cc:492] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:52.851634  1940 ts_tablet_manager.cc:1403] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.008s
I20260709 13:11:52.840516  1709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45048a76026e49149b143ba4eed71501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:52.871752  1940 raft_consensus.cc:359] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.872689  1940 raft_consensus.cc:385] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.873010  1940 raft_consensus.cc:740] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.873929  1940 consensus_queue.cc:260] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.884111  1943 tablet_bootstrap.cc:492] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:52.898602  1943 ts_tablet_manager.cc:1403] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.116s	user 0.023s	sys 0.060s
I20260709 13:11:52.894703  1941 tablet_bootstrap.cc:492] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:52.901360  1940 ts_tablet_manager.cc:1434] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.001s
I20260709 13:11:52.904884  1940 tablet_bootstrap.cc:492] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:52.911190  1940 tablet_bootstrap.cc:654] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.915884  1941 tablet_bootstrap.cc:654] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.916513  1943 raft_consensus.cc:359] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.917680  1943 raft_consensus.cc:385] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.918139  1943 raft_consensus.cc:740] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.919802  1943 consensus_queue.cc:260] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.923578  1943 ts_tablet_manager.cc:1434] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:11:52.925006  1943 tablet_bootstrap.cc:492] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:52.935371  1943 tablet_bootstrap.cc:654] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.936203  1940 tablet_bootstrap.cc:492] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:52.936800  1940 ts_tablet_manager.cc:1403] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.019s
I20260709 13:11:52.937477  1941 tablet_bootstrap.cc:492] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:52.938148  1941 ts_tablet_manager.cc:1403] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.016s
I20260709 13:11:52.941272  1941 raft_consensus.cc:359] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.942034  1941 raft_consensus.cc:385] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.942330  1941 raft_consensus.cc:740] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.943356  1941 consensus_queue.cc:260] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.943020  1940 raft_consensus.cc:359] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.944479  1940 raft_consensus.cc:385] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.944949  1940 raft_consensus.cc:740] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.946035  1941 ts_tablet_manager.cc:1434] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:52.947108  1941 tablet_bootstrap.cc:492] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:52.948369  1940 consensus_queue.cc:260] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.951825  1940 ts_tablet_manager.cc:1434] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.006s
I20260709 13:11:52.953102  1940 tablet_bootstrap.cc:492] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:52.953537  1941 tablet_bootstrap.cc:654] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.964910  1943 tablet_bootstrap.cc:492] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:52.965469  1943 ts_tablet_manager.cc:1403] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.014s
I20260709 13:11:52.966652  1940 tablet_bootstrap.cc:654] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.968658  1943 raft_consensus.cc:359] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.969470  1943 raft_consensus.cc:385] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.969810  1943 raft_consensus.cc:740] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.970675  1943 consensus_queue.cc:260] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.972995  1941 tablet_bootstrap.cc:492] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
W20260709 13:11:52.973843  1819 tablet.cc:2406] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:11:52.975504  1943 ts_tablet_manager.cc:1434] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:11:52.976596  1943 tablet_bootstrap.cc:492] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:52.982151  1941 ts_tablet_manager.cc:1403] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.010s
I20260709 13:11:52.983122  1943 tablet_bootstrap.cc:654] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.987042  1941 raft_consensus.cc:359] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:52.988279  1941 raft_consensus.cc:385] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.988718  1941 raft_consensus.cc:740] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.990007  1941 consensus_queue.cc:260] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.001511  1941 ts_tablet_manager.cc:1434] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:11:53.002619  1941 tablet_bootstrap.cc:492] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:53.003247  1943 tablet_bootstrap.cc:492] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:53.004006  1943 ts_tablet_manager.cc:1403] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.028s	user 0.021s	sys 0.005s
I20260709 13:11:53.006779  1943 raft_consensus.cc:359] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.007308  1943 raft_consensus.cc:385] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.007565  1943 raft_consensus.cc:740] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.008210  1943 consensus_queue.cc:260] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.003326  1940 tablet_bootstrap.cc:492] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:53.009063  1941 tablet_bootstrap.cc:654] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.009469  1940 ts_tablet_manager.cc:1403] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.057s	user 0.024s	sys 0.018s
I20260709 13:11:53.011961  1943 ts_tablet_manager.cc:1434] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:11:53.012894  1943 tablet_bootstrap.cc:492] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:53.014869  1940 raft_consensus.cc:359] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.015594  1940 raft_consensus.cc:385] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.015913  1940 raft_consensus.cc:740] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.016132  1941 tablet_bootstrap.cc:492] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:53.016698  1941 ts_tablet_manager.cc:1403] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:11:53.016845  1940 consensus_queue.cc:260] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.020155  1941 raft_consensus.cc:359] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.021116  1941 raft_consensus.cc:385] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.021539  1941 raft_consensus.cc:740] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.022995  1941 consensus_queue.cc:260] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.023478  1940 ts_tablet_manager.cc:1434] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 13:11:53.025097  1940 tablet_bootstrap.cc:492] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:53.025720  1941 ts_tablet_manager.cc:1434] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:11:53.027012  1941 tablet_bootstrap.cc:492] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:53.028260  1943 tablet_bootstrap.cc:654] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.032886  1940 tablet_bootstrap.cc:654] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.035344  1941 tablet_bootstrap.cc:654] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.062961  1940 tablet_bootstrap.cc:492] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:53.063508  1943 tablet_bootstrap.cc:492] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:53.063642  1940 ts_tablet_manager.cc:1403] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.009s
I20260709 13:11:53.064299  1943 ts_tablet_manager.cc:1403] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.052s	user 0.023s	sys 0.006s
I20260709 13:11:53.070587  1940 raft_consensus.cc:359] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.070940  1943 raft_consensus.cc:359] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.071517  1941 tablet_bootstrap.cc:492] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:53.071364  1940 raft_consensus.cc:385] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.071779  1943 raft_consensus.cc:385] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.072182  1940 raft_consensus.cc:740] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.072331  1941 ts_tablet_manager.cc:1403] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.011s
I20260709 13:11:53.072335  1943 raft_consensus.cc:740] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.073041  1940 consensus_queue.cc:260] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.075387  1940 ts_tablet_manager.cc:1434] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.002s
I20260709 13:11:53.076480  1940 tablet_bootstrap.cc:492] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:53.078001  1943 consensus_queue.cc:260] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.076503  1941 raft_consensus.cc:359] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.079490  1941 raft_consensus.cc:385] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.079785  1941 raft_consensus.cc:740] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.080446  1943 ts_tablet_manager.cc:1434] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:11:53.081471  1943 tablet_bootstrap.cc:492] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:53.083139  1940 tablet_bootstrap.cc:654] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.081459  1941 consensus_queue.cc:260] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.093691  1941 ts_tablet_manager.cc:1434] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 13:11:53.095286  1941 tablet_bootstrap.cc:492] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:53.097774  1943 tablet_bootstrap.cc:654] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.104334  1839 raft_consensus.cc:493] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.104897  1839 raft_consensus.cc:515] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.107146  1839 leader_election.cc:290] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.120520  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f8e265000542f3830defbaca47151f" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:53.123733  1924 raft_consensus.cc:493] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.124185  1793 raft_consensus.cc:2468] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:53.124228  1924 raft_consensus.cc:515] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.128026  1943 tablet_bootstrap.cc:492] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:53.128563  1943 ts_tablet_manager.cc:1403] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.012s
I20260709 13:11:53.128372  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45048a76026e49149b143ba4eed71501" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:53.123747  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f8e265000542f3830defbaca47151f" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
W20260709 13:11:53.132026  1756 leader_election.cc:343] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.132932  1866 raft_consensus.cc:493] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.133486  1866 raft_consensus.cc:515] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.133845  1609 leader_election.cc:304] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.135229  1839 raft_consensus.cc:2804] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.135717  1839 raft_consensus.cc:493] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.136121  1839 raft_consensus.cc:3060] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.136421  1866 leader_election.cc:290] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
W20260709 13:11:53.137712  1607 leader_election.cc:343] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.137809  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45048a76026e49149b143ba4eed71501" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:53.138819  1924 leader_election.cc:290] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:53.139320  1941 tablet_bootstrap.cc:654] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.145270  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d7e38e6d7aa41cab3272ec7143f3a6b" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:53.145934  1793 raft_consensus.cc:2468] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:53.146746  1943 raft_consensus.cc:359] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.147531  1684 leader_election.cc:304] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.151818  1866 raft_consensus.cc:2804] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.152390  1866 raft_consensus.cc:493] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.152877  1866 raft_consensus.cc:3060] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.147629  1839 raft_consensus.cc:515] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.155073  1954 raft_consensus.cc:493] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.155905  1839 leader_election.cc:290] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.155635  1954 raft_consensus.cc:515] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.156083  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f8e265000542f3830defbaca47151f" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:53.158031  1954 leader_election.cc:290] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.159674  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad46f2fe7044ff390bfdeb77c22fed5" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:53.158350  1940 tablet_bootstrap.cc:492] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:53.161182  1940 ts_tablet_manager.cc:1403] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.085s	user 0.035s	sys 0.014s
W20260709 13:11:53.161295  1607 leader_election.cc:343] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.163060  1866 raft_consensus.cc:515] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.164731  1941 tablet_bootstrap.cc:492] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:53.165302  1940 raft_consensus.cc:359] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.166150  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d7e38e6d7aa41cab3272ec7143f3a6b" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.166322  1940 raft_consensus.cc:385] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.166353  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f8e265000542f3830defbaca47151f" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
W20260709 13:11:53.166564  1757 leader_election.cc:343] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.166806  1940 raft_consensus.cc:740] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.167138  1791 raft_consensus.cc:3060] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.168004  1793 raft_consensus.cc:3060] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.168054  1940 consensus_queue.cc:260] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.171118  1940 ts_tablet_manager.cc:1434] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:11:53.172116  1940 tablet_bootstrap.cc:492] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:53.167129  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad46f2fe7044ff390bfdeb77c22fed5" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:53.173746  1792 raft_consensus.cc:2468] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:53.165298  1941 ts_tablet_manager.cc:1403] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.011s
I20260709 13:11:53.177137  1866 leader_election.cc:290] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.178345  1954 raft_consensus.cc:493] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.178632  1940 tablet_bootstrap.cc:654] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.178886  1954 raft_consensus.cc:515] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.179756  1941 raft_consensus.cc:359] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.180325  1941 raft_consensus.cc:385] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.180562  1941 raft_consensus.cc:740] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.181010  1954 leader_election.cc:290] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.176803  1943 raft_consensus.cc:385] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.181564  1943 raft_consensus.cc:740] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.167161  1757 leader_election.cc:304] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4286d1c49ba4bee8f2f4b7478053898; no voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5
I20260709 13:11:53.176259  1791 raft_consensus.cc:2468] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:53.176219  1793 raft_consensus.cc:2468] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:53.182305  1924 raft_consensus.cc:2749] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:53.182305  1943 consensus_queue.cc:260] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.183049  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c125d13005a14b7e922c42382b90747a" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:53.183801  1718 raft_consensus.cc:3060] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.184012  1717 raft_consensus.cc:2468] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:53.183225  1684 leader_election.cc:304] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.185468  1607 leader_election.cc:304] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:53.185617  1960 raft_consensus.cc:2804] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.186240  1960 raft_consensus.cc:697] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:53.187188  1960 consensus_queue.cc:237] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.187678  1940 tablet_bootstrap.cc:492] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:53.188208  1940 ts_tablet_manager.cc:1403] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.005s
I20260709 13:11:53.190356  1954 raft_consensus.cc:2804] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.190774  1954 raft_consensus.cc:493] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.191177  1954 raft_consensus.cc:3060] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.190984  1940 raft_consensus.cc:359] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.191767  1940 raft_consensus.cc:385] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.192196  1940 raft_consensus.cc:740] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.193003  1940 consensus_queue.cc:260] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.193344  1609 leader_election.cc:304] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:53.191538  1718 raft_consensus.cc:2468] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:53.181234  1941 consensus_queue.cc:260] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.194105  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c125d13005a14b7e922c42382b90747a" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:53.201403  1791 raft_consensus.cc:2468] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:53.202917  1941 ts_tablet_manager.cc:1434] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.026s	user 0.000s	sys 0.006s
I20260709 13:11:53.203943  1941 tablet_bootstrap.cc:492] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:53.205799  1924 raft_consensus.cc:493] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.206275  1924 raft_consensus.cc:515] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.208380  1924 leader_election.cc:290] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:53.195990  1839 raft_consensus.cc:2804] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.209882  1839 raft_consensus.cc:493] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.199617  1607 leader_election.cc:304] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:53.210238  1839 raft_consensus.cc:3060] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.211341  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:53.200640  1943 ts_tablet_manager.cc:1434] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.000s
I20260709 13:11:53.215122  1943 tablet_bootstrap.cc:492] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:53.210186  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:53.218153  1966 raft_consensus.cc:2804] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:11:53.218885  1756 leader_election.cc:343] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.219290  1966 raft_consensus.cc:697] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:53.222242  1839 raft_consensus.cc:515] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.222805  1966 consensus_queue.cc:237] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
W20260709 13:11:53.219749  1757 leader_election.cc:343] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.226979  1757 leader_election.cc:304] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4286d1c49ba4bee8f2f4b7478053898; no voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5
I20260709 13:11:53.227677  1924 raft_consensus.cc:2749] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:53.227818  1940 ts_tablet_manager.cc:1434] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.039s	user 0.014s	sys 0.023s
I20260709 13:11:53.225436  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad46f2fe7044ff390bfdeb77c22fed5" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:53.225529  1839 leader_election.cc:290] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.228690  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad46f2fe7044ff390bfdeb77c22fed5" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.226652  1941 tablet_bootstrap.cc:654] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.228916  1940 tablet_bootstrap.cc:492] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
W20260709 13:11:53.229846  1607 leader_election.cc:343] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.230615  1791 raft_consensus.cc:3060] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.239867  1940 tablet_bootstrap.cc:654] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.243198  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d7e38e6d7aa41cab3272ec7143f3a6b" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:53.244344  1644 raft_consensus.cc:2468] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:53.247020  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d7e38e6d7aa41cab3272ec7143f3a6b" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:53.247788  1644 raft_consensus.cc:3060] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.251233  1791 raft_consensus.cc:2468] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:53.252954  1609 leader_election.cc:304] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:53.254555  1839 raft_consensus.cc:2804] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.255081  1839 raft_consensus.cc:697] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:53.256062  1839 consensus_queue.cc:237] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.257604  1943 tablet_bootstrap.cc:654] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.261886  1644 raft_consensus.cc:2468] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:53.262984  1954 raft_consensus.cc:515] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.268435  1954 leader_election.cc:290] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.269019  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c125d13005a14b7e922c42382b90747a" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:53.271329  1718 raft_consensus.cc:3060] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.272600  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c125d13005a14b7e922c42382b90747a" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.273396  1791 raft_consensus.cc:3060] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.278349  1539 catalog_manager.cc:5697] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 1 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.289677  1941 tablet_bootstrap.cc:492] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:53.290328  1940 tablet_bootstrap.cc:492] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:53.290199  1791 raft_consensus.cc:2468] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:53.290225  1941 ts_tablet_manager.cc:1403] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.087s	user 0.022s	sys 0.027s
I20260709 13:11:53.290904  1940 ts_tablet_manager.cc:1403] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.062s	user 0.025s	sys 0.025s
I20260709 13:11:53.291589  1609 leader_election.cc:304] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.292573  1965 raft_consensus.cc:2804] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.292346  1542 catalog_manager.cc:5697] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.293529  1941 raft_consensus.cc:359] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.294127  1941 raft_consensus.cc:385] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.293913  1940 raft_consensus.cc:359] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.293037  1965 raft_consensus.cc:697] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:53.294710  1940 raft_consensus.cc:385] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.295083  1940 raft_consensus.cc:740] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.294415  1941 raft_consensus.cc:740] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.295476  1965 consensus_queue.cc:237] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.295862  1943 tablet_bootstrap.cc:492] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:53.295848  1940 consensus_queue.cc:260] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.296633  1943 ts_tablet_manager.cc:1403] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.082s	user 0.010s	sys 0.002s
I20260709 13:11:53.296664  1941 consensus_queue.cc:260] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.298767  1940 ts_tablet_manager.cc:1434] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:11:53.298911  1941 ts_tablet_manager.cc:1434] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:11:53.299844  1940 tablet_bootstrap.cc:492] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:53.301744  1718 raft_consensus.cc:2468] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:53.302193  1943 raft_consensus.cc:359] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.302973  1943 raft_consensus.cc:385] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.303309  1943 raft_consensus.cc:740] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.304103  1943 consensus_queue.cc:260] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.305497  1941 tablet_bootstrap.cc:492] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898: Bootstrap starting.
I20260709 13:11:53.311275  1943 ts_tablet_manager.cc:1434] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 13:11:53.313452  1941 tablet_bootstrap.cc:654] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.314181  1943 tablet_bootstrap.cc:492] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:53.308885  1940 tablet_bootstrap.cc:654] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.326435  1943 tablet_bootstrap.cc:654] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.327780  1940 tablet_bootstrap.cc:492] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:53.328431  1940 ts_tablet_manager.cc:1403] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.001s
I20260709 13:11:53.330749  1924 raft_consensus.cc:493] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.331382  1924 raft_consensus.cc:515] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.332057  1940 raft_consensus.cc:359] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.332989  1940 raft_consensus.cc:385] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.334007  1924 leader_election.cc:290] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:53.334903  1940 raft_consensus.cc:740] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.335474  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f853b6de82ff4282b7626c1009031ed3" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:53.335640  1940 consensus_queue.cc:260] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.336143  1718 raft_consensus.cc:2468] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 0.
I20260709 13:11:53.337080  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f853b6de82ff4282b7626c1009031ed3" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:53.337714  1644 raft_consensus.cc:2468] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 0.
I20260709 13:11:53.337466  1756 leader_election.cc:304] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.337644  1940 ts_tablet_manager.cc:1434] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:11:53.338557  1924 raft_consensus.cc:2804] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.339147  1924 raft_consensus.cc:493] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.339563  1924 raft_consensus.cc:3060] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.339655  1940 tablet_bootstrap.cc:492] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5: Bootstrap starting.
I20260709 13:11:53.342346  1943 tablet_bootstrap.cc:492] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:53.345350  1943 ts_tablet_manager.cc:1403] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.005s
I20260709 13:11:53.347088  1940 tablet_bootstrap.cc:654] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.347465  1924 raft_consensus.cc:515] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.348567  1943 raft_consensus.cc:359] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.349346  1943 raft_consensus.cc:385] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.349694  1943 raft_consensus.cc:740] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.350836  1941 tablet_bootstrap.cc:492] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898: No bootstrap required, opened a new log
I20260709 13:11:53.350551  1943 consensus_queue.cc:260] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.350987  1954 raft_consensus.cc:493] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.351346  1941 ts_tablet_manager.cc:1403] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898: Time spent bootstrapping tablet: real 0.046s	user 0.007s	sys 0.017s
I20260709 13:11:53.351691  1954 raft_consensus.cc:515] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.353188  1943 ts_tablet_manager.cc:1434] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:53.354193  1943 tablet_bootstrap.cc:492] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:53.354084  1941 raft_consensus.cc:359] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.354812  1941 raft_consensus.cc:385] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.356695  1941 raft_consensus.cc:740] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.356796  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f853b6de82ff4282b7626c1009031ed3" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:53.357578  1718 raft_consensus.cc:3060] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.357650  1941 consensus_queue.cc:260] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.360100  1941 ts_tablet_manager.cc:1434] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:11:53.362246  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e5087882aa34d15b0c6c1c81d8e6fc0" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:53.363078  1717 raft_consensus.cc:2468] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:53.363385  1943 tablet_bootstrap.cc:654] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.364545  1607 leader_election.cc:304] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:53.365010  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e5087882aa34d15b0c6c1c81d8e6fc0" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:53.365415  1966 raft_consensus.cc:2804] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.350844  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f853b6de82ff4282b7626c1009031ed3" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:53.365864  1966 raft_consensus.cc:493] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.366357  1966 raft_consensus.cc:3060] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.366393  1644 raft_consensus.cc:3060] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.355484  1965 raft_consensus.cc:493] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.369799  1965 raft_consensus.cc:515] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.356292  1924 leader_election.cc:290] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:53.365855  1791 raft_consensus.cc:2468] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:53.379688  1965 leader_election.cc:290] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.383138  1717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b" is_pre_election: true
I20260709 13:11:53.380661  1966 raft_consensus.cc:515] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.388912  1644 raft_consensus.cc:2468] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 1.
I20260709 13:11:53.381695  1954 leader_election.cc:290] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.392143  1966 leader_election.cc:290] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.392603  1718 raft_consensus.cc:2468] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 1.
I20260709 13:11:53.391033  1943 tablet_bootstrap.cc:492] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:53.393558  1943 ts_tablet_manager.cc:1403] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.040s	user 0.006s	sys 0.021s
I20260709 13:11:53.393823  1756 leader_election.cc:304] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.394726  1924 raft_consensus.cc:2804] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.395211  1924 raft_consensus.cc:697] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Becoming Leader. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:53.396090  1943 raft_consensus.cc:359] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.396037  1924 consensus_queue.cc:237] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.396517  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:53.389782  1717 raft_consensus.cc:2468] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:53.397176  1791 raft_consensus.cc:2468] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 0.
I20260709 13:11:53.397535  1969 raft_consensus.cc:493] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.397900  1969 raft_consensus.cc:515] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.398272  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e5087882aa34d15b0c6c1c81d8e6fc0" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.398943  1793 raft_consensus.cc:3060] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.401901  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:53.402475  1644 raft_consensus.cc:2468] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:53.403559  1969 leader_election.cc:290] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.403613  1683 leader_election.cc:304] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:53.404639  1969 raft_consensus.cc:2804] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.404841  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e5087882aa34d15b0c6c1c81d8e6fc0" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:53.405057  1969 raft_consensus.cc:493] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.405507  1718 raft_consensus.cc:3060] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.405507  1969 raft_consensus.cc:3060] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.404657  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:53.409966  1791 raft_consensus.cc:2468] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:53.405611  1609 leader_election.cc:304] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.412544  1718 raft_consensus.cc:2468] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:53.396854  1943 raft_consensus.cc:385] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.412914  1969 raft_consensus.cc:515] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.413492  1943 raft_consensus.cc:740] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.413599  1542 catalog_manager.cc:5697] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.414211  1943 consensus_queue.cc:260] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.415238  1542 catalog_manager.cc:5697] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.415846  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:53.416298  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.416421  1644 raft_consensus.cc:3060] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.416321  1943 ts_tablet_manager.cc:1434] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.001s
I20260709 13:11:53.416893  1791 raft_consensus.cc:3060] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.417871  1607 leader_election.cc:304] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:53.419381  1943 tablet_bootstrap.cc:492] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b: Bootstrap starting.
I20260709 13:11:53.420215  1793 raft_consensus.cc:2468] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 1.
I20260709 13:11:53.424080  1791 raft_consensus.cc:2468] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:53.424801  1969 leader_election.cc:290] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.425391  1943 tablet_bootstrap.cc:654] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.425706  1684 leader_election.cc:304] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.426072  1644 raft_consensus.cc:2468] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:53.426535  1969 raft_consensus.cc:2804] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.426617  1954 raft_consensus.cc:2764] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:11:53.427099  1969 raft_consensus.cc:697] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:53.428063  1969 consensus_queue.cc:237] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.428933  1954 raft_consensus.cc:2804] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.429814  1954 raft_consensus.cc:697] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:53.430914  1954 consensus_queue.cc:237] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.438105  1940 tablet_bootstrap.cc:492] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5: No bootstrap required, opened a new log
I20260709 13:11:53.438576  1940 ts_tablet_manager.cc:1403] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5: Time spent bootstrapping tablet: real 0.099s	user 0.020s	sys 0.021s
I20260709 13:11:53.441390  1940 raft_consensus.cc:359] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.442013  1940 raft_consensus.cc:385] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.442282  1940 raft_consensus.cc:740] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.442974  1940 consensus_queue.cc:260] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.444954  1940 ts_tablet_manager.cc:1434] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:53.446882  1539 catalog_manager.cc:5697] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67). New cstate: current_term: 1 leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:53.448029  1943 tablet_bootstrap.cc:492] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b: No bootstrap required, opened a new log
I20260709 13:11:53.448475  1943 ts_tablet_manager.cc:1403] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.007s
I20260709 13:11:53.452168  1943 raft_consensus.cc:359] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.452679  1943 raft_consensus.cc:385] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.452908  1943 raft_consensus.cc:740] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.453501  1943 consensus_queue.cc:260] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.455015  1943 ts_tablet_manager.cc:1434] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:53.458554  1541 catalog_manager.cc:5697] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 1 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.467443  1542 catalog_manager.cc:5697] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 1 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.508018  1960 raft_consensus.cc:493] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.508468  1960 raft_consensus.cc:515] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.510413  1960 leader_election.cc:290] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.510942  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:53.511399  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:53.511601  1644 raft_consensus.cc:2468] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:53.511988  1793 raft_consensus.cc:2468] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:53.513052  1684 leader_election.cc:304] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.513949  1960 raft_consensus.cc:2804] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.514281  1960 raft_consensus.cc:493] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.514621  1960 raft_consensus.cc:3060] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.520761  1960 raft_consensus.cc:515] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.522639  1960 leader_election.cc:290] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.523452  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:53.523623  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.524065  1644 raft_consensus.cc:3060] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.524437  1793 raft_consensus.cc:3060] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.529598  1644 raft_consensus.cc:2468] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:53.530563  1793 raft_consensus.cc:2468] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:53.530654  1683 leader_election.cc:304] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:53.531466  1960 raft_consensus.cc:2804] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.531976  1960 raft_consensus.cc:697] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:53.532774  1960 consensus_queue.cc:237] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.540493  1542 catalog_manager.cc:5697] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 1 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.584625  1960 consensus_queue.cc:1048] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.597549  1969 consensus_queue.cc:1048] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.638841  1969 raft_consensus.cc:493] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.639437  1969 raft_consensus.cc:515] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.640385  1718 raft_consensus.cc:3060] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.642077  1969 leader_election.cc:290] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.642551  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45048a76026e49149b143ba4eed71501" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5" is_pre_election: true
I20260709 13:11:53.642864  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45048a76026e49149b143ba4eed71501" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898" is_pre_election: true
I20260709 13:11:53.643450  1644 raft_consensus.cc:2468] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:53.643545  1793 raft_consensus.cc:2468] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 0.
I20260709 13:11:53.644673  1683 leader_election.cc:304] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:53.645641  1969 raft_consensus.cc:2804] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.646049  1969 raft_consensus.cc:493] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.646337  1969 raft_consensus.cc:3060] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.650215  1954 consensus_queue.cc:1048] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:53.651115  1966 consensus_queue.cc:1048] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.653271  1969 raft_consensus.cc:515] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.655571  1969 leader_election.cc:290] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.656690  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45048a76026e49149b143ba4eed71501" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.657704  1792 raft_consensus.cc:3060] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.664095  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45048a76026e49149b143ba4eed71501" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:53.664937  1644 raft_consensus.cc:3060] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.672911  1792 raft_consensus.cc:2468] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:53.673367  1644 raft_consensus.cc:2468] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 1.
I20260709 13:11:53.674145  1684 leader_election.cc:304] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.677254  1866 raft_consensus.cc:2804] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.677819  1866 raft_consensus.cc:697] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:53.678665  1866 consensus_queue.cc:237] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.686921  1539 catalog_manager.cc:5697] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 1 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.710399  1954 consensus_queue.cc:1048] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.723057  1954 consensus_queue.cc:1048] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.746292  1954 consensus_queue.cc:1048] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.757385  1954 consensus_queue.cc:1048] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.773142  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
mode: GRACEFUL
new_leader_uuid: "75ec73734cb347918a1298a2baa490e5"
 from {username='slave'} at 127.0.0.1:53950
I20260709 13:11:53.773852  1717 raft_consensus.cc:606] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Received request to transfer leadership to TS 75ec73734cb347918a1298a2baa490e5
I20260709 13:11:53.779358  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
mode: GRACEFUL
new_leader_uuid: "75ec73734cb347918a1298a2baa490e5"
 from {username='slave'} at 127.0.0.1:53950
I20260709 13:11:53.779755  1717 raft_consensus.cc:606] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Received request to transfer leadership to TS 75ec73734cb347918a1298a2baa490e5
I20260709 13:11:53.785312  1792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f853b6de82ff4282b7626c1009031ed3"
dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
mode: GRACEFUL
new_leader_uuid: "75ec73734cb347918a1298a2baa490e5"
 from {username='slave'} at 127.0.0.1:53306
I20260709 13:11:53.785853  1792 raft_consensus.cc:606] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Received request to transfer leadership to TS 75ec73734cb347918a1298a2baa490e5
I20260709 13:11:53.793955  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33f8e265000542f3830defbaca47151f"
dest_uuid: "75ec73734cb347918a1298a2baa490e5"
mode: GRACEFUL
new_leader_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:33130
I20260709 13:11:53.794561  1644 raft_consensus.cc:606] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Received request to transfer leadership to TS 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:53.800011  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ad46f2fe7044ff390bfdeb77c22fed5"
dest_uuid: "75ec73734cb347918a1298a2baa490e5"
mode: GRACEFUL
new_leader_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:33130
I20260709 13:11:53.800508  1644 raft_consensus.cc:606] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Received request to transfer leadership to TS 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:53.805516  1644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e5087882aa34d15b0c6c1c81d8e6fc0"
dest_uuid: "75ec73734cb347918a1298a2baa490e5"
mode: GRACEFUL
new_leader_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:33130
I20260709 13:11:53.806124  1644 raft_consensus.cc:606] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Received request to transfer leadership to TS 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:53.810868  1717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45048a76026e49149b143ba4eed71501"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
mode: GRACEFUL
new_leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
 from {username='slave'} at 127.0.0.1:53950
I20260709 13:11:53.811290  1717 raft_consensus.cc:606] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Received request to transfer leadership to TS d4286d1c49ba4bee8f2f4b7478053898
I20260709 13:11:53.823683  1924 consensus_queue.cc:1048] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.831645  1924 raft_consensus.cc:993] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898: : Instructing follower 75ec73734cb347918a1298a2baa490e5 to start an election
I20260709 13:11:53.832044  1924 raft_consensus.cc:1081] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Signalling peer 75ec73734cb347918a1298a2baa490e5 to start an election
I20260709 13:11:53.835806  1642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f853b6de82ff4282b7626c1009031ed3"
dest_uuid: "75ec73734cb347918a1298a2baa490e5"
 from {username='slave'} at 127.0.0.1:33116
I20260709 13:11:53.836383  1642 raft_consensus.cc:493] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:53.836761  1642 raft_consensus.cc:3060] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.840394  1961 consensus_queue.cc:1048] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:53.844894  1642 raft_consensus.cc:515] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.847509  1642 leader_election.cc:290] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 2 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.848584  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f853b6de82ff4282b7626c1009031ed3" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.849344  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f853b6de82ff4282b7626c1009031ed3" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:53.849175  1792 raft_consensus.cc:3055] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:53.850227  1792 raft_consensus.cc:740] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:53.851016  1792 consensus_queue.cc:260] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.851909  1792 raft_consensus.cc:3060] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.859614  1792 raft_consensus.cc:2468] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 2.
I20260709 13:11:53.860725  1609 leader_election.cc:304] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 3a60ed84fa294767b85c19013457b81b
I20260709 13:11:53.861992  1954 raft_consensus.cc:2804] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:53.862461  1954 raft_consensus.cc:697] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 2 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:53.863231  1954 consensus_queue.cc:237] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.869387  1541 catalog_manager.cc:5697] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 1 to 2, leader changed from d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67) to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.941114  1866 consensus_queue.cc:1048] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.948832  1960 consensus_queue.cc:1048] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.960238  1966 consensus_queue.cc:1048] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.962143  1969 raft_consensus.cc:993] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b: : Instructing follower 75ec73734cb347918a1298a2baa490e5 to start an election
I20260709 13:11:53.962634  1969 raft_consensus.cc:1081] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Signalling peer 75ec73734cb347918a1298a2baa490e5 to start an election
I20260709 13:11:53.966986  1642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb"
dest_uuid: "75ec73734cb347918a1298a2baa490e5"
 from {username='slave'} at 127.0.0.1:33128
I20260709 13:11:53.968713  1642 raft_consensus.cc:493] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:53.969146  1642 raft_consensus.cc:3060] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.969190  1966 raft_consensus.cc:993] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5: : Instructing follower 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:53.969877  1966 raft_consensus.cc:1081] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Signalling peer 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:53.972069  1718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e5087882aa34d15b0c6c1c81d8e6fc0"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:53248
I20260709 13:11:53.972587  1718 raft_consensus.cc:493] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:53.972921  1718 raft_consensus.cc:3060] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.976569  2003 consensus_queue.cc:1048] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.977047  1642 raft_consensus.cc:515] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.979363  1642 leader_election.cc:290] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 2 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.979907  1718 raft_consensus.cc:515] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.980528  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:53.980921  1717 raft_consensus.cc:1240] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Rejecting Update request from peer 75ec73734cb347918a1298a2baa490e5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:53.981225  1719 raft_consensus.cc:3055] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:53.980629  1966 consensus_queue.cc:1048] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:53.982770  1793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba207fc5f4944e4ad08e006f58a49eb" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.983495  1793 raft_consensus.cc:3060] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.981588  1719 raft_consensus.cc:740] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:53.984851  1718 leader_election.cc:290] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:53.985278  1642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e5087882aa34d15b0c6c1c81d8e6fc0" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:53.985157  1719 consensus_queue.cc:260] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.985957  1642 raft_consensus.cc:3055] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:53.986310  1642 raft_consensus.cc:740] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:53.986893  1719 raft_consensus.cc:3060] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.987093  1642 consensus_queue.cc:260] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.990341  1966 consensus_queue.cc:1059] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.992321  1793 raft_consensus.cc:2468] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 2.
I20260709 13:11:53.993415  1609 leader_election.cc:304] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:53.987141  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e5087882aa34d15b0c6c1c81d8e6fc0" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:53.994084  1642 raft_consensus.cc:3060] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.994539  1969 raft_consensus.cc:993] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b: : Instructing follower 75ec73734cb347918a1298a2baa490e5 to start an election
I20260709 13:11:53.995007  1969 raft_consensus.cc:1081] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Signalling peer 75ec73734cb347918a1298a2baa490e5 to start an election
I20260709 13:11:53.995759  1839 raft_consensus.cc:2804] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:53.996304  1839 raft_consensus.cc:697] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 2 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:53.997345  1643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b"
dest_uuid: "75ec73734cb347918a1298a2baa490e5"
 from {username='slave'} at 127.0.0.1:33128
I20260709 13:11:53.997246  1839 consensus_queue.cc:237] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:53.998028  1643 raft_consensus.cc:493] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:53.998445  1643 raft_consensus.cc:3060] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.999460  1969 consensus_queue.cc:1048] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.004634  1719 raft_consensus.cc:2468] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 2.
I20260709 13:11:54.015908  1642 raft_consensus.cc:2468] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 2.
I20260709 13:11:54.018218  1683 leader_election.cc:304] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: d4286d1c49ba4bee8f2f4b7478053898
I20260709 13:11:54.019409  2003 raft_consensus.cc:2804] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:54.019881  2003 raft_consensus.cc:697] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:54.019972  1643 raft_consensus.cc:515] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.021005  1644 raft_consensus.cc:1240] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Rejecting Update request from peer 3a60ed84fa294767b85c19013457b81b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:54.022833  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:54.020699  2003 consensus_queue.cc:237] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.023476  1960 consensus_queue.cc:1059] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.024335  1719 raft_consensus.cc:3055] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:54.024542  1643 leader_election.cc:290] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 2 election: Requested vote from peers 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:54.024716  1719 raft_consensus.cc:740] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:54.025637  1719 consensus_queue.cc:260] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.025830  1792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf793f30e6ae497799b0f68f771e840b" candidate_uuid: "75ec73734cb347918a1298a2baa490e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:54.026432  1792 raft_consensus.cc:3060] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.026638  1719 raft_consensus.cc:3060] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.032291  1719 raft_consensus.cc:2468] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 2.
W20260709 13:11:54.032672  1969 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:11:54.033149  1792 raft_consensus.cc:2468] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75ec73734cb347918a1298a2baa490e5 in term 2.
I20260709 13:11:54.032526  1541 catalog_manager.cc:5697] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 1 to 2, leader changed from 3a60ed84fa294767b85c19013457b81b (127.21.31.66) to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.034412  1609 leader_election.cc:304] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:54.035583  1839 raft_consensus.cc:2804] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:54.036063  1839 raft_consensus.cc:697] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 2 LEADER]: Becoming Leader. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:54.036806  1839 consensus_queue.cc:237] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.044024  1542 catalog_manager.cc:5697] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 1 to 2, leader changed from 75ec73734cb347918a1298a2baa490e5 (127.21.31.65) to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 2 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.049538  1539 catalog_manager.cc:5697] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 reported cstate change: term changed from 1 to 2, leader changed from 3a60ed84fa294767b85c19013457b81b (127.21.31.66) to 75ec73734cb347918a1298a2baa490e5 (127.21.31.65). New cstate: current_term: 2 leader_uuid: "75ec73734cb347918a1298a2baa490e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.096398  1960 consensus_queue.cc:1048] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.103237  2003 raft_consensus.cc:993] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b: : Instructing follower d4286d1c49ba4bee8f2f4b7478053898 to start an election
I20260709 13:11:54.103955  1969 raft_consensus.cc:1081] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Signalling peer d4286d1c49ba4bee8f2f4b7478053898 to start an election
I20260709 13:11:54.105690  1791 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45048a76026e49149b143ba4eed71501"
dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
 from {username='slave'} at 127.0.0.1:53296
I20260709 13:11:54.106833  1791 raft_consensus.cc:493] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:54.107213  1791 raft_consensus.cc:3060] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.108544  1969 consensus_queue.cc:1048] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.114145  1791 raft_consensus.cc:515] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.116966  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45048a76026e49149b143ba4eed71501" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:54.117471  1791 leader_election.cc:290] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 2 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:54.117542  1643 raft_consensus.cc:3060] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.118711  1719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45048a76026e49149b143ba4eed71501" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:54.119937  1719 raft_consensus.cc:3055] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:54.120304  1719 raft_consensus.cc:740] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:54.121201  1719 consensus_queue.cc:260] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.123152  1719 raft_consensus.cc:3060] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.124718  1643 raft_consensus.cc:2468] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 2.
I20260709 13:11:54.125780  1757 leader_election.cc:304] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:54.126540  1961 raft_consensus.cc:2804] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:54.127017  1961 raft_consensus.cc:697] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 LEADER]: Becoming Leader. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:54.127825  1961 consensus_queue.cc:237] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.129844  1719 raft_consensus.cc:2468] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 2.
I20260709 13:11:54.135095  1542 catalog_manager.cc:5697] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 reported cstate change: term changed from 1 to 2, leader changed from 3a60ed84fa294767b85c19013457b81b (127.21.31.66) to d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:54.136714  1966 raft_consensus.cc:993] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5: : Instructing follower 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:54.137203  1839 raft_consensus.cc:1081] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Signalling peer 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:54.138629  1719 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ad46f2fe7044ff390bfdeb77c22fed5"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:53248
I20260709 13:11:54.139040  1719 raft_consensus.cc:493] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:54.139304  1719 raft_consensus.cc:3060] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.144241  1719 raft_consensus.cc:515] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.145916  1719 leader_election.cc:290] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:54.146679  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad46f2fe7044ff390bfdeb77c22fed5" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:54.146955  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad46f2fe7044ff390bfdeb77c22fed5" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:54.147231  1643 raft_consensus.cc:3055] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:54.147564  1791 raft_consensus.cc:3060] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.147581  1643 raft_consensus.cc:740] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:54.148514  1643 consensus_queue.cc:260] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.149410  1643 raft_consensus.cc:3060] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.152524  1791 raft_consensus.cc:2468] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 2.
I20260709 13:11:54.153621  1684 leader_election.cc:304] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:54.154114  1643 raft_consensus.cc:2468] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 2.
I20260709 13:11:54.154273  2003 raft_consensus.cc:2804] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:54.154781  2003 raft_consensus.cc:697] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:54.155591  2003 consensus_queue.cc:237] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.162678  1542 catalog_manager.cc:5697] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 1 to 2, leader changed from 75ec73734cb347918a1298a2baa490e5 (127.21.31.65) to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 2 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.306180  1791 raft_consensus.cc:1275] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Refusing update from remote peer 75ec73734cb347918a1298a2baa490e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.307418  1839 consensus_queue.cc:1048] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.317106  1719 raft_consensus.cc:3060] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.318728  1966 raft_consensus.cc:993] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5: : Instructing follower 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:54.319227  1966 raft_consensus.cc:1081] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Signalling peer 3a60ed84fa294767b85c19013457b81b to start an election
I20260709 13:11:54.321161  1718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33f8e265000542f3830defbaca47151f"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
 from {username='slave'} at 127.0.0.1:53248
I20260709 13:11:54.321677  1718 raft_consensus.cc:493] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:54.321985  1718 raft_consensus.cc:3060] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.325670  1719 raft_consensus.cc:1275] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Refusing update from remote peer 75ec73734cb347918a1298a2baa490e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.327051  1839 consensus_queue.cc:1048] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.328161  1718 raft_consensus.cc:515] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.330526  1643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f8e265000542f3830defbaca47151f" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:54.330824  1718 leader_election.cc:290] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67:38515)
I20260709 13:11:54.331079  1643 raft_consensus.cc:3055] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:54.331413  1643 raft_consensus.cc:740] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:54.332127  1791 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f8e265000542f3830defbaca47151f" candidate_uuid: "3a60ed84fa294767b85c19013457b81b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
I20260709 13:11:54.332211  1643 consensus_queue.cc:260] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.332785  1791 raft_consensus.cc:3060] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.333567  1643 raft_consensus.cc:3060] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.339162  1643 raft_consensus.cc:2468] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 2.
I20260709 13:11:54.339792  1791 raft_consensus.cc:2468] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a60ed84fa294767b85c19013457b81b in term 2.
I20260709 13:11:54.340308  1683 leader_election.cc:304] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, 75ec73734cb347918a1298a2baa490e5; no voters: 
I20260709 13:11:54.341136  2003 raft_consensus.cc:2804] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:54.341630  2003 raft_consensus.cc:697] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Becoming Leader. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:54.343657  2003 consensus_queue.cc:237] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:54.353000  1539 catalog_manager.cc:5697] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b reported cstate change: term changed from 1 to 2, leader changed from 75ec73734cb347918a1298a2baa490e5 (127.21.31.65) to 3a60ed84fa294767b85c19013457b81b (127.21.31.66). New cstate: current_term: 2 leader_uuid: "3a60ed84fa294767b85c19013457b81b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.380088  1719 raft_consensus.cc:1275] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Refusing update from remote peer 75ec73734cb347918a1298a2baa490e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.381265  1839 consensus_queue.cc:1048] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.388983  1792 raft_consensus.cc:1275] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Refusing update from remote peer 75ec73734cb347918a1298a2baa490e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.390940  1839 consensus_queue.cc:1048] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.444506  1719 raft_consensus.cc:1275] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Refusing update from remote peer 75ec73734cb347918a1298a2baa490e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.446105  1839 consensus_queue.cc:1048] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.454425  1792 raft_consensus.cc:1275] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Refusing update from remote peer 75ec73734cb347918a1298a2baa490e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.456866  1954 consensus_queue.cc:1048] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.520089  1643 raft_consensus.cc:1275] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.521346  2003 consensus_queue.cc:1048] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.533164  1792 raft_consensus.cc:3060] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.541978  1792 raft_consensus.cc:1275] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.543110  1969 consensus_queue.cc:1048] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.563078  1719 raft_consensus.cc:1275] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Refusing update from remote peer d4286d1c49ba4bee8f2f4b7478053898: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.564254  1961 consensus_queue.cc:1048] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.571395  1643 raft_consensus.cc:1275] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Refusing update from remote peer d4286d1c49ba4bee8f2f4b7478053898: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.572906  1961 consensus_queue.cc:1048] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.606163  1643 raft_consensus.cc:1275] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.607651  1969 consensus_queue.cc:1048] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.615146  1792 raft_consensus.cc:1275] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.617486  1960 consensus_queue.cc:1048] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.823283  1792 raft_consensus.cc:1275] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.824580  1960 consensus_queue.cc:1048] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:54.833045  1643 raft_consensus.cc:1275] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Refusing update from remote peer 3a60ed84fa294767b85c19013457b81b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:54.834880  2011 consensus_queue.cc:1048] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:56.814380  1542 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60402:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:11:56.836107  1542 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60402:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:11:56.862319 21629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:57.075073  2062 tablet.cc:1724] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:57.141862  2052 log.cc:927] Time spent T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:11:57.161384  2068 tablet.cc:1724] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:11:57.163374  2044 log.cc:927] Time spent T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898: Append to log took a long time: real 0.161s	user 0.005s	sys 0.000s
I20260709 13:11:57.175944  2067 tablet.cc:1724] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.169420  2065 tablet.cc:1724] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.213243  2075 tablet.cc:1724] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.186878  2073 tablet.cc:1724] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.222677  2071 tablet.cc:1724] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.227010  1714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5"
dest_uuid: "3a60ed84fa294767b85c19013457b81b"
mode: GRACEFUL
new_leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
 from {username='slave'} at 127.0.0.1:53964
I20260709 13:11:57.227694  1714 raft_consensus.cc:606] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Received request to transfer leadership to TS d4286d1c49ba4bee8f2f4b7478053898
W20260709 13:11:57.233444  2063 log.cc:927] Time spent T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.006s
I20260709 13:11:57.245829  2072 tablet.cc:1724] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.258088  2076 tablet.cc:1724] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.258781  2062 tablet.cc:1724] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.270702  2082 tablet.cc:1724] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.266542  2081 tablet.cc:1724] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.295783  2067 tablet.cc:1724] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.305912  2081 tablet.cc:1724] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.310915  2068 tablet.cc:1724] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.329566  2081 tablet.cc:1724] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.320925  2076 tablet.cc:1724] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.340337  2073 tablet.cc:1724] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.368947  2078 tablet.cc:1724] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.370581  2075 tablet.cc:1724] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.372465  1640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee5c391761d44bc4b94a7fc2aeb37505"
dest_uuid: "75ec73734cb347918a1298a2baa490e5"
mode: GRACEFUL
new_leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
 from {username='slave'} at 127.0.0.1:33146
I20260709 13:11:57.373075  1640 raft_consensus.cc:606] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Received request to transfer leadership to TS d4286d1c49ba4bee8f2f4b7478053898
I20260709 13:11:57.376364 21629 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:11:57.376952 21629 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:57.377724  2067 tablet.cc:1724] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.380234  2062 tablet.cc:1724] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.337659  2065 tablet.cc:1724] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.386807  2073 tablet.cc:1724] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.393735  2082 tablet.cc:1724] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.361765  2071 tablet.cc:1724] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.415643  2067 tablet.cc:1724] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:11:57.434847  2011 raft_consensus.cc:993] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b: : Instructing follower d4286d1c49ba4bee8f2f4b7478053898 to start an election
I20260709 13:11:57.435293  2011 raft_consensus.cc:1081] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Signalling peer d4286d1c49ba4bee8f2f4b7478053898 to start an election
I20260709 13:11:57.438308  1787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5"
dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
 from {username='slave'} at 127.0.0.1:53296
I20260709 13:11:57.439023  1787 raft_consensus.cc:493] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:57.439939  1787 raft_consensus.cc:3060] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.444428  1541 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5: soft_deleted_table [id=91c336d26c21493dab14d37b15fc1070] alter complete (version 1)
I20260709 13:11:57.447082  1787 raft_consensus.cc:515] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:57.449783  1640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:57.450073  1787 leader_election.cc:290] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 2 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:57.450348  1640 raft_consensus.cc:3060] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.451633  1718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a3f75fbfbaf4606a0d646b57979e7d5" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:57.452152  1718 raft_consensus.cc:3055] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:57.452456  1718 raft_consensus.cc:740] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a60ed84fa294767b85c19013457b81b, State: Running, Role: LEADER
I20260709 13:11:57.453157  1718 consensus_queue.cc:260] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:57.454360  1718 raft_consensus.cc:3060] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.457461  1640 raft_consensus.cc:2468] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 2.
I20260709 13:11:57.458597  1757 leader_election.cc:304] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ec73734cb347918a1298a2baa490e5, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:57.459379  2056 raft_consensus.cc:2804] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:57.459796  2056 raft_consensus.cc:697] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 LEADER]: Becoming Leader. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:57.460500  2056 consensus_queue.cc:237] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:57.460805  1718 raft_consensus.cc:2468] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 2.
I20260709 13:11:57.468824  1542 catalog_manager.cc:5697] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 reported cstate change: term changed from 1 to 2, leader changed from 3a60ed84fa294767b85c19013457b81b (127.21.31.66) to d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:57.820549  2083 raft_consensus.cc:993] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5: : Instructing follower d4286d1c49ba4bee8f2f4b7478053898 to start an election
I20260709 13:11:57.821136  2077 raft_consensus.cc:1081] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Signalling peer d4286d1c49ba4bee8f2f4b7478053898 to start an election
I20260709 13:11:57.822604  1787 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee5c391761d44bc4b94a7fc2aeb37505"
dest_uuid: "d4286d1c49ba4bee8f2f4b7478053898"
 from {username='slave'} at 127.0.0.1:42374
I20260709 13:11:57.822999  1787 raft_consensus.cc:493] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:57.823236  1787 raft_consensus.cc:3060] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.827947  1787 raft_consensus.cc:515] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:57.829905  1787 leader_election.cc:290] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 2 election: Requested vote from peers 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271), 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197)
I20260709 13:11:57.830564  1644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5c391761d44bc4b94a7fc2aeb37505" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75ec73734cb347918a1298a2baa490e5"
I20260709 13:11:57.831058  1716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5c391761d44bc4b94a7fc2aeb37505" candidate_uuid: "d4286d1c49ba4bee8f2f4b7478053898" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a60ed84fa294767b85c19013457b81b"
I20260709 13:11:57.831188  1644 raft_consensus.cc:3055] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:57.831609  1644 raft_consensus.cc:740] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75ec73734cb347918a1298a2baa490e5, State: Running, Role: LEADER
I20260709 13:11:57.831670  1716 raft_consensus.cc:3060] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.832414  1644 consensus_queue.cc:260] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:57.833634  1644 raft_consensus.cc:3060] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.838366  1716 raft_consensus.cc:2468] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 2.
I20260709 13:11:57.839584  1756 leader_election.cc:304] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a60ed84fa294767b85c19013457b81b, d4286d1c49ba4bee8f2f4b7478053898; no voters: 
I20260709 13:11:57.839845  1644 raft_consensus.cc:2468] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4286d1c49ba4bee8f2f4b7478053898 in term 2.
I20260709 13:11:57.840372  2056 raft_consensus.cc:2804] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:57.840863  2056 raft_consensus.cc:697] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 LEADER]: Becoming Leader. State: Replica: d4286d1c49ba4bee8f2f4b7478053898, State: Running, Role: LEADER
I20260709 13:11:57.841666  2056 consensus_queue.cc:237] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } }
I20260709 13:11:57.849560  1542 catalog_manager.cc:5697] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 reported cstate change: term changed from 1 to 2, leader changed from 75ec73734cb347918a1298a2baa490e5 (127.21.31.65) to d4286d1c49ba4bee8f2f4b7478053898 (127.21.31.67). New cstate: current_term: 2 leader_uuid: "d4286d1c49ba4bee8f2f4b7478053898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4286d1c49ba4bee8f2f4b7478053898" member_type: VOTER last_known_addr { host: "127.21.31.67" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:57.895889  1716 raft_consensus.cc:1275] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Refusing update from remote peer d4286d1c49ba4bee8f2f4b7478053898: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.897076  2056 consensus_queue.cc:1048] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.908100  1644 raft_consensus.cc:1275] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Refusing update from remote peer d4286d1c49ba4bee8f2f4b7478053898: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.910394  2056 consensus_queue.cc:1048] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:58.339893  1640 raft_consensus.cc:1275] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Refusing update from remote peer d4286d1c49ba4bee8f2f4b7478053898: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:58.341130  2056 consensus_queue.cc:1048] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ec73734cb347918a1298a2baa490e5" member_type: VOTER last_known_addr { host: "127.21.31.65" port: 35271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:58.349941  1716 raft_consensus.cc:1275] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Refusing update from remote peer d4286d1c49ba4bee8f2f4b7478053898: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:58.351998  2096 consensus_queue.cc:1048] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a60ed84fa294767b85c19013457b81b" member_type: VOTER last_known_addr { host: "127.21.31.66" port: 42197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:58.434940 21629 tablet_server.cc:179] TabletServer@127.21.31.65:0 shutting down...
W20260709 13:11:58.465507  1683 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 10 similar messages]
W20260709 13:11:58.469786  1683 consensus_peers.cc:597] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b -> Peer 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271): Couldn't send request to peer 75ec73734cb347918a1298a2baa490e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:58.471242 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:58.471999 21629 tablet_replica.cc:333] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.472582 21629 raft_consensus.cc:2243] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.473055 21629 raft_consensus.cc:2272] T 45048a76026e49149b143ba4eed71501 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.475169 21629 tablet_replica.cc:333] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.475751 21629 raft_consensus.cc:2243] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.476459 21629 raft_consensus.cc:2272] T bf793f30e6ae497799b0f68f771e840b P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.478302 21629 tablet_replica.cc:333] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.478843 21629 raft_consensus.cc:2243] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.479266 21629 raft_consensus.cc:2272] T 33f8e265000542f3830defbaca47151f P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.480886 21629 tablet_replica.cc:333] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.481453 21629 raft_consensus.cc:2243] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.482187 21629 raft_consensus.cc:2272] T d7510352befc4c2f9feceab054b5443c P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.484056 21629 tablet_replica.cc:333] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.484575 21629 raft_consensus.cc:2243] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.484992 21629 raft_consensus.cc:2272] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.486718 21629 tablet_replica.cc:333] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.487289 21629 raft_consensus.cc:2243] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.488020 21629 raft_consensus.cc:2272] T a5d20f7524334d9ebaee08b4b4a64b49 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.489989 21629 tablet_replica.cc:333] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.490510 21629 raft_consensus.cc:2243] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.491585 21629 raft_consensus.cc:2272] T f853b6de82ff4282b7626c1009031ed3 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.494190 21629 tablet_replica.cc:333] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.494906 21629 raft_consensus.cc:2243] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.495505 21629 raft_consensus.cc:2272] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.497905 21629 tablet_replica.cc:333] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.498723 21629 raft_consensus.cc:2243] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.499658 21629 raft_consensus.cc:2272] T 8ba207fc5f4944e4ad08e006f58a49eb P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.501760 21629 tablet_replica.cc:333] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.502336 21629 raft_consensus.cc:2243] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.502782 21629 raft_consensus.cc:2272] T 66ff68d2564e4b2eab19e1e61bf4c207 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.504878 21629 tablet_replica.cc:333] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.505456 21629 raft_consensus.cc:2243] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.505903 21629 raft_consensus.cc:2272] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.507990 21629 tablet_replica.cc:333] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.508530 21629 raft_consensus.cc:2243] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.509382 21629 raft_consensus.cc:2272] T c125d13005a14b7e922c42382b90747a P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.511405 21629 tablet_replica.cc:333] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.512007 21629 raft_consensus.cc:2243] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.512457 21629 raft_consensus.cc:2272] T 60a5949a56134e88802dd55f641521e8 P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.514585 21629 tablet_replica.cc:333] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.515182 21629 raft_consensus.cc:2243] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.515599 21629 raft_consensus.cc:2272] T ee5c391761d44bc4b94a7fc2aeb37505 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.518972 21629 tablet_replica.cc:333] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.519610 21629 raft_consensus.cc:2243] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.520251 21629 raft_consensus.cc:2272] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.523200 21629 tablet_replica.cc:333] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.523736 21629 raft_consensus.cc:2243] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.524530 21629 raft_consensus.cc:2272] T ba63fe49e7ae44fca02e50d33499027c P 75ec73734cb347918a1298a2baa490e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.526589 21629 tablet_replica.cc:333] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.527083 21629 raft_consensus.cc:2243] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.527508 21629 raft_consensus.cc:2272] T 763c80373e664640b2ccb1aba6ecd425 P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.529419 21629 tablet_replica.cc:333] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5: stopping tablet replica
I20260709 13:11:58.529902 21629 raft_consensus.cc:2243] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.530318 21629 raft_consensus.cc:2272] T f1cad82a28d94f7aae10cc833bbcb6fe P 75ec73734cb347918a1298a2baa490e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.557564 21629 tablet_server.cc:196] TabletServer@127.21.31.65:0 shutdown complete.
I20260709 13:11:58.592298 21629 tablet_server.cc:179] TabletServer@127.21.31.66:0 shutting down...
W20260709 13:11:58.624337  1756 consensus_peers.cc:597] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 -> Peer 3a60ed84fa294767b85c19013457b81b (127.21.31.66:42197): Couldn't send request to peer 3a60ed84fa294767b85c19013457b81b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:58.625638 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:58.626673 21629 tablet_replica.cc:333] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.627584 21629 raft_consensus.cc:2243] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.628168 21629 raft_consensus.cc:2272] T bf793f30e6ae497799b0f68f771e840b P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:58.629849  1683 consensus_peers.cc:597] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b -> Peer 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271): Couldn't send request to peer 75ec73734cb347918a1298a2baa490e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:35271: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:58.630554 21629 tablet_replica.cc:333] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.631201 21629 raft_consensus.cc:2243] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.631645 21629 raft_consensus.cc:2272] T 45048a76026e49149b143ba4eed71501 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.633479 21629 tablet_replica.cc:333] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.634018 21629 raft_consensus.cc:2243] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.634735 21629 raft_consensus.cc:2272] T 33f8e265000542f3830defbaca47151f P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.636407 21629 tablet_replica.cc:333] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.636945 21629 raft_consensus.cc:2243] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.637459 21629 raft_consensus.cc:2272] T d7510352befc4c2f9feceab054b5443c P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.639364 21629 tablet_replica.cc:333] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.639885 21629 raft_consensus.cc:2243] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.640306 21629 raft_consensus.cc:2272] T f853b6de82ff4282b7626c1009031ed3 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.642055 21629 tablet_replica.cc:333] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.642590 21629 raft_consensus.cc:2243] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.643333 21629 raft_consensus.cc:2272] T 7d7e38e6d7aa41cab3272ec7143f3a6b P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.645211 21629 tablet_replica.cc:333] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.646133 21629 raft_consensus.cc:2243] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.646714 21629 raft_consensus.cc:2272] T 8ba207fc5f4944e4ad08e006f58a49eb P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.648545 21629 tablet_replica.cc:333] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.649154 21629 raft_consensus.cc:2243] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.649958 21629 raft_consensus.cc:2272] T 60a5949a56134e88802dd55f641521e8 P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.652123 21629 tablet_replica.cc:333] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.652700 21629 raft_consensus.cc:2243] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.653092 21629 raft_consensus.cc:2272] T ee5c391761d44bc4b94a7fc2aeb37505 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.656126 21629 tablet_replica.cc:333] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.656891 21629 raft_consensus.cc:2243] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.657440 21629 raft_consensus.cc:2272] T c125d13005a14b7e922c42382b90747a P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.659363 21629 tablet_replica.cc:333] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.659930 21629 raft_consensus.cc:2243] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.660847 21629 raft_consensus.cc:2272] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.662760 21629 tablet_replica.cc:333] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.663321 21629 raft_consensus.cc:2243] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.663789 21629 raft_consensus.cc:2272] T a5d20f7524334d9ebaee08b4b4a64b49 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.665831 21629 tablet_replica.cc:333] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.666313 21629 raft_consensus.cc:2243] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.666736 21629 raft_consensus.cc:2272] T 8a3f75fbfbaf4606a0d646b57979e7d5 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.669904 21629 tablet_replica.cc:333] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.670469 21629 raft_consensus.cc:2243] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.671224 21629 raft_consensus.cc:2272] T f1cad82a28d94f7aae10cc833bbcb6fe P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.673199 21629 tablet_replica.cc:333] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.673815 21629 raft_consensus.cc:2243] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.674671 21629 raft_consensus.cc:2272] T 763c80373e664640b2ccb1aba6ecd425 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.677039 21629 tablet_replica.cc:333] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.677806 21629 raft_consensus.cc:2243] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.679061 21629 raft_consensus.cc:2272] T 5ad46f2fe7044ff390bfdeb77c22fed5 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.681229 21629 tablet_replica.cc:333] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.681857 21629 raft_consensus.cc:2243] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.682442 21629 raft_consensus.cc:2272] T 66ff68d2564e4b2eab19e1e61bf4c207 P 3a60ed84fa294767b85c19013457b81b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.684732 21629 tablet_replica.cc:333] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b: stopping tablet replica
I20260709 13:11:58.685223 21629 raft_consensus.cc:2243] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.685686 21629 raft_consensus.cc:2272] T ba63fe49e7ae44fca02e50d33499027c P 3a60ed84fa294767b85c19013457b81b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.725226 21629 tablet_server.cc:196] TabletServer@127.21.31.66:0 shutdown complete.
I20260709 13:11:58.756641 21629 tablet_server.cc:179] TabletServer@127.21.31.67:0 shutting down...
I20260709 13:11:58.786099 21629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:58.786891 21629 tablet_replica.cc:333] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.787492 21629 raft_consensus.cc:2243] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.788295 21629 raft_consensus.cc:2272] T 45048a76026e49149b143ba4eed71501 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.790197 21629 tablet_replica.cc:333] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.790755 21629 raft_consensus.cc:2243] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.791183 21629 raft_consensus.cc:2272] T 8ba207fc5f4944e4ad08e006f58a49eb P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.792836 21629 tablet_replica.cc:333] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.793424 21629 raft_consensus.cc:2243] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.793833 21629 raft_consensus.cc:2272] T f853b6de82ff4282b7626c1009031ed3 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.795471 21629 tablet_replica.cc:333] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.795994 21629 raft_consensus.cc:2243] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.796402 21629 raft_consensus.cc:2272] T 7d7e38e6d7aa41cab3272ec7143f3a6b P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.798345 21629 tablet_replica.cc:333] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.799024 21629 raft_consensus.cc:2243] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.799451 21629 raft_consensus.cc:2272] T 33f8e265000542f3830defbaca47151f P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.801187 21629 tablet_replica.cc:333] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.801740 21629 raft_consensus.cc:2243] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.802234 21629 raft_consensus.cc:2272] T bf793f30e6ae497799b0f68f771e840b P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:58.803406  1757 consensus_peers.cc:597] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 -> Peer 75ec73734cb347918a1298a2baa490e5 (127.21.31.65:35271): Couldn't send request to peer 75ec73734cb347918a1298a2baa490e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.31.65:35271: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:58.804266 21629 tablet_replica.cc:333] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.804798 21629 raft_consensus.cc:2243] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.805506 21629 raft_consensus.cc:2272] T 8a3f75fbfbaf4606a0d646b57979e7d5 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.807778 21629 tablet_replica.cc:333] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.808269 21629 raft_consensus.cc:2243] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.808611 21629 raft_consensus.cc:2272] T 763c80373e664640b2ccb1aba6ecd425 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.810359 21629 tablet_replica.cc:333] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.810806 21629 raft_consensus.cc:2243] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.811169 21629 raft_consensus.cc:2272] T f1cad82a28d94f7aae10cc833bbcb6fe P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.812951 21629 tablet_replica.cc:333] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.813520 21629 raft_consensus.cc:2243] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.813925 21629 raft_consensus.cc:2272] T 5ad46f2fe7044ff390bfdeb77c22fed5 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.815654 21629 tablet_replica.cc:333] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.816480 21629 raft_consensus.cc:2243] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.817240 21629 raft_consensus.cc:2272] T 66ff68d2564e4b2eab19e1e61bf4c207 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.819657 21629 tablet_replica.cc:333] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.820158 21629 raft_consensus.cc:2243] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.820540 21629 raft_consensus.cc:2272] T ba63fe49e7ae44fca02e50d33499027c P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.822450 21629 tablet_replica.cc:333] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.822964 21629 raft_consensus.cc:2243] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.823696 21629 raft_consensus.cc:2272] T ee5c391761d44bc4b94a7fc2aeb37505 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.825914 21629 tablet_replica.cc:333] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.826396 21629 raft_consensus.cc:2243] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.826795 21629 raft_consensus.cc:2272] T c125d13005a14b7e922c42382b90747a P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.828395 21629 tablet_replica.cc:333] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.828884 21629 raft_consensus.cc:2243] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.829273 21629 raft_consensus.cc:2272] T 60a5949a56134e88802dd55f641521e8 P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.831164 21629 tablet_replica.cc:333] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.831670 21629 raft_consensus.cc:2243] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.832051 21629 raft_consensus.cc:2272] T d7510352befc4c2f9feceab054b5443c P d4286d1c49ba4bee8f2f4b7478053898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.833943 21629 tablet_replica.cc:333] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.834396 21629 raft_consensus.cc:2243] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.834790 21629 raft_consensus.cc:2272] T 5e5087882aa34d15b0c6c1c81d8e6fc0 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.836505 21629 tablet_replica.cc:333] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898: stopping tablet replica
I20260709 13:11:58.836917 21629 raft_consensus.cc:2243] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:58.837375 21629 raft_consensus.cc:2272] T a5d20f7524334d9ebaee08b4b4a64b49 P d4286d1c49ba4bee8f2f4b7478053898 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.874734 21629 tablet_server.cc:196] TabletServer@127.21.31.67:0 shutdown complete.
I20260709 13:11:58.901975 21629 master.cc:562] Master@127.21.31.126:34049 shutting down...
I20260709 13:11:58.929824 21629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:58.930369 21629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:58.930687 21629 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7f318e82dda5499dace937e8c351b9d5: stopping tablet replica
I20260709 13:11:58.950466 21629 master.cc:584] Master@127.21.31.126:34049 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11364 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (24149 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (359696 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:11:59.006985 21629 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 403 messages since previous log ~14 seconds ago
I20260709 13:11:59.007184 21629 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 403 messages since previous log ~14 seconds ago
I20260709 13:11:59.007328 21629 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 403 messages since previous log ~14 seconds ago
I20260709 13:11:59.007470 21629 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~32 seconds ago
I20260709 13:11:59.007611 21629 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~32 seconds ago
I20260709 13:11:59.007769 21629 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 4 messages since previous log ~133 seconds ago
I20260709 13:11:59.007952 21629 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 3 messages since previous log ~0 seconds ago
I20260709 13:11:59.008191 21629 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 26 messages since previous log ~6 seconds ago
